May 26 23:41:09.600694 ip-10-0-136-129 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 26 23:41:09.600706 ip-10-0-136-129 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 26 23:41:09.600713 ip-10-0-136-129 systemd[1]: kubelet.service: Failed with result 'resources'.
May 26 23:41:09.600975 ip-10-0-136-129 systemd[1]: Failed to start Kubernetes Kubelet.
May 26 23:41:19.719988 ip-10-0-136-129 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
May 26 23:41:19.720008 ip-10-0-136-129 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 6104476d2c5741d0b3660fa4a7de0dde --
May 26 23:43:55.695282 ip-10-0-136-129 systemd[1]: Starting Kubernetes Kubelet...
May 26 23:43:56.175673 ip-10-0-136-129 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 26 23:43:56.175673 ip-10-0-136-129 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 26 23:43:56.175673 ip-10-0-136-129 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 26 23:43:56.175673 ip-10-0-136-129 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 26 23:43:56.175673 ip-10-0-136-129 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 26 23:43:56.179474 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.179385    2566 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 26 23:43:56.182976 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.182961    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 26 23:43:56.182976 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.182976    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.182979    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.182990    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.182994    2566 feature_gate.go:328] unrecognized feature gate: Example2
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.182997    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183001    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183004    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183006    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183009    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183012    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183014    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183017    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183019    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183022    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183025    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183028    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183030    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183033    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183036    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183038    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 26 23:43:56.183037 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183042    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183044    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183047    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183050    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183053    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183055    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183058    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183060    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183063    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183066    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183068    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183071    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183073    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183076    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183079    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183082    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183085    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183088    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183090    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183093    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 26 23:43:56.183518 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183095    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183098    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183100    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183102    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183105    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183107    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183110    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183114    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183118    2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183121    2566 feature_gate.go:328] unrecognized feature gate: Example
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183124    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183127    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183130    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183133    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183136    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183140    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183142    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183145    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183147    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183150    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 26 23:43:56.184034 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183152    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183156    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183159    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183162    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183165    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183168    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183171    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183176    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183178    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183181    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183183    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183186    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183188    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183191    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183193    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183196    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183198    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183201    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183204    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 26 23:43:56.184517 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183206    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183209    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183211    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183213    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183216    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183218    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183597    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183604    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183607    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183609    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183612    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183615    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183617    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183620    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183623    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183626    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183628    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183633    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 26 23:43:56.184982 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183636    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183639    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183642    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183644    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183647    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183649    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183652    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183654    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183657    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183659    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183662    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183664    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183667    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183669    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183672    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183674    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183677    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183679    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183682    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183685    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 26 23:43:56.185413 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183688    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183691    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183693    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183696    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183698    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183702    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183704    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183707    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183709    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183712    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183714    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183717    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183719    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183722    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183724    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183727    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183730    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183732    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183734    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183737    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 26 23:43:56.185914 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183739    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183742    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183744    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183747    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183751    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183754    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183757    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183759    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183763    2566 feature_gate.go:328] unrecognized feature gate: Example2
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183766    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183768    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183771    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183773    2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183777    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183780    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183782    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183784    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183787    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183789    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183792    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 26 23:43:56.186403 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183794    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183797    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183799    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183802    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183804    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183806    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183809    2566 feature_gate.go:328] unrecognized feature gate: Example
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183811    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183814    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183817    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183819    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183821    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183824    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.183826    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184595    2566 flags.go:64] FLAG: --address="0.0.0.0"
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184605    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184611    2566 flags.go:64] FLAG: --anonymous-auth="true"
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184616    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184620    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184623    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184627    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184633    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 26 23:43:56.186935 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184636    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184639    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184642    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184646    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184649    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184652    2566 flags.go:64] FLAG: --cgroup-root=""
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184655    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184658    2566 flags.go:64] FLAG: --client-ca-file=""
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184661    2566 flags.go:64] FLAG: --cloud-config=""
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184664    2566 flags.go:64] FLAG: --cloud-provider="external"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184667    2566 flags.go:64] FLAG: --cluster-dns="[]"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184672    2566 flags.go:64] FLAG: --cluster-domain=""
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184674    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184678    2566 flags.go:64] FLAG: --config-dir=""
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184680    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184684    2566 flags.go:64] FLAG: --container-log-max-files="5"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184688    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184691    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184694    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184698    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184701    2566 flags.go:64] FLAG: --contention-profiling="false"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184704    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184707    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184710    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184713    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 26 23:43:56.187478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184717    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184720    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184723    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184726    2566 flags.go:64] FLAG: --enable-load-reader="false"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184729    2566 flags.go:64] FLAG: --enable-server="true"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184732    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184736    2566 flags.go:64] FLAG: --event-burst="100"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184739    2566 flags.go:64] FLAG: --event-qps="50"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184742    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184745    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184748    2566 flags.go:64] FLAG: --eviction-hard=""
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184752    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184755    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184758    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184761    2566 flags.go:64] FLAG: --eviction-soft=""
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184764    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184766    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184769    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184772    2566 flags.go:64] FLAG: --experimental-mounter-path=""
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184775    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184778    2566 flags.go:64] FLAG: --fail-swap-on="true"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184781    2566 flags.go:64] FLAG: --feature-gates=""
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184785    2566 flags.go:64] FLAG: --file-check-frequency="20s"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184788    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184791    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 26 23:43:56.188122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184794    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184797    2566 flags.go:64] FLAG: --healthz-port="10248"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184801    2566 flags.go:64] FLAG: --help="false"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184804    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-136-129.ec2.internal"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184807    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184810    2566 flags.go:64] FLAG: --http-check-frequency="20s"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184813    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184816    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184820    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184823    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184826    2566 flags.go:64] FLAG: --image-service-endpoint=""
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184829    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184832    2566 flags.go:64] FLAG: --kube-api-burst="100"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184835    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184838    2566 flags.go:64] FLAG: --kube-api-qps="50"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184841    2566 flags.go:64] FLAG: --kube-reserved=""
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184844    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184847    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184850    2566 flags.go:64] FLAG: --kubelet-cgroups=""
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184853    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184856    2566 flags.go:64] FLAG: --lock-file=""
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184859    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184862    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184865    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 26 23:43:56.188722 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184870    2566 flags.go:64] FLAG: --log-json-split-stream="false"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184873    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184876    2566 flags.go:64] FLAG: --log-text-split-stream="false"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184879    2566 flags.go:64] FLAG: --logging-format="text"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184881    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184885    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184888    2566 flags.go:64] FLAG: --manifest-url=""
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184890    2566 flags.go:64] FLAG: --manifest-url-header=""
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184908    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184911    2566 flags.go:64] FLAG: --max-open-files="1000000"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184915    2566 flags.go:64] FLAG: --max-pods="110"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184919    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184922    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184925    2566 flags.go:64] FLAG: --memory-manager-policy="None"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184928    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184930    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184934    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184937    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184944    2566 flags.go:64] FLAG: --node-status-max-images="50"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184947    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184950    2566 flags.go:64] FLAG: --oom-score-adj="-999"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184956    2566 flags.go:64] FLAG: --pod-cidr=""
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184959    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 26 23:43:56.189343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184965    2566 flags.go:64] FLAG: --pod-manifest-path=""
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184968    2566 flags.go:64] FLAG: --pod-max-pids="-1"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184971    2566 flags.go:64] FLAG: --pods-per-core="0"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184974    2566 flags.go:64] FLAG: --port="10250"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184977    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184980    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04c4b662320355338"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184983    2566 flags.go:64] FLAG: --qos-reserved=""
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184986    2566 flags.go:64] FLAG: --read-only-port="10255"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184989    2566 flags.go:64] FLAG: --register-node="true"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184992    2566 flags.go:64] FLAG: --register-schedulable="true"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184995    2566 flags.go:64] FLAG: --register-with-taints=""
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.184999    2566 flags.go:64] FLAG: --registry-burst="10"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185001    2566 flags.go:64] FLAG: --registry-qps="5"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185004    2566 flags.go:64] FLAG: --reserved-cpus=""
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185007    2566 flags.go:64] FLAG: --reserved-memory=""
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185011    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185013    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185017    2566 flags.go:64] FLAG: --rotate-certificates="false"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185020    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185022    2566 flags.go:64] FLAG: --runonce="false"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185025    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185029    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185031    2566 flags.go:64] FLAG: --seccomp-default="false"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185034    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185037    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185040    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 26 23:43:56.189873 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185043    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185047    2566 flags.go:64] FLAG: --storage-driver-password="root"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185049    2566 flags.go:64] FLAG: --storage-driver-secure="false"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185052    2566 flags.go:64] FLAG: --storage-driver-table="stats"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185055    2566 flags.go:64] FLAG: --storage-driver-user="root"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185060    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185063    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185066    2566 flags.go:64] FLAG: --system-cgroups=""
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185069    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185074    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185077    2566 flags.go:64] FLAG: --tls-cert-file=""
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185079    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185088    2566 flags.go:64] FLAG: --tls-min-version=""
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185091    2566 flags.go:64] FLAG: --tls-private-key-file=""
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185093    2566 flags.go:64] FLAG: --topology-manager-policy="none"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185096    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185099    2566 flags.go:64] FLAG: --topology-manager-scope="container"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185102    2566 flags.go:64] FLAG: --v="2"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185106    2566 flags.go:64] FLAG: --version="false"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185110    2566 flags.go:64] FLAG: --vmodule=""
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185115    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.185118    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185211    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185215    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 26 23:43:56.190533 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185218    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185221    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185223    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185226    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185229    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185232    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185234    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185237    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185240    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185243    2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185246    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185248    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185252    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185256    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185259    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185261    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185264    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185266    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185269    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185271    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 26 23:43:56.191127 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185274    2566 feature_gate.go:328] unrecognized feature gate: Example2
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185277    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185279    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185282    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185284    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185287    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185289    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185292    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185295    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185297    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185300    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185302    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185305    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185307    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185310    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185312    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185315    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185319    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185321    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 26 23:43:56.191654 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185324    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185326    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185329    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185331    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185334    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185338    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185342    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185345    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185347    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185349    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185352    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185356    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185359    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185362    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185365    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185367    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185370    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185372    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185375    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 26 23:43:56.192466 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185377    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185380    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185382    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185385    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185387    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185390    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185392    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185395    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185397    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185400    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185402    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185404    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185408    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185411    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185413    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185417    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185421    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185424    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185428    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 26 23:43:56.193254 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185432    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 26 23:43:56.194011 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185435    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 26 23:43:56.194011 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185438    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 26 23:43:56.194011 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185441    2566 feature_gate.go:328] unrecognized feature gate: Example
May 26 23:43:56.194011 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185443    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 26 23:43:56.194011 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185446    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 26 23:43:56.194011 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.185449    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 26 23:43:56.194011 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.186035    2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 26 23:43:56.194484 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.194465    2566 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 26 23:43:56.194538 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.194489    2566 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 26 23:43:56.194588 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194566    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 26 23:43:56.194588 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194575    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 26 23:43:56.194588 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194580    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 26 23:43:56.194588 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194585    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194591    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194595    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194599    2566 feature_gate.go:328] unrecognized feature gate: Example
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194604    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194609    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194614    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194619    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194624    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194628    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194632    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194637    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194641    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194645    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194649    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194654    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194661    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194668    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194673    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 26 23:43:56.194768 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194686    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194691    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194695    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194699    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194704    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194708    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194713    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194717    2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194722    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194725    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194730    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194734    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194738    2566 feature_gate.go:328] unrecognized feature gate: Example2
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194742    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194747    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194752    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194756    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194761    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194766    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194770    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 26 23:43:56.195618 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194774    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194778    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194783    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194788    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194792    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194796    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194801    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194805    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194809    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194813    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194817    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194821    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194826    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194830    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194835    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194839    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194843    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194847    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194851    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194856    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 26 23:43:56.196178 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194860    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194864    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194868    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194872    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194879    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194886    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194891    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194916    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194921    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194925    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194930    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194934    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194938    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194942    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194947    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194952    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194956    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194960    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194964    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 26 23:43:56.196733 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194968    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194972    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194976    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194980    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.194984    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.194992    2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195170    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195178    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195183    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195188    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195192    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195196    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195200    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195204    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195208    2566 feature_gate.go:328] unrecognized feature gate: Example2
May 26 23:43:56.197443 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195213    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195217    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195221    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195225    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195229    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195234    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195238    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195242    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195247    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195251    2566 feature_gate.go:328] unrecognized feature gate: Example
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195255    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195259    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195263    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195268    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195273    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195277    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195281    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195286    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195290    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195294    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 26 23:43:56.197853 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195299    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195303    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195307    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195311    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195315    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195319    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195323    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195327    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195332    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195336    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195340    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195344    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195350    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195355    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195359    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195363    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195367    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195371    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195376    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 26 23:43:56.198440 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195380    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195384    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195388    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195392    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195397    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195401    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195405    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195410    2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195414    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195418    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195423    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195427    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195439    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195443    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195447    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195451    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195455    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195459    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195463    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195467    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 26 23:43:56.199234 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195471    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195475    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195479    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195484    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195488    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195492    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195497    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195500    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195507    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195513    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195518    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195522    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195526    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195531    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195535    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195539    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195543    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 26 23:43:56.199783 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:56.195547    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 26 23:43:56.200423 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.195554    2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 26 23:43:56.200423 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.196427    2566 server.go:962] "Client rotation is on, will bootstrap in background"
May 26 23:43:56.200423 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.199931    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 26 23:43:56.200814 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.200801    2566 server.go:1019] "Starting client certificate rotation"
May 26 23:43:56.200930 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.200913    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 26 23:43:56.200964 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.200955    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 26 23:43:56.226490 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.226466    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 26 23:43:56.228503 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.228478    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 26 23:43:56.247111 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.247092    2566 log.go:25] "Validated CRI v1 runtime API"
May 26 23:43:56.253144 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.253127    2566 log.go:25] "Validated CRI v1 image API"
May 26 23:43:56.254505 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.254489    2566 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 26 23:43:56.257067 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.257043    2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 92eaf41c-8279-40fb-85db-979635536b15:/dev/nvme0n1p3 f256921f-1550-45c7-8480-4dba83286b4e:/dev/nvme0n1p4]
May 26 23:43:56.257155 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.257066    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 26 23:43:56.259208 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.259188    2566 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 26 23:43:56.261305 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.261192    2566 manager.go:217] Machine: {Timestamp:2026-05-26 23:43:56.260666963 +0000 UTC m=+0.436698115 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3086350 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec253c8bcce689ca51eed9f0db4921ae SystemUUID:ec253c8b-cce6-89ca-51ee-d9f0db4921ae BootID:6104476d-2c57-41d0-b366-0fa4a7de0dde Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2e:ca:5c:8b:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2e:ca:5c:8b:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:2e:fb:cf:ac:7e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 26 23:43:56.261414 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.261338    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 26 23:43:56.261470 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.261441    2566 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 26 23:43:56.264424 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.264395    2566 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 26 23:43:56.264589 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.264427    2566 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-129.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 26 23:43:56.264669 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.264604    2566 topology_manager.go:138] "Creating topology manager with none policy"
May 26 23:43:56.264669 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.264617    2566 container_manager_linux.go:306] "Creating device plugin manager"
May 26 23:43:56.264669 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.264627    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 26 23:43:56.264669 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.264642    2566 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 26 23:43:56.266021 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.265999    2566 state_mem.go:36] "Initialized new in-memory state store"
May 26 23:43:56.266335 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.266321    2566 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 26 23:43:56.269218 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.269207    2566 kubelet.go:491] "Attempting to sync node with API server"
May 26 23:43:56.269275 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.269225    2566 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 26 23:43:56.269275 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.269241    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 26 23:43:56.269275 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.269254    2566 kubelet.go:397] "Adding apiserver pod source"
May 26 23:43:56.269275 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.269265    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 26 23:43:56.270478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.270465    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 26 23:43:56.270543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.270488    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 26 23:43:56.275914 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.275878    2566 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 26 23:43:56.277803 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.277789    2566 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 26 23:43:56.279267 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.279248    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wj87t"
May 26 23:43:56.279683 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.279667    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 26 23:43:56.279717 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.279695    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 26 23:43:56.279717 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.279706    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 26 23:43:56.279717 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.279715    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 26 23:43:56.279795 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.279728    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 26 23:43:56.279795 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.279738    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 26 23:43:56.279795 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.279746    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 26 23:43:56.279795 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.279755    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 26 23:43:56.279795 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.279765    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 26 23:43:56.279795 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.279774    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 26 23:43:56.279965 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.279800    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 26 23:43:56.279965 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.279809    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 26 23:43:56.281460 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.281448    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 26 23:43:56.281496 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.281462    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 26 23:43:56.284484 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.284465    2566 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-129.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 26 23:43:56.284563 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.284498    2566 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-129.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 26 23:43:56.284615 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.284574    2566 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 26 23:43:56.285005 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.284992    2566 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 26 23:43:56.285051 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.285033    2566 server.go:1295] "Started kubelet"
May 26 23:43:56.285111 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.285087    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 26 23:43:56.285179 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.285128    2566 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 26 23:43:56.285224 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.285201    2566 server_v1.go:47] "podresources" method="list" useActivePods=true
May 26 23:43:56.285881 ip-10-0-136-129 systemd[1]: Started Kubernetes Kubelet.
May 26 23:43:56.286486 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.286361    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wj87t"
May 26 23:43:56.286486 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.286445    2566 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 26 23:43:56.287298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.287286    2566 server.go:317] "Adding debug handlers to kubelet server"
May 26 23:43:56.291935 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.291919    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 26 23:43:56.292482 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.292465    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 26 23:43:56.292577 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.292562    2566 volume_manager.go:295] "The desired_state_of_world populator starts"
May 26 23:43:56.292577 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.292578    2566 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 26 23:43:56.294563 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.294538    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-129.ec2.internal\" not found"
May 26 23:43:56.294637 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.294622    2566 reconstruct.go:97] "Volume reconstruction finished"
May 26 23:43:56.294637 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.294630    2566 reconciler.go:26] "Reconciler: start to sync state"
May 26 23:43:56.294816 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.294802    2566 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 26 23:43:56.295889 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.295868    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 26 23:43:56.295979 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.295891    2566 factory.go:55] Registering systemd factory
May 26 23:43:56.295979 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.295923    2566 factory.go:223] Registration of the systemd container factory successfully
May 26 23:43:56.296162 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.296147    2566 factory.go:153] Registering CRI-O factory
May 26 23:43:56.296209 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.296167    2566 factory.go:223] Registration of the crio container factory successfully
May 26 23:43:56.296257 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.296217    2566 factory.go:103] Registering Raw factory
May 26 23:43:56.296257 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.296234    2566 manager.go:1196] Started watching for new ooms in manager
May 26 23:43:56.296346 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.296301    2566 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 26 23:43:56.297305 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.297284    2566 manager.go:319] Starting recovery of all containers
May 26 23:43:56.300106 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.300077    2566 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 26 23:43:56.300732 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.300712    2566 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-129.ec2.internal\" not found" node="ip-10-0-136-129.ec2.internal"
May 26 23:43:56.308730 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.308583    2566 manager.go:324] Recovery completed
May 26 23:43:56.310225 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.310204    2566 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
May 26 23:43:56.313328 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.313317    2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 26 23:43:56.315212 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.315196    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasSufficientMemory"
May 26 23:43:56.315294 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.315229    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasNoDiskPressure"
May 26 23:43:56.315294 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.315243    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasSufficientPID"
May 26 23:43:56.315769 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.315755    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 26 23:43:56.315769 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.315766    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 26 23:43:56.315880 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.315782    2566 state_mem.go:36] "Initialized new in-memory state store"
May 26 23:43:56.318011 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.318000    2566 policy_none.go:49] "None policy: Start"
May 26 23:43:56.318060 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.318015    2566 memory_manager.go:186] "Starting memorymanager" policy="None"
May 26 23:43:56.318060 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.318025    2566 state_mem.go:35] "Initializing new in-memory state store"
May 26 23:43:56.357505 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.357488    2566 manager.go:341] "Starting Device Plugin manager"
May 26 23:43:56.361922 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.357532    2566 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 26 23:43:56.361922 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.357545    2566 server.go:85] "Starting device plugin registration server"
May 26 23:43:56.361922 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.357764    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
May 26 23:43:56.361922 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.357778    2566 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 26 23:43:56.361922 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.357847    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 26 23:43:56.361922 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.357966    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 26 23:43:56.361922 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.357974    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 26 23:43:56.361922 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.358576    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 26 23:43:56.361922 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.358612    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-129.ec2.internal\" not found"
May 26 23:43:56.425385 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.425351    2566 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 26 23:43:56.426644 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.426622    2566 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 26 23:43:56.426644 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.426645    2566 status_manager.go:230] "Starting to sync pod status with apiserver"
May 26 23:43:56.426787 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.426660    2566 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 26 23:43:56.426787 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.426668    2566 kubelet.go:2451] "Starting kubelet main sync loop"
May 26 23:43:56.426787 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.426701    2566 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 26 23:43:56.433385 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.433365    2566 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 26 23:43:56.458575 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.458562    2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 26 23:43:56.459594 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.459579    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasSufficientMemory"
May 26 23:43:56.459668 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.459613    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasNoDiskPressure"
May 26 23:43:56.459668 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.459626    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasSufficientPID"
May 26 23:43:56.459668 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.459648    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-129.ec2.internal"
May 26 23:43:56.468954 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.468941    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-129.ec2.internal"
May 26 23:43:56.468996 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.468966    2566 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-129.ec2.internal\": node \"ip-10-0-136-129.ec2.internal\" not found"
May 26 23:43:56.494878 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.494858    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-129.ec2.internal\" not found"
May 26 23:43:56.527028 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.526997    2566 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-129.ec2.internal"]
May 26 23:43:56.527113 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.527072    2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 26 23:43:56.529094 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.529079    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasSufficientMemory"
May 26 23:43:56.529168 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.529112    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasNoDiskPressure"
May 26 23:43:56.529168 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.529141    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasSufficientPID"
May 26 23:43:56.530679 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.530666    2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 26 23:43:56.530870 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.530854    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal"
May 26 23:43:56.530969 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.530886    2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 26 23:43:56.531381 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.531366    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasSufficientMemory"
May 26 23:43:56.531476 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.531372    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasSufficientMemory"
May 26 23:43:56.531476 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.531438    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasNoDiskPressure"
May 26 23:43:56.531476 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.531400    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasNoDiskPressure"
May 26 23:43:56.531476 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.531455    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasSufficientPID"
May 26 23:43:56.531476 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.531462    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasSufficientPID"
May 26 23:43:56.532705 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.532689    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-129.ec2.internal"
May 26 23:43:56.532784 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.532718    2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 26 23:43:56.533388 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.533372    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasSufficientMemory"
May 26 23:43:56.533466 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.533398    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasNoDiskPressure"
May 26 23:43:56.533466 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.533410    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeHasSufficientPID"
May 26 23:43:56.557714 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.557682    2566 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-129.ec2.internal\" not found" node="ip-10-0-136-129.ec2.internal"
May 26 23:43:56.562000 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.561983    2566 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-129.ec2.internal\" not found" node="ip-10-0-136-129.ec2.internal"
May 26 23:43:56.595673 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.595648    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-129.ec2.internal\" not found"
May 26 23:43:56.596824 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.596805    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b5e5513b500b482a07e342702239169c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal\" (UID: \"b5e5513b500b482a07e342702239169c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal"
May 26 23:43:56.596879 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.596834    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5e5513b500b482a07e342702239169c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal\" (UID: \"b5e5513b500b482a07e342702239169c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal"
May 26 23:43:56.596879 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.596852    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bebf15267ab3a3a2579421856274a9a6-config\") pod \"kube-apiserver-proxy-ip-10-0-136-129.ec2.internal\" (UID: \"bebf15267ab3a3a2579421856274a9a6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-129.ec2.internal"
May 26 23:43:56.696424 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.696358    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-129.ec2.internal\" not found"
May 26 23:43:56.697546 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.697525    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b5e5513b500b482a07e342702239169c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal\" (UID: \"b5e5513b500b482a07e342702239169c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal"
May 26 23:43:56.697619 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.697570    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5e5513b500b482a07e342702239169c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal\" (UID: \"b5e5513b500b482a07e342702239169c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal"
May 26 23:43:56.697619 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.697600    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bebf15267ab3a3a2579421856274a9a6-config\") pod \"kube-apiserver-proxy-ip-10-0-136-129.ec2.internal\" (UID: \"bebf15267ab3a3a2579421856274a9a6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-129.ec2.internal"
May 26 23:43:56.697710 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.697613    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b5e5513b500b482a07e342702239169c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal\" (UID: \"b5e5513b500b482a07e342702239169c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal"
May 26 23:43:56.697710 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.697649    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bebf15267ab3a3a2579421856274a9a6-config\") pod \"kube-apiserver-proxy-ip-10-0-136-129.ec2.internal\" (UID: \"bebf15267ab3a3a2579421856274a9a6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-129.ec2.internal"
May 26 23:43:56.697710 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.697677    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5e5513b500b482a07e342702239169c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal\" (UID: \"b5e5513b500b482a07e342702239169c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal"
May 26 23:43:56.797044 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.797009    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-129.ec2.internal\" not found"
May 26 23:43:56.860275 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.860252    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal"
May 26 23:43:56.864955 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:56.864939    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-129.ec2.internal"
May 26 23:43:56.897672 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.897646    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-129.ec2.internal\" not found"
May 26 23:43:56.998152 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:56.998072    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-129.ec2.internal\" not found"
May 26 23:43:57.098509 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.098474    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-129.ec2.internal\" not found"
May 26 23:43:57.198905 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.198879    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-129.ec2.internal\" not found"
May 26 23:43:57.201022 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.201011    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 26 23:43:57.201137 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.201123    2566 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 26 23:43:57.201198 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.201171    2566 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 26 23:43:57.226790 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.226767    2566 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 26 23:43:57.269876 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.269831    2566 apiserver.go:52] "Watching apiserver"
May 26 23:43:57.279605 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.279579    2566 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 26 23:43:57.279889 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.279869    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-4r7ws","openshift-multus/multus-additional-cni-plugins-vjq8g","openshift-multus/network-metrics-daemon-6mdld","openshift-network-diagnostics/network-check-target-fplbc","kube-system/konnectivity-agent-ml74s","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb","openshift-dns/node-resolver-2bg9t","openshift-image-registry/node-ca-vjw25","openshift-multus/multus-pmsgm","openshift-network-operator/iptables-alerter-z68mm","openshift-ovn-kubernetes/ovnkube-node-nh9zh"]
May 26 23:43:57.281857 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.281828    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.283113 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.283093    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.283263 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.283248    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:43:57.283366 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.283336    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:43:57.284043 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.284027    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jswg6\""
May 26 23:43:57.284043 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.284035    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 26 23:43:57.284163 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.284032    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 26 23:43:57.284908 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.284873    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:43:57.285019 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.284986    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:43:57.285275 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.285254    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 26 23:43:57.285356 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.285323    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-d4pv6\""
May 26 23:43:57.285440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.285425    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 26 23:43:57.285597 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.285579    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 26 23:43:57.285637 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.285611    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 26 23:43:57.285755 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.285740    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 26 23:43:57.285981 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.285962    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ml74s"
May 26 23:43:57.287148 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.287125    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.287961 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.287945    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 26 23:43:57.288175 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.288150    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-25 23:38:56 +0000 UTC" deadline="2027-10-22 01:09:34.418135402 +0000 UTC"
May 26 23:43:57.288216 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.288176    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2bg9t"
May 26 23:43:57.288216 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.288182    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12313h25m37.129956528s"
May 26 23:43:57.288216 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.288160    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 26 23:43:57.288355 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.288340    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rnn6d\""
May 26 23:43:57.288946 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.288932    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 26 23:43:57.289027 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.289014    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 26 23:43:57.289247 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.289235    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 26 23:43:57.289330 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.289316    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vjw25"
May 26 23:43:57.289586 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.289572    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-48p7d\""
May 26 23:43:57.290399 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.290381    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nmd5t\""
May 26 23:43:57.290530 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.290504    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.291161 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.290934    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 26 23:43:57.291161 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.290968    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 26 23:43:57.291319 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.291304    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 26 23:43:57.291684 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.291307    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 26 23:43:57.291684 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.291504    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f5t2j\""
May 26 23:43:57.291684 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.291552    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 26 23:43:57.292007 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.291988    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 26 23:43:57.292164 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.292149    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z68mm"
May 26 23:43:57.292660 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.292641    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2wddf\""
May 26 23:43:57.292732 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.292648    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 26 23:43:57.293406 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.293389    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.294300 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.294283    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 26 23:43:57.294391 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.294344    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 26 23:43:57.294391 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.294384    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 26 23:43:57.294539 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.294525    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tb7xr\""
May 26 23:43:57.294601 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.294577    2566 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal"
May 26 23:43:57.295176 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.295157    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 26 23:43:57.295411 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.295397    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 26 23:43:57.299464 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.295462    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 26 23:43:57.299464 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.298101    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 26 23:43:57.299464 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.298166    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 26 23:43:57.299464 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.298298    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4bhlb\""
May 26 23:43:57.299464 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.298479    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 26 23:43:57.299464 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.299227    2566 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 26 23:43:57.300609 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.300269    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-run-netns\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.300609 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.300305    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52bec009-08b7-4931-8c9f-50435f32eb50-host\") pod \"node-ca-vjw25\" (UID: \"52bec009-08b7-4931-8c9f-50435f32eb50\") " pod="openshift-image-registry/node-ca-vjw25"
May 26 23:43:57.300609 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.300356    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.300609 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.300388    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-socket-dir\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.300609 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.300417    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-sysctl-d\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.300855 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.300473    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-var-lib-cni-multus\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.300855 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.300758    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m2ct\" (UniqueName: \"kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct\") pod \"network-check-target-fplbc\" (UID: \"969b31ff-aeec-4ea2-aaae-9675876698b0\") " pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:43:57.300976 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.300850    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-lib-modules\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.300976 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.300921    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f31218b0-fbd7-43cf-b279-47cc71a12652-tmp-dir\") pod \"node-resolver-2bg9t\" (UID: \"f31218b0-fbd7-43cf-b279-47cc71a12652\") " pod="openshift-dns/node-resolver-2bg9t"
May 26 23:43:57.301075 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.300961    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-cni-bin\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.301130 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.301072    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc54c161-1832-427d-b1b4-5e89b5f06223-ovnkube-config\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.301130 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.301121    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc54c161-1832-427d-b1b4-5e89b5f06223-ovn-node-metrics-cert\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.301217 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.301159    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29dfp\" (UniqueName: \"kubernetes.io/projected/fc54c161-1832-427d-b1b4-5e89b5f06223-kube-api-access-29dfp\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.301217 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.301194    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.301318 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.301239    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-var-lib-cni-bin\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.301318 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.301278    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-system-cni-dir\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.301482 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.301317    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-node-log\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.301701 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.301402    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-sys\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.301753 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.301723    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-run-ovn\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.302064 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302040    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8fh8\" (UniqueName: \"kubernetes.io/projected/c684f499-3fe8-4ae1-9c4b-8969786d1079-kube-api-access-x8fh8\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.302176 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302078    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-multus-cni-dir\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.302176 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302104    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-run-k8s-cni-cncf-io\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.302176 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8ltv\" (UniqueName: \"kubernetes.io/projected/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-kube-api-access-z8ltv\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.302176 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302162    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-slash\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.302368 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302186    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-run-openvswitch\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.302368 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302210    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-system-cni-dir\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.302368 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302233    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-os-release\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.302368 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302255    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-multus-socket-dir-parent\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.302368 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302298    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c98e9301-f2aa-41c5-a412-6cc60125e170-iptables-alerter-script\") pod \"iptables-alerter-z68mm\" (UID: \"c98e9301-f2aa-41c5-a412-6cc60125e170\") " pod="openshift-network-operator/iptables-alerter-z68mm"
May 26 23:43:57.302368 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302329    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-tuned\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.302368 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302347    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5534bef0-4f4f-42e7-84d1-dd88853880b8-tmp\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.302368 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302365    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f31218b0-fbd7-43cf-b279-47cc71a12652-hosts-file\") pod \"node-resolver-2bg9t\" (UID: \"f31218b0-fbd7-43cf-b279-47cc71a12652\") " pod="openshift-dns/node-resolver-2bg9t"
May 26 23:43:57.302707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302426    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc54c161-1832-427d-b1b4-5e89b5f06223-env-overrides\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.302707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302464    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.302707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302485    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-registration-dir\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.302707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302503    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-var-lib-kubelet\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.302707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302519    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/95b1c1fa-d2ee-4912-b837-7cb5db42db65-agent-certs\") pod \"konnectivity-agent-ml74s\" (UID: \"95b1c1fa-d2ee-4912-b837-7cb5db42db65\") " pod="kube-system/konnectivity-agent-ml74s"
May 26 23:43:57.302707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302548    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-run-multus-certs\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.302707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302574    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-etc-kubernetes\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.302707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302589    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-kubelet\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.302707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302604    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-var-lib-openvswitch\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.302707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302628    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-cnibin\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.302707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302648    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-sysctl-conf\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.302707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302663    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-var-lib-kubelet\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.302707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302700    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-os-release\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302736    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-cni-binary-copy\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302762    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52bec009-08b7-4931-8c9f-50435f32eb50-serviceca\") pod \"node-ca-vjw25\" (UID: \"52bec009-08b7-4931-8c9f-50435f32eb50\") " pod="openshift-image-registry/node-ca-vjw25"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302785    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-run-systemd\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302800    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-etc-openvswitch\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302821    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-run-ovn-kubernetes\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302851    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-hostroot\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302871    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87f2k\" (UniqueName: \"kubernetes.io/projected/c98e9301-f2aa-41c5-a412-6cc60125e170-kube-api-access-87f2k\") pod \"iptables-alerter-z68mm\" (UID: \"c98e9301-f2aa-41c5-a412-6cc60125e170\") " pod="openshift-network-operator/iptables-alerter-z68mm"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302909    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-sysconfig\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302930    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-systemd-units\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302945    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc54c161-1832-427d-b1b4-5e89b5f06223-ovnkube-script-lib\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302959    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng7p5\" (UniqueName: \"kubernetes.io/projected/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-kube-api-access-ng7p5\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.302975    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/95b1c1fa-d2ee-4912-b837-7cb5db42db65-konnectivity-ca\") pod \"konnectivity-agent-ml74s\" (UID: \"95b1c1fa-d2ee-4912-b837-7cb5db42db65\") " pod="kube-system/konnectivity-agent-ml74s"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303004    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-etc-selinux\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303026    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-host\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303044    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.303135 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303061    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-multus-daemon-config\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303074    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-kubernetes\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303093    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-run\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303115    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvzr6\" (UniqueName: \"kubernetes.io/projected/6d68dfa4-c945-4139-b299-2664709a5651-kube-api-access-gvzr6\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303148    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-cni-binary-copy\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303172    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-device-dir\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303205    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-sys-fs\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303225    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-cnibin\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303249    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c98e9301-f2aa-41c5-a412-6cc60125e170-host-slash\") pod \"iptables-alerter-z68mm\" (UID: \"c98e9301-f2aa-41c5-a412-6cc60125e170\") " pod="openshift-network-operator/iptables-alerter-z68mm"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303263    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rljgx\" (UniqueName: \"kubernetes.io/projected/f31218b0-fbd7-43cf-b279-47cc71a12652-kube-api-access-rljgx\") pod \"node-resolver-2bg9t\" (UID: \"f31218b0-fbd7-43cf-b279-47cc71a12652\") " pod="openshift-dns/node-resolver-2bg9t"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303277    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-run-netns\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303290    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-cni-netd\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303305    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-multus-conf-dir\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303319    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-modprobe-d\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303352    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-systemd\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303367    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8pj2\" (UniqueName: \"kubernetes.io/projected/5534bef0-4f4f-42e7-84d1-dd88853880b8-kube-api-access-z8pj2\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.303642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303400    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:43:57.304142 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303414    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb8nx\" (UniqueName: \"kubernetes.io/projected/52bec009-08b7-4931-8c9f-50435f32eb50-kube-api-access-bb8nx\") pod \"node-ca-vjw25\" (UID: \"52bec009-08b7-4931-8c9f-50435f32eb50\") " pod="openshift-image-registry/node-ca-vjw25"
May 26 23:43:57.304142 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303428    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-log-socket\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.304142 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.303445    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.305187 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.305171    2566 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 26 23:43:57.309769 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.309744    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal"]
May 26 23:43:57.310412 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.310395    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 26 23:43:57.310497 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.310453    2566 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-129.ec2.internal"
May 26 23:43:57.318551 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.318535    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 26 23:43:57.318818 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.318802    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-129.ec2.internal"]
May 26 23:43:57.326321 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.326306    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kvklg"
May 26 23:43:57.334501 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.334483    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kvklg"
May 26 23:43:57.403621 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.403603    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-multus-conf-dir\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.403725 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.403628    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-modprobe-d\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.403725 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.403648    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-systemd\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.403725 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.403663    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z8pj2\" (UniqueName: \"kubernetes.io/projected/5534bef0-4f4f-42e7-84d1-dd88853880b8-kube-api-access-z8pj2\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.403725 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.403682    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:43:57.403852 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.403725    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-multus-conf-dir\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.403852 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.403773    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-modprobe-d\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.403852 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.403802    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:43:57.403852 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.403830    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bb8nx\" (UniqueName: \"kubernetes.io/projected/52bec009-08b7-4931-8c9f-50435f32eb50-kube-api-access-bb8nx\") pod \"node-ca-vjw25\" (UID: \"52bec009-08b7-4931-8c9f-50435f32eb50\") " pod="openshift-image-registry/node-ca-vjw25"
May 26 23:43:57.404002 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.403848    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-systemd\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.404002 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.403912    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs podName:6d68dfa4-c945-4139-b299-2664709a5651 nodeName:}" failed. No retries permitted until 2026-05-26 23:43:57.903842135 +0000 UTC m=+2.079873294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs") pod "network-metrics-daemon-6mdld" (UID: "6d68dfa4-c945-4139-b299-2664709a5651") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:43:57.404002 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.403934    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-log-socket\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.404002 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.403966    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.404002 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.403998    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-run-netns\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.404171 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404019    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52bec009-08b7-4931-8c9f-50435f32eb50-host\") pod \"node-ca-vjw25\" (UID: \"52bec009-08b7-4931-8c9f-50435f32eb50\") " pod="openshift-image-registry/node-ca-vjw25"
May 26 23:43:57.404171 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404048    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.404171 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404081    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-run-netns\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.404171 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404108    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-socket-dir\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.404171 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404110    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-log-socket\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.404171 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404148    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-sysctl-d\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.404171 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404155    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52bec009-08b7-4931-8c9f-50435f32eb50-host\") pod \"node-ca-vjw25\" (UID: \"52bec009-08b7-4931-8c9f-50435f32eb50\") " pod="openshift-image-registry/node-ca-vjw25"
May 26 23:43:57.404171 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404146    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.404455 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404208    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.404455 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404213    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-var-lib-cni-multus\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.404455 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404262    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7m2ct\" (UniqueName: \"kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct\") pod \"network-check-target-fplbc\" (UID: \"969b31ff-aeec-4ea2-aaae-9675876698b0\") " pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:43:57.404455 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404273    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-sysctl-d\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.404455 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404266    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-var-lib-cni-multus\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.404455 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404268    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-socket-dir\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.404455 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404342    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-lib-modules\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.404455 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404418    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f31218b0-fbd7-43cf-b279-47cc71a12652-tmp-dir\") pod \"node-resolver-2bg9t\" (UID: \"f31218b0-fbd7-43cf-b279-47cc71a12652\") " pod="openshift-dns/node-resolver-2bg9t"
May 26 23:43:57.404719 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404470    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-cni-bin\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.404719 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404515    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-lib-modules\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.404719 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404560    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-cni-bin\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.404719 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404595    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc54c161-1832-427d-b1b4-5e89b5f06223-ovnkube-config\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.404719 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404622    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc54c161-1832-427d-b1b4-5e89b5f06223-ovn-node-metrics-cert\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.404719 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404638    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-29dfp\" (UniqueName: \"kubernetes.io/projected/fc54c161-1832-427d-b1b4-5e89b5f06223-kube-api-access-29dfp\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.404719 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404646    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f31218b0-fbd7-43cf-b279-47cc71a12652-tmp-dir\") pod \"node-resolver-2bg9t\" (UID: \"f31218b0-fbd7-43cf-b279-47cc71a12652\") " pod="openshift-dns/node-resolver-2bg9t"
May 26 23:43:57.404719 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404662    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.405099 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404735    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-var-lib-cni-bin\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.405099 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404772    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-system-cni-dir\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.405099 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404814    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-node-log\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.405099 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404850    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-var-lib-cni-bin\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.405099 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404940    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-sys\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.405099 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404950    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-node-log\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.405099 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.404967    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-run-ovn\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.405099 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405054    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-sys\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.405099 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405069    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x8fh8\" (UniqueName: \"kubernetes.io/projected/c684f499-3fe8-4ae1-9c4b-8969786d1079-kube-api-access-x8fh8\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.405099 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405099    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-multus-cni-dir\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405110    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-run-ovn\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405172    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-run-k8s-cni-cncf-io\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405204    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z8ltv\" (UniqueName: \"kubernetes.io/projected/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-kube-api-access-z8ltv\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405231    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-slash\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405231    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405297    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-multus-cni-dir\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405307    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-run-openvswitch\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405380    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-slash\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405387    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-run-k8s-cni-cncf-io\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405259    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-run-openvswitch\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405398    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-system-cni-dir\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405421    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-system-cni-dir\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405450    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-os-release\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405479    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-multus-socket-dir-parent\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.405543 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405495    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-system-cni-dir\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.406216 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405563    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-os-release\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.406216 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405563    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c98e9301-f2aa-41c5-a412-6cc60125e170-iptables-alerter-script\") pod \"iptables-alerter-z68mm\" (UID: \"c98e9301-f2aa-41c5-a412-6cc60125e170\") " pod="openshift-network-operator/iptables-alerter-z68mm"
May 26 23:43:57.406216 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405608    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-tuned\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.406216 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405636    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-multus-socket-dir-parent\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.406216 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405725    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc54c161-1832-427d-b1b4-5e89b5f06223-ovnkube-config\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.406216 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.405281    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.406557 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.406516    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c98e9301-f2aa-41c5-a412-6cc60125e170-iptables-alerter-script\") pod \"iptables-alerter-z68mm\" (UID: \"c98e9301-f2aa-41c5-a412-6cc60125e170\") " pod="openshift-network-operator/iptables-alerter-z68mm"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.406676    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5534bef0-4f4f-42e7-84d1-dd88853880b8-tmp\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.406744    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f31218b0-fbd7-43cf-b279-47cc71a12652-hosts-file\") pod \"node-resolver-2bg9t\" (UID: \"f31218b0-fbd7-43cf-b279-47cc71a12652\") " pod="openshift-dns/node-resolver-2bg9t"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.406791    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc54c161-1832-427d-b1b4-5e89b5f06223-env-overrides\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.406825    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.406851    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-registration-dir\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.406927    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-var-lib-kubelet\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.406961    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/95b1c1fa-d2ee-4912-b837-7cb5db42db65-agent-certs\") pod \"konnectivity-agent-ml74s\" (UID: \"95b1c1fa-d2ee-4912-b837-7cb5db42db65\") " pod="kube-system/konnectivity-agent-ml74s"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407007    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-run-multus-certs\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407053    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-etc-kubernetes\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407084    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-kubelet\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407116    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-var-lib-openvswitch\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407193    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-cnibin\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407266    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-sysctl-conf\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407299    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-var-lib-kubelet\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407347    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-os-release\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407375    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-cni-binary-copy\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407411    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52bec009-08b7-4931-8c9f-50435f32eb50-serviceca\") pod \"node-ca-vjw25\" (UID: \"52bec009-08b7-4931-8c9f-50435f32eb50\") " pod="openshift-image-registry/node-ca-vjw25"
May 26 23:43:57.410918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407440    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-run-systemd\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407483    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-etc-openvswitch\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407512    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-run-ovn-kubernetes\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407541    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-hostroot\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407639    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-87f2k\" (UniqueName: \"kubernetes.io/projected/c98e9301-f2aa-41c5-a412-6cc60125e170-kube-api-access-87f2k\") pod \"iptables-alerter-z68mm\" (UID: \"c98e9301-f2aa-41c5-a412-6cc60125e170\") " pod="openshift-network-operator/iptables-alerter-z68mm"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407682    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-sysconfig\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407734    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-systemd-units\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407803    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc54c161-1832-427d-b1b4-5e89b5f06223-ovnkube-script-lib\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407843    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ng7p5\" (UniqueName: \"kubernetes.io/projected/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-kube-api-access-ng7p5\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.407984    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/95b1c1fa-d2ee-4912-b837-7cb5db42db65-konnectivity-ca\") pod \"konnectivity-agent-ml74s\" (UID: \"95b1c1fa-d2ee-4912-b837-7cb5db42db65\") " pod="kube-system/konnectivity-agent-ml74s"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.408034    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-etc-selinux\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.408070    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-host\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.408164    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.408225    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-multus-daemon-config\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.408257    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-kubernetes\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.408286    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-run\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.408318    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gvzr6\" (UniqueName: \"kubernetes.io/projected/6d68dfa4-c945-4139-b299-2664709a5651-kube-api-access-gvzr6\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:43:57.411764 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.408955    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-tuned\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409047    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-etc-kubernetes\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409092    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc54c161-1832-427d-b1b4-5e89b5f06223-ovn-node-metrics-cert\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409107    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52bec009-08b7-4931-8c9f-50435f32eb50-serviceca\") pod \"node-ca-vjw25\" (UID: \"52bec009-08b7-4931-8c9f-50435f32eb50\") " pod="openshift-image-registry/node-ca-vjw25"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409176    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f31218b0-fbd7-43cf-b279-47cc71a12652-hosts-file\") pod \"node-resolver-2bg9t\" (UID: \"f31218b0-fbd7-43cf-b279-47cc71a12652\") " pod="openshift-dns/node-resolver-2bg9t"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409180    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-run-systemd\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409206    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-registration-dir\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409373    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-var-lib-kubelet\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409406    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409500    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-kubelet\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409519    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-run-ovn-kubernetes\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409552    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-etc-openvswitch\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409599    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-hostroot\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409652    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-var-lib-openvswitch\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409734    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-cnibin\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409828    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-sysctl-conf\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409844    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-sysconfig\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409936    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-os-release\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.412570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409940    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-systemd-units\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.409983    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-run-multus-certs\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.410028    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-host-var-lib-kubelet\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.410504    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc54c161-1832-427d-b1b4-5e89b5f06223-env-overrides\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.410599    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-cni-binary-copy\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.410660    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-device-dir\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.410758    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-sys-fs\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.410786    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-cnibin\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.410821    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c98e9301-f2aa-41c5-a412-6cc60125e170-host-slash\") pod \"iptables-alerter-z68mm\" (UID: \"c98e9301-f2aa-41c5-a412-6cc60125e170\") " pod="openshift-network-operator/iptables-alerter-z68mm"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.411260    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.411286    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.411312    2566 projected.go:194] Error preparing data for projected volume kube-api-access-7m2ct for pod openshift-network-diagnostics/network-check-target-fplbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.411454    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-device-dir\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.411620    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-sys-fs\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.411671    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c684f499-3fe8-4ae1-9c4b-8969786d1079-etc-selinux\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.411683    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-cnibin\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.411801    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-etc-kubernetes\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.411994    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc54c161-1832-427d-b1b4-5e89b5f06223-ovnkube-script-lib\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.413440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.412065    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rljgx\" (UniqueName: \"kubernetes.io/projected/f31218b0-fbd7-43cf-b279-47cc71a12652-kube-api-access-rljgx\") pod \"node-resolver-2bg9t\" (UID: \"f31218b0-fbd7-43cf-b279-47cc71a12652\") " pod="openshift-dns/node-resolver-2bg9t"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.412074    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c98e9301-f2aa-41c5-a412-6cc60125e170-host-slash\") pod \"iptables-alerter-z68mm\" (UID: \"c98e9301-f2aa-41c5-a412-6cc60125e170\") " pod="openshift-network-operator/iptables-alerter-z68mm"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.412191    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct podName:969b31ff-aeec-4ea2-aaae-9675876698b0 nodeName:}" failed. No retries permitted until 2026-05-26 23:43:57.912160987 +0000 UTC m=+2.088192146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7m2ct" (UniqueName: "kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct") pod "network-check-target-fplbc" (UID: "969b31ff-aeec-4ea2-aaae-9675876698b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.412395    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/95b1c1fa-d2ee-4912-b837-7cb5db42db65-konnectivity-ca\") pod \"konnectivity-agent-ml74s\" (UID: \"95b1c1fa-d2ee-4912-b837-7cb5db42db65\") " pod="kube-system/konnectivity-agent-ml74s"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.412415    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-run-netns\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.412408    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-cni-binary-copy\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.412454    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-cni-netd\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.412457    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-host\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.412751    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/95b1c1fa-d2ee-4912-b837-7cb5db42db65-agent-certs\") pod \"konnectivity-agent-ml74s\" (UID: \"95b1c1fa-d2ee-4912-b837-7cb5db42db65\") " pod="kube-system/konnectivity-agent-ml74s"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.412785    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-cni-binary-copy\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.412980    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5534bef0-4f4f-42e7-84d1-dd88853880b8-tmp\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.413344    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5534bef0-4f4f-42e7-84d1-dd88853880b8-run\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.413350    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.413281    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8pj2\" (UniqueName: \"kubernetes.io/projected/5534bef0-4f4f-42e7-84d1-dd88853880b8-kube-api-access-z8pj2\") pod \"tuned-4r7ws\" (UID: \"5534bef0-4f4f-42e7-84d1-dd88853880b8\") " pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.413495    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-run-netns\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.413556    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc54c161-1832-427d-b1b4-5e89b5f06223-host-cni-netd\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.414472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.413648    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-multus-daemon-config\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.415720 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.415695    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb8nx\" (UniqueName: \"kubernetes.io/projected/52bec009-08b7-4931-8c9f-50435f32eb50-kube-api-access-bb8nx\") pod \"node-ca-vjw25\" (UID: \"52bec009-08b7-4931-8c9f-50435f32eb50\") " pod="openshift-image-registry/node-ca-vjw25"
May 26 23:43:57.416547 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.416527    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29dfp\" (UniqueName: \"kubernetes.io/projected/fc54c161-1832-427d-b1b4-5e89b5f06223-kube-api-access-29dfp\") pod \"ovnkube-node-nh9zh\" (UID: \"fc54c161-1832-427d-b1b4-5e89b5f06223\") " pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.416940 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.416919    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8ltv\" (UniqueName: \"kubernetes.io/projected/a26f4c89-994b-46d2-aae3-67c5c13e7f0a-kube-api-access-z8ltv\") pod \"multus-pmsgm\" (UID: \"a26f4c89-994b-46d2-aae3-67c5c13e7f0a\") " pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.417376 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.417331    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8fh8\" (UniqueName: \"kubernetes.io/projected/c684f499-3fe8-4ae1-9c4b-8969786d1079-kube-api-access-x8fh8\") pod \"aws-ebs-csi-driver-node-zjxxb\" (UID: \"c684f499-3fe8-4ae1-9c4b-8969786d1079\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.418181 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.418161    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvzr6\" (UniqueName: \"kubernetes.io/projected/6d68dfa4-c945-4139-b299-2664709a5651-kube-api-access-gvzr6\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:43:57.418279 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.418241    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87f2k\" (UniqueName: \"kubernetes.io/projected/c98e9301-f2aa-41c5-a412-6cc60125e170-kube-api-access-87f2k\") pod \"iptables-alerter-z68mm\" (UID: \"c98e9301-f2aa-41c5-a412-6cc60125e170\") " pod="openshift-network-operator/iptables-alerter-z68mm"
May 26 23:43:57.419402 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.419380    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rljgx\" (UniqueName: \"kubernetes.io/projected/f31218b0-fbd7-43cf-b279-47cc71a12652-kube-api-access-rljgx\") pod \"node-resolver-2bg9t\" (UID: \"f31218b0-fbd7-43cf-b279-47cc71a12652\") " pod="openshift-dns/node-resolver-2bg9t"
May 26 23:43:57.422210 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.422190    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng7p5\" (UniqueName: \"kubernetes.io/projected/8e2588a1-9d18-4ea2-8ec4-e14b672058b4-kube-api-access-ng7p5\") pod \"multus-additional-cni-plugins-vjq8g\" (UID: \"8e2588a1-9d18-4ea2-8ec4-e14b672058b4\") " pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.432942 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.432925    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:43:57.459558 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:57.459522    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc54c161_1832_427d_b1b4_5e89b5f06223.slice/crio-97f1cee7fae1d5f91ba6a83b80b15c2a256f32bafcb4ec62c2df3dffff698721 WatchSource:0}: Error finding container 97f1cee7fae1d5f91ba6a83b80b15c2a256f32bafcb4ec62c2df3dffff698721: Status 404 returned error can't find the container with id 97f1cee7fae1d5f91ba6a83b80b15c2a256f32bafcb4ec62c2df3dffff698721
May 26 23:43:57.459864 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:57.459836    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbebf15267ab3a3a2579421856274a9a6.slice/crio-98ed9dffc5e66882b78ae9b5f71ee17b98fda227a48f70b49eb6cee95d21af92 WatchSource:0}: Error finding container 98ed9dffc5e66882b78ae9b5f71ee17b98fda227a48f70b49eb6cee95d21af92: Status 404 returned error can't find the container with id 98ed9dffc5e66882b78ae9b5f71ee17b98fda227a48f70b49eb6cee95d21af92
May 26 23:43:57.460057 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:57.460042    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5e5513b500b482a07e342702239169c.slice/crio-ed0942a3cc44ef6bbbf58aff8f7977011047b8f04a8d79a649a27b983c43c4a5 WatchSource:0}: Error finding container ed0942a3cc44ef6bbbf58aff8f7977011047b8f04a8d79a649a27b983c43c4a5: Status 404 returned error can't find the container with id ed0942a3cc44ef6bbbf58aff8f7977011047b8f04a8d79a649a27b983c43c4a5
May 26 23:43:57.463637 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.463623    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 26 23:43:57.590043 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.590019    2566 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 26 23:43:57.612615 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.612595    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4r7ws"
May 26 23:43:57.618626 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:57.618605    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5534bef0_4f4f_42e7_84d1_dd88853880b8.slice/crio-eee6be64101d38489a253b12320c8e030ee7b991967371f4e9afbd13997feacf WatchSource:0}: Error finding container eee6be64101d38489a253b12320c8e030ee7b991967371f4e9afbd13997feacf: Status 404 returned error can't find the container with id eee6be64101d38489a253b12320c8e030ee7b991967371f4e9afbd13997feacf
May 26 23:43:57.633251 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.633234    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vjq8g"
May 26 23:43:57.637816 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.637799    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ml74s"
May 26 23:43:57.639228 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:57.639205    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2588a1_9d18_4ea2_8ec4_e14b672058b4.slice/crio-7de746fb9e781de9aca72fc2e3ec07a020d462a825eb96b391ef7bf7928cca7b WatchSource:0}: Error finding container 7de746fb9e781de9aca72fc2e3ec07a020d462a825eb96b391ef7bf7928cca7b: Status 404 returned error can't find the container with id 7de746fb9e781de9aca72fc2e3ec07a020d462a825eb96b391ef7bf7928cca7b
May 26 23:43:57.646009 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:57.645980    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b1c1fa_d2ee_4912_b837_7cb5db42db65.slice/crio-4efc1095fb81425d83db3a654c1de3a877e9db26dc5f148b1689b6c7d742c01f WatchSource:0}: Error finding container 4efc1095fb81425d83db3a654c1de3a877e9db26dc5f148b1689b6c7d742c01f: Status 404 returned error can't find the container with id 4efc1095fb81425d83db3a654c1de3a877e9db26dc5f148b1689b6c7d742c01f
May 26 23:43:57.651867 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.651845    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb"
May 26 23:43:57.657223 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:57.657203    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc684f499_3fe8_4ae1_9c4b_8969786d1079.slice/crio-ee1101c8571afad7aa97dc004d04add84c37fc247afe70d087338dc14f199e2f WatchSource:0}: Error finding container ee1101c8571afad7aa97dc004d04add84c37fc247afe70d087338dc14f199e2f: Status 404 returned error can't find the container with id ee1101c8571afad7aa97dc004d04add84c37fc247afe70d087338dc14f199e2f
May 26 23:43:57.666218 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.666200    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2bg9t"
May 26 23:43:57.671423 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:57.671401    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf31218b0_fbd7_43cf_b279_47cc71a12652.slice/crio-ac56c5ad5b242f9d4d0d86566c1fec8c96521ccbac5700940a1a0bf114ed7560 WatchSource:0}: Error finding container ac56c5ad5b242f9d4d0d86566c1fec8c96521ccbac5700940a1a0bf114ed7560: Status 404 returned error can't find the container with id ac56c5ad5b242f9d4d0d86566c1fec8c96521ccbac5700940a1a0bf114ed7560
May 26 23:43:57.672045 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.672026    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vjw25"
May 26 23:43:57.678592 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:57.678573    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52bec009_08b7_4931_8c9f_50435f32eb50.slice/crio-cfc8d28d752315e6d666d358b319cdf09f56a047fccd573c670dbda948103afc WatchSource:0}: Error finding container cfc8d28d752315e6d666d358b319cdf09f56a047fccd573c670dbda948103afc: Status 404 returned error can't find the container with id cfc8d28d752315e6d666d358b319cdf09f56a047fccd573c670dbda948103afc
May 26 23:43:57.694155 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.694140    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pmsgm"
May 26 23:43:57.700348 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:57.700329    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda26f4c89_994b_46d2_aae3_67c5c13e7f0a.slice/crio-313b186db30cfafb1f58c2c8f86627d324f4c73e61c17bcfd42977af65210fde WatchSource:0}: Error finding container 313b186db30cfafb1f58c2c8f86627d324f4c73e61c17bcfd42977af65210fde: Status 404 returned error can't find the container with id 313b186db30cfafb1f58c2c8f86627d324f4c73e61c17bcfd42977af65210fde
May 26 23:43:57.713636 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.713618    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z68mm"
May 26 23:43:57.719812 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:43:57.719795    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc98e9301_f2aa_41c5_a412_6cc60125e170.slice/crio-e289a7e3c966b6309ba652701cabedb4ad014a5cb6a3a1abe2d23b41f7a76ca0 WatchSource:0}: Error finding container e289a7e3c966b6309ba652701cabedb4ad014a5cb6a3a1abe2d23b41f7a76ca0: Status 404 returned error can't find the container with id e289a7e3c966b6309ba652701cabedb4ad014a5cb6a3a1abe2d23b41f7a76ca0
May 26 23:43:57.916844 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.916768    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:43:57.916844 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:57.916816    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7m2ct\" (UniqueName: \"kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct\") pod \"network-check-target-fplbc\" (UID: \"969b31ff-aeec-4ea2-aaae-9675876698b0\") " pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:43:57.917080 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.916940    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 26 23:43:57.917080 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.916957    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 26 23:43:57.917080 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.916966    2566 projected.go:194] Error preparing data for projected volume kube-api-access-7m2ct for pod openshift-network-diagnostics/network-check-target-fplbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:43:57.917080 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.917008    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct podName:969b31ff-aeec-4ea2-aaae-9675876698b0 nodeName:}" failed. No retries permitted until 2026-05-26 23:43:58.916995562 +0000 UTC m=+3.093026702 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7m2ct" (UniqueName: "kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct") pod "network-check-target-fplbc" (UID: "969b31ff-aeec-4ea2-aaae-9675876698b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:43:57.917080 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.917067    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:43:57.917336 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:57.917093    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs podName:6d68dfa4-c945-4139-b299-2664709a5651 nodeName:}" failed. No retries permitted until 2026-05-26 23:43:58.917086142 +0000 UTC m=+3.093117281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs") pod "network-metrics-daemon-6mdld" (UID: "6d68dfa4-c945-4139-b299-2664709a5651") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:43:58.215921 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.215819    2566 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 26 23:43:58.335808 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.335768    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-25 23:38:57 +0000 UTC" deadline="2027-12-31 19:18:40.956455582 +0000 UTC"
May 26 23:43:58.335808 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.335808    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14011h34m42.620652575s"
May 26 23:43:58.436862 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.436805    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb" event={"ID":"c684f499-3fe8-4ae1-9c4b-8969786d1079","Type":"ContainerStarted","Data":"ee1101c8571afad7aa97dc004d04add84c37fc247afe70d087338dc14f199e2f"}
May 26 23:43:58.444818 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.444775    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ml74s" event={"ID":"95b1c1fa-d2ee-4912-b837-7cb5db42db65","Type":"ContainerStarted","Data":"4efc1095fb81425d83db3a654c1de3a877e9db26dc5f148b1689b6c7d742c01f"}
May 26 23:43:58.484509 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.484423    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjq8g" event={"ID":"8e2588a1-9d18-4ea2-8ec4-e14b672058b4","Type":"ContainerStarted","Data":"7de746fb9e781de9aca72fc2e3ec07a020d462a825eb96b391ef7bf7928cca7b"}
May 26 23:43:58.487289 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.487258    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh" event={"ID":"fc54c161-1832-427d-b1b4-5e89b5f06223","Type":"ContainerStarted","Data":"97f1cee7fae1d5f91ba6a83b80b15c2a256f32bafcb4ec62c2df3dffff698721"}
May 26 23:43:58.506861 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.506824    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-129.ec2.internal" event={"ID":"bebf15267ab3a3a2579421856274a9a6","Type":"ContainerStarted","Data":"98ed9dffc5e66882b78ae9b5f71ee17b98fda227a48f70b49eb6cee95d21af92"}
May 26 23:43:58.509783 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.509755    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vjw25" event={"ID":"52bec009-08b7-4931-8c9f-50435f32eb50","Type":"ContainerStarted","Data":"cfc8d28d752315e6d666d358b319cdf09f56a047fccd573c670dbda948103afc"}
May 26 23:43:58.515239 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.515101    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4r7ws" event={"ID":"5534bef0-4f4f-42e7-84d1-dd88853880b8","Type":"ContainerStarted","Data":"eee6be64101d38489a253b12320c8e030ee7b991967371f4e9afbd13997feacf"}
May 26 23:43:58.531652 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.531623    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal" event={"ID":"b5e5513b500b482a07e342702239169c","Type":"ContainerStarted","Data":"ed0942a3cc44ef6bbbf58aff8f7977011047b8f04a8d79a649a27b983c43c4a5"}
May 26 23:43:58.543221 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.543194    2566 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 26 23:43:58.543833 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.543772    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z68mm" event={"ID":"c98e9301-f2aa-41c5-a412-6cc60125e170","Type":"ContainerStarted","Data":"e289a7e3c966b6309ba652701cabedb4ad014a5cb6a3a1abe2d23b41f7a76ca0"}
May 26 23:43:58.558477 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.558443    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pmsgm" event={"ID":"a26f4c89-994b-46d2-aae3-67c5c13e7f0a","Type":"ContainerStarted","Data":"313b186db30cfafb1f58c2c8f86627d324f4c73e61c17bcfd42977af65210fde"}
May 26 23:43:58.563839 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.563803    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2bg9t" event={"ID":"f31218b0-fbd7-43cf-b279-47cc71a12652","Type":"ContainerStarted","Data":"ac56c5ad5b242f9d4d0d86566c1fec8c96521ccbac5700940a1a0bf114ed7560"}
May 26 23:43:58.925384 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.925338    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:43:58.925582 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:58.925404    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7m2ct\" (UniqueName: \"kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct\") pod \"network-check-target-fplbc\" (UID: \"969b31ff-aeec-4ea2-aaae-9675876698b0\") " pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:43:58.925582 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:58.925571    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 26 23:43:58.925695 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:58.925592    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 26 23:43:58.925695 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:58.925605    2566 projected.go:194] Error preparing data for projected volume kube-api-access-7m2ct for pod openshift-network-diagnostics/network-check-target-fplbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:43:58.925695 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:58.925667    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct podName:969b31ff-aeec-4ea2-aaae-9675876698b0 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:00.925647588 +0000 UTC m=+5.101678733 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7m2ct" (UniqueName: "kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct") pod "network-check-target-fplbc" (UID: "969b31ff-aeec-4ea2-aaae-9675876698b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:43:58.926210 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:58.926154    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:43:58.926303 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:58.926215    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs podName:6d68dfa4-c945-4139-b299-2664709a5651 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:00.926198248 +0000 UTC m=+5.102229392 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs") pod "network-metrics-daemon-6mdld" (UID: "6d68dfa4-c945-4139-b299-2664709a5651") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:43:59.336995 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:59.336934    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-25 23:38:57 +0000 UTC" deadline="2028-02-24 12:38:39.272269102 +0000 UTC"
May 26 23:43:59.336995 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:59.336975    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15324h54m39.935297752s"
May 26 23:43:59.426878 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:59.426849    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:43:59.426878 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:43:59.426869    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:43:59.427127 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:59.426977    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:43:59.427445 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:43:59.427416    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:00.943297 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:00.943259    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:00.943745 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:00.943328    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7m2ct\" (UniqueName: \"kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct\") pod \"network-check-target-fplbc\" (UID: \"969b31ff-aeec-4ea2-aaae-9675876698b0\") " pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:00.943745 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:00.943466    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 26 23:44:00.943745 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:00.943483    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 26 23:44:00.943745 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:00.943495    2566 projected.go:194] Error preparing data for projected volume kube-api-access-7m2ct for pod openshift-network-diagnostics/network-check-target-fplbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:44:00.943745 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:00.943553    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct podName:969b31ff-aeec-4ea2-aaae-9675876698b0 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:04.943534473 +0000 UTC m=+9.119565632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7m2ct" (UniqueName: "kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct") pod "network-check-target-fplbc" (UID: "969b31ff-aeec-4ea2-aaae-9675876698b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:44:00.944024 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:00.943975    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:44:00.944080 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:00.944026    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs podName:6d68dfa4-c945-4139-b299-2664709a5651 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:04.944011088 +0000 UTC m=+9.120042243 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs") pod "network-metrics-daemon-6mdld" (UID: "6d68dfa4-c945-4139-b299-2664709a5651") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:44:01.427817 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:01.427185    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:01.427817 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:01.427325    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:01.427817 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:01.427688    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:01.427817 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:01.427770    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:03.427257 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:03.427226    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:03.427668 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:03.427348    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:03.427733 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:03.427674    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:03.427794 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:03.427772    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:04.980434 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:04.980388    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:04.980855 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:04.980452    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7m2ct\" (UniqueName: \"kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct\") pod \"network-check-target-fplbc\" (UID: \"969b31ff-aeec-4ea2-aaae-9675876698b0\") " pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:04.980855 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:04.980552    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:44:04.980855 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:04.980573    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 26 23:44:04.980855 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:04.980587    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 26 23:44:04.980855 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:04.980599    2566 projected.go:194] Error preparing data for projected volume kube-api-access-7m2ct for pod openshift-network-diagnostics/network-check-target-fplbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:44:04.980855 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:04.980625    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs podName:6d68dfa4-c945-4139-b299-2664709a5651 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:12.980604985 +0000 UTC m=+17.156636148 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs") pod "network-metrics-daemon-6mdld" (UID: "6d68dfa4-c945-4139-b299-2664709a5651") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:44:04.980855 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:04.980650    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct podName:969b31ff-aeec-4ea2-aaae-9675876698b0 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:12.980642402 +0000 UTC m=+17.156673543 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7m2ct" (UniqueName: "kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct") pod "network-check-target-fplbc" (UID: "969b31ff-aeec-4ea2-aaae-9675876698b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:44:05.427710 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:05.427674    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:05.427914 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:05.427851    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:05.428250 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:05.428232    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:05.428332 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:05.428316    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:06.204652 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:06.204616    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-2zdft"]
May 26 23:44:06.209120 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:06.208808    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:06.209120 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:06.208878    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zdft" podUID="2077a3a9-df5b-4a8c-8ed8-b7572ce31153"
May 26 23:44:06.291172 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:06.291135    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-kubelet-config\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:06.291355 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:06.291188    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-dbus\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:06.291355 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:06.291220    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:06.393180 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:06.392481    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-kubelet-config\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:06.393180 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:06.392746    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-dbus\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:06.393180 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:06.392771    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:06.393180 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:06.392592    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-kubelet-config\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:06.393180 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:06.392926    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 26 23:44:06.393180 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:06.392945    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-dbus\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:06.393180 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:06.392982    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret podName:2077a3a9-df5b-4a8c-8ed8-b7572ce31153 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:06.89296392 +0000 UTC m=+11.068995067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret") pod "global-pull-secret-syncer-2zdft" (UID: "2077a3a9-df5b-4a8c-8ed8-b7572ce31153") : object "kube-system"/"original-pull-secret" not registered
May 26 23:44:06.897258 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:06.897214    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:06.897444 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:06.897365    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 26 23:44:06.897444 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:06.897427    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret podName:2077a3a9-df5b-4a8c-8ed8-b7572ce31153 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:07.897409727 +0000 UTC m=+12.073440880 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret") pod "global-pull-secret-syncer-2zdft" (UID: "2077a3a9-df5b-4a8c-8ed8-b7572ce31153") : object "kube-system"/"original-pull-secret" not registered
May 26 23:44:07.427516 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:07.427477    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:07.427951 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:07.427600    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:07.427951 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:07.427946    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:07.428094 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:07.428054    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:07.904946 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:07.904887    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:07.905107 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:07.905034    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 26 23:44:07.905107 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:07.905100    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret podName:2077a3a9-df5b-4a8c-8ed8-b7572ce31153 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:09.90508616 +0000 UTC m=+14.081117306 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret") pod "global-pull-secret-syncer-2zdft" (UID: "2077a3a9-df5b-4a8c-8ed8-b7572ce31153") : object "kube-system"/"original-pull-secret" not registered
May 26 23:44:08.427030 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:08.426996    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:08.427184 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:08.427135    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zdft" podUID="2077a3a9-df5b-4a8c-8ed8-b7572ce31153"
May 26 23:44:09.427475 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:09.427433    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:09.427921 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:09.427557    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:09.427921 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:09.427613    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:09.427921 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:09.427718    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:09.920668 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:09.920615    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:09.920996 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:09.920769    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 26 23:44:09.920996 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:09.920836    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret podName:2077a3a9-df5b-4a8c-8ed8-b7572ce31153 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:13.920822036 +0000 UTC m=+18.096853181 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret") pod "global-pull-secret-syncer-2zdft" (UID: "2077a3a9-df5b-4a8c-8ed8-b7572ce31153") : object "kube-system"/"original-pull-secret" not registered
May 26 23:44:10.428237 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:10.427689    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:10.428237 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:10.427823    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zdft" podUID="2077a3a9-df5b-4a8c-8ed8-b7572ce31153"
May 26 23:44:11.427854 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:11.427813    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:11.428049 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:11.427813    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:11.428049 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:11.427970    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:11.428049 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:11.428033    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:12.427785 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:12.427676    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:12.428158 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:12.427799    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zdft" podUID="2077a3a9-df5b-4a8c-8ed8-b7572ce31153"
May 26 23:44:13.045872 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:13.045832    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:13.046110 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:13.045887    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7m2ct\" (UniqueName: \"kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct\") pod \"network-check-target-fplbc\" (UID: \"969b31ff-aeec-4ea2-aaae-9675876698b0\") " pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:13.046110 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:13.046000    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:44:13.046110 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:13.046038    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 26 23:44:13.046110 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:13.046062    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 26 23:44:13.046110 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:13.046076    2566 projected.go:194] Error preparing data for projected volume kube-api-access-7m2ct for pod openshift-network-diagnostics/network-check-target-fplbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:44:13.046110 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:13.046079    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs podName:6d68dfa4-c945-4139-b299-2664709a5651 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:29.046058205 +0000 UTC m=+33.222089358 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs") pod "network-metrics-daemon-6mdld" (UID: "6d68dfa4-c945-4139-b299-2664709a5651") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:44:13.046400 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:13.046117    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct podName:969b31ff-aeec-4ea2-aaae-9675876698b0 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:29.046104223 +0000 UTC m=+33.222135363 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7m2ct" (UniqueName: "kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct") pod "network-check-target-fplbc" (UID: "969b31ff-aeec-4ea2-aaae-9675876698b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:44:13.427043 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:13.427011    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:13.427043 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:13.427034    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:13.427259 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:13.427129    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:13.427324 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:13.427275    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:13.952523 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:13.952484    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:13.952956 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:13.952651    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 26 23:44:13.952956 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:13.952726    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret podName:2077a3a9-df5b-4a8c-8ed8-b7572ce31153 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:21.952706083 +0000 UTC m=+26.128737233 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret") pod "global-pull-secret-syncer-2zdft" (UID: "2077a3a9-df5b-4a8c-8ed8-b7572ce31153") : object "kube-system"/"original-pull-secret" not registered
May 26 23:44:14.427691 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:14.427649    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:14.427857 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:14.427782    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zdft" podUID="2077a3a9-df5b-4a8c-8ed8-b7572ce31153"
May 26 23:44:15.427835 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:15.427792    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:15.428316 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:15.427792    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:15.428316 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:15.427948    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:15.428316 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:15.428025    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:16.428270 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:16.428237    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:16.428619 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:16.428317    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zdft" podUID="2077a3a9-df5b-4a8c-8ed8-b7572ce31153"
May 26 23:44:17.426839 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.426818    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:17.426930 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.426847    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:17.426966 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:17.426940    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:17.427096 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:17.427071    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:17.606795 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.606575    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pmsgm" event={"ID":"a26f4c89-994b-46d2-aae3-67c5c13e7f0a","Type":"ContainerStarted","Data":"a04a651ab05796c44b8a0f1c24f43e13777d235b78c09ba7df0928f1319b3b2a"}
May 26 23:44:17.607981 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.607958    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2bg9t" event={"ID":"f31218b0-fbd7-43cf-b279-47cc71a12652","Type":"ContainerStarted","Data":"0c2efd6faba03ba8804c8dacb52a31dd27579552e4c191249305b298babc4f30"}
May 26 23:44:17.609178 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.609159    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb" event={"ID":"c684f499-3fe8-4ae1-9c4b-8969786d1079","Type":"ContainerStarted","Data":"383392d49a2d02a1fe9df33924457c33e30733b4cc81c34c87cdfd42658a44e8"}
May 26 23:44:17.610259 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.610238    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ml74s" event={"ID":"95b1c1fa-d2ee-4912-b837-7cb5db42db65","Type":"ContainerStarted","Data":"7841a39f2c0b81054704102f899ea7a0a205701ac164337f6d9cf2c0d9077fe3"}
May 26 23:44:17.611417 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.611396    2566 generic.go:358] "Generic (PLEG): container finished" podID="8e2588a1-9d18-4ea2-8ec4-e14b672058b4" containerID="16caa7abd0aa42ffc2f5d8f3212d3a0b889a6e50fb51426a58850f5866353e07" exitCode=0
May 26 23:44:17.611499 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.611470    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjq8g" event={"ID":"8e2588a1-9d18-4ea2-8ec4-e14b672058b4","Type":"ContainerDied","Data":"16caa7abd0aa42ffc2f5d8f3212d3a0b889a6e50fb51426a58850f5866353e07"}
May 26 23:44:17.613862 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.613828    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh" event={"ID":"fc54c161-1832-427d-b1b4-5e89b5f06223","Type":"ContainerStarted","Data":"27737a844ad633ed5c2b322b82f3b912db9716b3d8c781df70158b7f1fddd079"}
May 26 23:44:17.613969 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.613871    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh" event={"ID":"fc54c161-1832-427d-b1b4-5e89b5f06223","Type":"ContainerStarted","Data":"81bc1362ccadf3a07dcd7929465be7baada2061d0040036e1d090198cd995d4e"}
May 26 23:44:17.613969 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.613884    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh" event={"ID":"fc54c161-1832-427d-b1b4-5e89b5f06223","Type":"ContainerStarted","Data":"2782d55415b5ff78e0a9bff48ac1af3b8340789b9bc8b30bb7c2d043152cd2c3"}
May 26 23:44:17.613969 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.613914    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh" event={"ID":"fc54c161-1832-427d-b1b4-5e89b5f06223","Type":"ContainerStarted","Data":"67f3d62b3cdc6a36330a4c33bfdf6d12f761706901199528a938fa08eba48a42"}
May 26 23:44:17.613969 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.613928    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh" event={"ID":"fc54c161-1832-427d-b1b4-5e89b5f06223","Type":"ContainerStarted","Data":"3e07644d8c362d87f558eb570297dfd07101835fcf3ffc6dcf13c79cc0c2a645"}
May 26 23:44:17.615039 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.615022    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-129.ec2.internal" event={"ID":"bebf15267ab3a3a2579421856274a9a6","Type":"ContainerStarted","Data":"9d02b9471155502a8d6a91cfbc2ec8b675cdb8d95c8c352f4e2e2bc99549b616"}
May 26 23:44:17.616270 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.616249    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vjw25" event={"ID":"52bec009-08b7-4931-8c9f-50435f32eb50","Type":"ContainerStarted","Data":"c5874c2a1f5c4030cbf80a32f525207a2065c6c83bd627ddd97e8df5f6c4fe33"}
May 26 23:44:17.617258 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.617241    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4r7ws" event={"ID":"5534bef0-4f4f-42e7-84d1-dd88853880b8","Type":"ContainerStarted","Data":"809be0bc1a24cd403a14c3500c24137bce84bbb3d519e55d582ca85fb8810ca9"}
May 26 23:44:17.618383 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.618363    2566 generic.go:358] "Generic (PLEG): container finished" podID="b5e5513b500b482a07e342702239169c" containerID="6105b3b16f2a63d2c38f0730eca7db4cd19a028be91904c2a72aa9d01ddcd7a9" exitCode=0
May 26 23:44:17.618454 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.618406    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal" event={"ID":"b5e5513b500b482a07e342702239169c","Type":"ContainerDied","Data":"6105b3b16f2a63d2c38f0730eca7db4cd19a028be91904c2a72aa9d01ddcd7a9"}
May 26 23:44:17.631047 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.630999    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-pmsgm" podStartSLOduration=2.200874962 podStartE2EDuration="21.630988871s" podCreationTimestamp="2026-05-26 23:43:56 +0000 UTC" firstStartedPulling="2026-05-26 23:43:57.70173543 +0000 UTC m=+1.877766571" lastFinishedPulling="2026-05-26 23:44:17.131849337 +0000 UTC m=+21.307880480" observedRunningTime="2026-05-26 23:44:17.630738946 +0000 UTC m=+21.806770106" watchObservedRunningTime="2026-05-26 23:44:17.630988871 +0000 UTC m=+21.807020031"
May 26 23:44:17.660284 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.660242    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4r7ws" podStartSLOduration=2.355964844 podStartE2EDuration="21.660229827s" podCreationTimestamp="2026-05-26 23:43:56 +0000 UTC" firstStartedPulling="2026-05-26 23:43:57.620565163 +0000 UTC m=+1.796596304" lastFinishedPulling="2026-05-26 23:44:16.924830132 +0000 UTC m=+21.100861287" observedRunningTime="2026-05-26 23:44:17.659822437 +0000 UTC m=+21.835853598" watchObservedRunningTime="2026-05-26 23:44:17.660229827 +0000 UTC m=+21.836260997"
May 26 23:44:17.720548 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.720509    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ml74s" podStartSLOduration=2.478464641 podStartE2EDuration="21.720493737s" podCreationTimestamp="2026-05-26 23:43:56 +0000 UTC" firstStartedPulling="2026-05-26 23:43:57.647621436 +0000 UTC m=+1.823652579" lastFinishedPulling="2026-05-26 23:44:16.889650529 +0000 UTC m=+21.065681675" observedRunningTime="2026-05-26 23:44:17.699042513 +0000 UTC m=+21.875073674" watchObservedRunningTime="2026-05-26 23:44:17.720493737 +0000 UTC m=+21.896524898"
May 26 23:44:17.720666 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.720646    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-2bg9t" podStartSLOduration=2.528083646 podStartE2EDuration="21.720641437s" podCreationTimestamp="2026-05-26 23:43:56 +0000 UTC" firstStartedPulling="2026-05-26 23:43:57.673086441 +0000 UTC m=+1.849117582" lastFinishedPulling="2026-05-26 23:44:16.865644232 +0000 UTC m=+21.041675373" observedRunningTime="2026-05-26 23:44:17.720139779 +0000 UTC m=+21.896170941" watchObservedRunningTime="2026-05-26 23:44:17.720641437 +0000 UTC m=+21.896672596"
May 26 23:44:17.735597 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.735561    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-129.ec2.internal" podStartSLOduration=20.735548171 podStartE2EDuration="20.735548171s" podCreationTimestamp="2026-05-26 23:43:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 23:44:17.735415027 +0000 UTC m=+21.911446189" watchObservedRunningTime="2026-05-26 23:44:17.735548171 +0000 UTC m=+21.911579333"
May 26 23:44:17.750223 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:17.750183    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vjw25" podStartSLOduration=2.540250839 podStartE2EDuration="21.750172276s" podCreationTimestamp="2026-05-26 23:43:56 +0000 UTC" firstStartedPulling="2026-05-26 23:43:57.679732567 +0000 UTC m=+1.855763710" lastFinishedPulling="2026-05-26 23:44:16.889654006 +0000 UTC m=+21.065685147" observedRunningTime="2026-05-26 23:44:17.750062494 +0000 UTC m=+21.926093654" watchObservedRunningTime="2026-05-26 23:44:17.750172276 +0000 UTC m=+21.926203437"
May 26 23:44:18.077738 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:18.077702    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ml74s"
May 26 23:44:18.078363 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:18.078341    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ml74s"
May 26 23:44:18.427532 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:18.427497    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:18.427699 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:18.427613    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zdft" podUID="2077a3a9-df5b-4a8c-8ed8-b7572ce31153"
May 26 23:44:18.623435 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:18.623396    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh" event={"ID":"fc54c161-1832-427d-b1b4-5e89b5f06223","Type":"ContainerStarted","Data":"8030ac5ca7716587081255bde63dde343b3e9a0cf9f3d26f197d0e5d251e5638"}
May 26 23:44:18.625085 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:18.625048    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal" event={"ID":"b5e5513b500b482a07e342702239169c","Type":"ContainerStarted","Data":"c91e55e3870cf5437c59a873bf535f76df36deed9d6ee3ae8513b09c535339e2"}
May 26 23:44:18.626389 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:18.626364    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z68mm" event={"ID":"c98e9301-f2aa-41c5-a412-6cc60125e170","Type":"ContainerStarted","Data":"84ab425f19ce26f5878baa89e825fcf29822b4e8fa68934dd497b5890c4e3da2"}
May 26 23:44:18.626705 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:18.626670    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ml74s"
May 26 23:44:18.627244 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:18.627227    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ml74s"
May 26 23:44:18.640147 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:18.640112    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-129.ec2.internal" podStartSLOduration=21.640102151 podStartE2EDuration="21.640102151s" podCreationTimestamp="2026-05-26 23:43:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 23:44:18.639815302 +0000 UTC m=+22.815846469" watchObservedRunningTime="2026-05-26 23:44:18.640102151 +0000 UTC m=+22.816133313"
May 26 23:44:18.654031 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:18.653989    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z68mm" podStartSLOduration=3.452322029 podStartE2EDuration="22.653975337s" podCreationTimestamp="2026-05-26 23:43:56 +0000 UTC" firstStartedPulling="2026-05-26 23:43:57.721197178 +0000 UTC m=+1.897228317" lastFinishedPulling="2026-05-26 23:44:16.922850485 +0000 UTC m=+21.098881625" observedRunningTime="2026-05-26 23:44:18.653337131 +0000 UTC m=+22.829368294" watchObservedRunningTime="2026-05-26 23:44:18.653975337 +0000 UTC m=+22.830006500"
May 26 23:44:18.929196 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:18.929175    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 26 23:44:19.370796 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:19.370675    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-26T23:44:18.9291914Z","UUID":"13b65b92-b86c-4ef7-adf5-063091874044","Handler":null,"Name":"","Endpoint":""}
May 26 23:44:19.372974 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:19.372922    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 26 23:44:19.372974 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:19.372960    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 26 23:44:19.427238 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:19.427206    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:19.427238 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:19.427225    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:19.427456 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:19.427362    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:19.427516 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:19.427495    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:19.630429 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:19.630349    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb" event={"ID":"c684f499-3fe8-4ae1-9c4b-8969786d1079","Type":"ContainerStarted","Data":"9770cf94ce8d712e963f9d725d35b0e9c27cb21205c1f04d7a93b28145a8e2c5"}
May 26 23:44:20.427507 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:20.427467    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:20.427797 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:20.427610    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zdft" podUID="2077a3a9-df5b-4a8c-8ed8-b7572ce31153"
May 26 23:44:20.634937 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:20.634884    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh" event={"ID":"fc54c161-1832-427d-b1b4-5e89b5f06223","Type":"ContainerStarted","Data":"4a6f3adf9ee9c8cd3559f287c998a01192bac96d4f9c891e593894c1c4b13822"}
May 26 23:44:21.427824 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:21.427784    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:21.427824 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:21.427801    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:21.428053 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:21.427915    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:21.428053 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:21.428008    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:22.014917 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:22.014876    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:22.015307 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:22.015005    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 26 23:44:22.015307 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:22.015079    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret podName:2077a3a9-df5b-4a8c-8ed8-b7572ce31153 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:38.015059849 +0000 UTC m=+42.191090988 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret") pod "global-pull-secret-syncer-2zdft" (UID: "2077a3a9-df5b-4a8c-8ed8-b7572ce31153") : object "kube-system"/"original-pull-secret" not registered
May 26 23:44:22.427819 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:22.427650    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:22.427984 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:22.427913    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zdft" podUID="2077a3a9-df5b-4a8c-8ed8-b7572ce31153"
May 26 23:44:22.642631 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:22.642598    2566 generic.go:358] "Generic (PLEG): container finished" podID="8e2588a1-9d18-4ea2-8ec4-e14b672058b4" containerID="3403f934701d93a42147c518dedae2ebb4ecf42cb52b3c6028dc2f9dd37d0683" exitCode=0
May 26 23:44:22.642800 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:22.642691    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjq8g" event={"ID":"8e2588a1-9d18-4ea2-8ec4-e14b672058b4","Type":"ContainerDied","Data":"3403f934701d93a42147c518dedae2ebb4ecf42cb52b3c6028dc2f9dd37d0683"}
May 26 23:44:22.645946 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:22.645921    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh" event={"ID":"fc54c161-1832-427d-b1b4-5e89b5f06223","Type":"ContainerStarted","Data":"0c4872ff4b152bccc7c8813e2734f20a54822ca17d0ab508e77407913a8d9971"}
May 26 23:44:22.646258 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:22.646238    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:44:22.646343 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:22.646264    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:44:22.647848 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:22.647826    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb" event={"ID":"c684f499-3fe8-4ae1-9c4b-8969786d1079","Type":"ContainerStarted","Data":"4dc1d49008f8985ea6b40403f87753601957ece21204c0bebdcf2b20aaf3d264"}
May 26 23:44:22.661757 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:22.661740    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:44:22.685723 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:22.685688    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh" podStartSLOduration=7.186974105 podStartE2EDuration="26.685677779s" podCreationTimestamp="2026-05-26 23:43:56 +0000 UTC" firstStartedPulling="2026-05-26 23:43:57.463837779 +0000 UTC m=+1.639868919" lastFinishedPulling="2026-05-26 23:44:16.962541445 +0000 UTC m=+21.138572593" observedRunningTime="2026-05-26 23:44:22.685413419 +0000 UTC m=+26.861444580" watchObservedRunningTime="2026-05-26 23:44:22.685677779 +0000 UTC m=+26.861708962"
May 26 23:44:22.700513 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:22.700479    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjxxb" podStartSLOduration=2.363152745 podStartE2EDuration="26.700470663s" podCreationTimestamp="2026-05-26 23:43:56 +0000 UTC" firstStartedPulling="2026-05-26 23:43:57.658489983 +0000 UTC m=+1.834521126" lastFinishedPulling="2026-05-26 23:44:21.995807905 +0000 UTC m=+26.171839044" observedRunningTime="2026-05-26 23:44:22.700132072 +0000 UTC m=+26.876163233" watchObservedRunningTime="2026-05-26 23:44:22.700470663 +0000 UTC m=+26.876501824"
May 26 23:44:23.427658 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:23.427627    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:23.428489 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:23.427625    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:23.428489 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:23.427731    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:23.428489 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:23.427812    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:23.649974 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:23.649945    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:44:23.664961 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:23.664887    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:44:24.235455 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:24.235258    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2zdft"]
May 26 23:44:24.235580 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:24.235551    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:24.235678 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:24.235658    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zdft" podUID="2077a3a9-df5b-4a8c-8ed8-b7572ce31153"
May 26 23:44:24.238496 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:24.238471    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6mdld"]
May 26 23:44:24.238635 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:24.238552    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:24.238695 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:24.238633    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:24.239060 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:24.239041    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fplbc"]
May 26 23:44:24.239162 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:24.239121    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:24.239224 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:24.239207    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:24.653177 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:24.653146    2566 generic.go:358] "Generic (PLEG): container finished" podID="8e2588a1-9d18-4ea2-8ec4-e14b672058b4" containerID="18e4339e9d3b11fd064218b5aa9ea5f86123f23c643410d68216454e0e65ae2d" exitCode=0
May 26 23:44:24.653553 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:24.653234    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjq8g" event={"ID":"8e2588a1-9d18-4ea2-8ec4-e14b672058b4","Type":"ContainerDied","Data":"18e4339e9d3b11fd064218b5aa9ea5f86123f23c643410d68216454e0e65ae2d"}
May 26 23:44:25.427023 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:25.426992    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:25.427171 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:25.427094    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:26.428478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:26.428442    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:26.428876 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:26.428523    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:26.428876 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:26.428613    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:26.428876 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:26.428726    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zdft" podUID="2077a3a9-df5b-4a8c-8ed8-b7572ce31153"
May 26 23:44:26.660304 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:26.660021    2566 generic.go:358] "Generic (PLEG): container finished" podID="8e2588a1-9d18-4ea2-8ec4-e14b672058b4" containerID="4b40be19599d0a937d8861fb7aa6379f0bc38da7bdd78995ab19e7124da71439" exitCode=0
May 26 23:44:26.660304 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:26.660092    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjq8g" event={"ID":"8e2588a1-9d18-4ea2-8ec4-e14b672058b4","Type":"ContainerDied","Data":"4b40be19599d0a937d8861fb7aa6379f0bc38da7bdd78995ab19e7124da71439"}
May 26 23:44:27.427533 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:27.427502    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:27.427699 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:27.427624    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:28.427452 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:28.427418    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:28.427949 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:28.427416    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:28.427949 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:28.427548    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zdft" podUID="2077a3a9-df5b-4a8c-8ed8-b7572ce31153"
May 26 23:44:28.427949 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:28.427621    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:29.073743 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:29.073711    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7m2ct\" (UniqueName: \"kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct\") pod \"network-check-target-fplbc\" (UID: \"969b31ff-aeec-4ea2-aaae-9675876698b0\") " pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:29.073932 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:29.073813    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:29.073932 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:29.073857    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 26 23:44:29.073932 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:29.073880    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 26 23:44:29.073932 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:29.073910    2566 projected.go:194] Error preparing data for projected volume kube-api-access-7m2ct for pod openshift-network-diagnostics/network-check-target-fplbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:44:29.074091 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:29.073948    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:44:29.074091 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:29.073973    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct podName:969b31ff-aeec-4ea2-aaae-9675876698b0 nodeName:}" failed. No retries permitted until 2026-05-26 23:45:01.07395334 +0000 UTC m=+65.249984527 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7m2ct" (UniqueName: "kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct") pod "network-check-target-fplbc" (UID: "969b31ff-aeec-4ea2-aaae-9675876698b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 23:44:29.074091 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:29.074000    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs podName:6d68dfa4-c945-4139-b299-2664709a5651 nodeName:}" failed. No retries permitted until 2026-05-26 23:45:01.073983615 +0000 UTC m=+65.250014760 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs") pod "network-metrics-daemon-6mdld" (UID: "6d68dfa4-c945-4139-b299-2664709a5651") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 23:44:29.427519 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:29.427448    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:29.427972 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:29.427555    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:44:30.427771 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:30.427733    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:30.428293 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:30.427733    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:30.428293 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:30.427851    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zdft" podUID="2077a3a9-df5b-4a8c-8ed8-b7572ce31153"
May 26 23:44:30.428293 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:30.427981    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fplbc" podUID="969b31ff-aeec-4ea2-aaae-9675876698b0"
May 26 23:44:31.096269 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.096241    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-129.ec2.internal" event="NodeReady"
May 26 23:44:31.096463 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.096374    2566 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 26 23:44:31.131680 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.131648    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d69c8569-7g8xq"]
May 26 23:44:31.136020 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.135996    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.138614 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.138592    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t75lz\""
May 26 23:44:31.138614 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.138609    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 26 23:44:31.138766 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.138694    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 26 23:44:31.138838 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.138824    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 26 23:44:31.146311 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.145247    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 26 23:44:31.146311 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.145670    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d69c8569-7g8xq"]
May 26 23:44:31.149516 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.149496    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lzfn9"]
May 26 23:44:31.151637 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.151614    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9jd8w"]
May 26 23:44:31.151762 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.151746    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:31.154503 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.154483    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 26 23:44:31.154592 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.154533    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:44:31.155336 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.154966    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 26 23:44:31.155336 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.155168    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8pnjl\""
May 26 23:44:31.157239 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.157220    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 26 23:44:31.157379 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.157224    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 26 23:44:31.157574 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.157289    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gx4rr\""
May 26 23:44:31.157686 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.157572    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 26 23:44:31.162196 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.162161    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lzfn9"]
May 26 23:44:31.162918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.162881    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9jd8w"]
May 26 23:44:31.290990 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.290958    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c562bb9-35d6-43d8-9208-35fac148e6ec-image-registry-private-configuration\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.291136 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.290998    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txhlj\" (UniqueName: \"kubernetes.io/projected/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-kube-api-access-txhlj\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:44:31.291136 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.291096    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq2pj\" (UniqueName: \"kubernetes.io/projected/7f61ca38-a898-4dea-9951-8550a6fb2629-kube-api-access-xq2pj\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:31.291252 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.291137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c562bb9-35d6-43d8-9208-35fac148e6ec-installation-pull-secrets\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.291252 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.291170    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-bound-sa-token\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.291252 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.291219    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zchjw\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-kube-api-access-zchjw\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.291412 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.291259    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f61ca38-a898-4dea-9951-8550a6fb2629-tmp-dir\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:31.291412 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.291298    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.291412 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.291359    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c562bb9-35d6-43d8-9208-35fac148e6ec-ca-trust-extracted\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.291552 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.291424    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f61ca38-a898-4dea-9951-8550a6fb2629-config-volume\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:31.291552 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.291461    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c562bb9-35d6-43d8-9208-35fac148e6ec-trusted-ca\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.291552 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.291482    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:44:31.291552 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.291506    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-certificates\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.291708 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.291564    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:31.392270 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392191    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:31.392270 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392248    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c562bb9-35d6-43d8-9208-35fac148e6ec-image-registry-private-configuration\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.392487 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392283    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-txhlj\" (UniqueName: \"kubernetes.io/projected/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-kube-api-access-txhlj\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:44:31.392487 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392328    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xq2pj\" (UniqueName: \"kubernetes.io/projected/7f61ca38-a898-4dea-9951-8550a6fb2629-kube-api-access-xq2pj\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:31.392487 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.392334    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 23:44:31.392487 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392356    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c562bb9-35d6-43d8-9208-35fac148e6ec-installation-pull-secrets\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.392487 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392387    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-bound-sa-token\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.392487 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.392421    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls podName:7f61ca38-a898-4dea-9951-8550a6fb2629 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:31.892387721 +0000 UTC m=+36.068418866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls") pod "dns-default-lzfn9" (UID: "7f61ca38-a898-4dea-9951-8550a6fb2629") : secret "dns-default-metrics-tls" not found
May 26 23:44:31.392487 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392453    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zchjw\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-kube-api-access-zchjw\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.392487 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392481    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f61ca38-a898-4dea-9951-8550a6fb2629-tmp-dir\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:31.392828 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392506    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.392828 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392538    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c562bb9-35d6-43d8-9208-35fac148e6ec-ca-trust-extracted\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.392828 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392567    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f61ca38-a898-4dea-9951-8550a6fb2629-config-volume\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:31.392828 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392583    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c562bb9-35d6-43d8-9208-35fac148e6ec-trusted-ca\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.392828 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392598    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:44:31.392828 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.392615    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-certificates\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.393100 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.393007    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 26 23:44:31.393100 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.393022    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d69c8569-7g8xq: secret "image-registry-tls" not found
May 26 23:44:31.393100 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.393072    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls podName:5c562bb9-35d6-43d8-9208-35fac148e6ec nodeName:}" failed. No retries permitted until 2026-05-26 23:44:31.893055954 +0000 UTC m=+36.069087094 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls") pod "image-registry-d69c8569-7g8xq" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec") : secret "image-registry-tls" not found
May 26 23:44:31.393311 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.393292    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 23:44:31.393395 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.393358    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert podName:c09ef6c1-6d34-446d-ba8b-f0bcc511af50 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:31.893342731 +0000 UTC m=+36.069373885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert") pod "ingress-canary-9jd8w" (UID: "c09ef6c1-6d34-446d-ba8b-f0bcc511af50") : secret "canary-serving-cert" not found
May 26 23:44:31.393452 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.393433    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f61ca38-a898-4dea-9951-8550a6fb2629-tmp-dir\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:31.393730 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.393632    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f61ca38-a898-4dea-9951-8550a6fb2629-config-volume\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:31.393730 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.393688    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c562bb9-35d6-43d8-9208-35fac148e6ec-ca-trust-extracted\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.394069 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.394045    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-certificates\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.394231 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.394207    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c562bb9-35d6-43d8-9208-35fac148e6ec-trusted-ca\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.397484 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.397379    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c562bb9-35d6-43d8-9208-35fac148e6ec-image-registry-private-configuration\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.398590 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.398571    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c562bb9-35d6-43d8-9208-35fac148e6ec-installation-pull-secrets\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.403788 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.403767    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq2pj\" (UniqueName: \"kubernetes.io/projected/7f61ca38-a898-4dea-9951-8550a6fb2629-kube-api-access-xq2pj\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:31.404093 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.404068    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txhlj\" (UniqueName: \"kubernetes.io/projected/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-kube-api-access-txhlj\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:44:31.406503 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.406483    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-bound-sa-token\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.406591 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.406531    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zchjw\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-kube-api-access-zchjw\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.427537 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.427511    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:44:31.430022 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.430002    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 26 23:44:31.430336 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.430034    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9n4z9\""
May 26 23:44:31.898508 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.898472    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:31.898704 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.898532    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:44:31.898704 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:31.898600    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:31.898704 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.898633    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 26 23:44:31.898704 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.898655    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d69c8569-7g8xq: secret "image-registry-tls" not found
May 26 23:44:31.898887 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.898702    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 23:44:31.898887 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.898637    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 23:44:31.898887 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.898713    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls podName:5c562bb9-35d6-43d8-9208-35fac148e6ec nodeName:}" failed. No retries permitted until 2026-05-26 23:44:32.898693061 +0000 UTC m=+37.074724204 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls") pod "image-registry-d69c8569-7g8xq" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec") : secret "image-registry-tls" not found
May 26 23:44:31.898887 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.898784    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert podName:c09ef6c1-6d34-446d-ba8b-f0bcc511af50 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:32.898770698 +0000 UTC m=+37.074801840 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert") pod "ingress-canary-9jd8w" (UID: "c09ef6c1-6d34-446d-ba8b-f0bcc511af50") : secret "canary-serving-cert" not found
May 26 23:44:31.898887 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:31.898799    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls podName:7f61ca38-a898-4dea-9951-8550a6fb2629 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:32.898789545 +0000 UTC m=+37.074820685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls") pod "dns-default-lzfn9" (UID: "7f61ca38-a898-4dea-9951-8550a6fb2629") : secret "dns-default-metrics-tls" not found
May 26 23:44:32.427570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:32.427541    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:44:32.427941 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:32.427777    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:32.430286 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:32.430231    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 26 23:44:32.431106 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:32.431068    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-88rwp\""
May 26 23:44:32.431106 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:32.431092    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 26 23:44:32.431969 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:32.431948    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 26 23:44:32.673815 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:32.673782    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjq8g" event={"ID":"8e2588a1-9d18-4ea2-8ec4-e14b672058b4","Type":"ContainerStarted","Data":"8d73b7ba20e627e8efa4a18a452826fdb1fe775a37cce1bfb68a3235cc1746a2"}
May 26 23:44:32.906880 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:32.906822    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:32.906880 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:32.906864    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:44:32.907072 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:32.906955    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:32.907072 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:32.906965    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 23:44:32.907072 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:32.906967    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 26 23:44:32.907072 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:32.906987    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d69c8569-7g8xq: secret "image-registry-tls" not found
May 26 23:44:32.907072 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:32.907012    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert podName:c09ef6c1-6d34-446d-ba8b-f0bcc511af50 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:34.906997651 +0000 UTC m=+39.083028790 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert") pod "ingress-canary-9jd8w" (UID: "c09ef6c1-6d34-446d-ba8b-f0bcc511af50") : secret "canary-serving-cert" not found
May 26 23:44:32.907072 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:32.907030    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls podName:5c562bb9-35d6-43d8-9208-35fac148e6ec nodeName:}" failed. No retries permitted until 2026-05-26 23:44:34.907018217 +0000 UTC m=+39.083049356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls") pod "image-registry-d69c8569-7g8xq" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec") : secret "image-registry-tls" not found
May 26 23:44:32.907336 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:32.907099    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 23:44:32.907336 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:32.907156    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls podName:7f61ca38-a898-4dea-9951-8550a6fb2629 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:34.907140842 +0000 UTC m=+39.083171996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls") pod "dns-default-lzfn9" (UID: "7f61ca38-a898-4dea-9951-8550a6fb2629") : secret "dns-default-metrics-tls" not found
May 26 23:44:33.677471 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:33.677444    2566 generic.go:358] "Generic (PLEG): container finished" podID="8e2588a1-9d18-4ea2-8ec4-e14b672058b4" containerID="8d73b7ba20e627e8efa4a18a452826fdb1fe775a37cce1bfb68a3235cc1746a2" exitCode=0
May 26 23:44:33.677973 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:33.677496    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjq8g" event={"ID":"8e2588a1-9d18-4ea2-8ec4-e14b672058b4","Type":"ContainerDied","Data":"8d73b7ba20e627e8efa4a18a452826fdb1fe775a37cce1bfb68a3235cc1746a2"}
May 26 23:44:34.681529 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:34.681491    2566 generic.go:358] "Generic (PLEG): container finished" podID="8e2588a1-9d18-4ea2-8ec4-e14b672058b4" containerID="893aaba21701c9f5e8c17df4a99f10ce2a0cd0e7e6f63a0aef1c0d8987e48a3c" exitCode=0
May 26 23:44:34.682010 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:34.681551    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjq8g" event={"ID":"8e2588a1-9d18-4ea2-8ec4-e14b672058b4","Type":"ContainerDied","Data":"893aaba21701c9f5e8c17df4a99f10ce2a0cd0e7e6f63a0aef1c0d8987e48a3c"}
May 26 23:44:34.923595 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:34.923558    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:34.923732 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:34.923613    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:44:34.923732 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:34.923662    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:34.923732 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:34.923725    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 26 23:44:34.923832 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:34.923740    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d69c8569-7g8xq: secret "image-registry-tls" not found
May 26 23:44:34.923832 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:34.923744    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 23:44:34.923832 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:34.923756    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 23:44:34.923832 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:34.923792    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert podName:c09ef6c1-6d34-446d-ba8b-f0bcc511af50 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:38.923778081 +0000 UTC m=+43.099809222 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert") pod "ingress-canary-9jd8w" (UID: "c09ef6c1-6d34-446d-ba8b-f0bcc511af50") : secret "canary-serving-cert" not found
May 26 23:44:34.923832 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:34.923806    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls podName:5c562bb9-35d6-43d8-9208-35fac148e6ec nodeName:}" failed. No retries permitted until 2026-05-26 23:44:38.923799576 +0000 UTC m=+43.099830716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls") pod "image-registry-d69c8569-7g8xq" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec") : secret "image-registry-tls" not found
May 26 23:44:34.923832 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:34.923817    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls podName:7f61ca38-a898-4dea-9951-8550a6fb2629 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:38.923811984 +0000 UTC m=+43.099843123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls") pod "dns-default-lzfn9" (UID: "7f61ca38-a898-4dea-9951-8550a6fb2629") : secret "dns-default-metrics-tls" not found
May 26 23:44:35.686498 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:35.686464    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjq8g" event={"ID":"8e2588a1-9d18-4ea2-8ec4-e14b672058b4","Type":"ContainerStarted","Data":"40fec15fd7fdeeb2f2d4d9b54800d0506b9c772adfab4908c4a0e9bc7ae6d15f"}
May 26 23:44:35.710869 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:35.710812    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vjq8g" podStartSLOduration=4.955880946 podStartE2EDuration="39.71079676s" podCreationTimestamp="2026-05-26 23:43:56 +0000 UTC" firstStartedPulling="2026-05-26 23:43:57.642646436 +0000 UTC m=+1.818677576" lastFinishedPulling="2026-05-26 23:44:32.397562238 +0000 UTC m=+36.573593390" observedRunningTime="2026-05-26 23:44:35.710749268 +0000 UTC m=+39.886780429" watchObservedRunningTime="2026-05-26 23:44:35.71079676 +0000 UTC m=+39.886827924"
May 26 23:44:38.048373 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:38.048340    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:38.051018 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:38.050995    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2077a3a9-df5b-4a8c-8ed8-b7572ce31153-original-pull-secret\") pod \"global-pull-secret-syncer-2zdft\" (UID: \"2077a3a9-df5b-4a8c-8ed8-b7572ce31153\") " pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:38.149631 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:38.149605    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zdft"
May 26 23:44:38.329621 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:38.329593    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2zdft"]
May 26 23:44:38.332932 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:44:38.332888    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2077a3a9_df5b_4a8c_8ed8_b7572ce31153.slice/crio-5195321fde67f0cb9be15a8db03e68c5bfcd486ce2021e5551b902767f74b76a WatchSource:0}: Error finding container 5195321fde67f0cb9be15a8db03e68c5bfcd486ce2021e5551b902767f74b76a: Status 404 returned error can't find the container with id 5195321fde67f0cb9be15a8db03e68c5bfcd486ce2021e5551b902767f74b76a
May 26 23:44:38.692630 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:38.692535    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2zdft" event={"ID":"2077a3a9-df5b-4a8c-8ed8-b7572ce31153","Type":"ContainerStarted","Data":"5195321fde67f0cb9be15a8db03e68c5bfcd486ce2021e5551b902767f74b76a"}
May 26 23:44:38.953866 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:38.953783    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:44:38.953866 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:38.953839    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:38.954091 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:38.953919    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:38.954091 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:38.953953    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 23:44:38.954091 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:38.954023    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert podName:c09ef6c1-6d34-446d-ba8b-f0bcc511af50 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:46.954002636 +0000 UTC m=+51.130033782 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert") pod "ingress-canary-9jd8w" (UID: "c09ef6c1-6d34-446d-ba8b-f0bcc511af50") : secret "canary-serving-cert" not found
May 26 23:44:38.954091 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:38.954034    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 26 23:44:38.954091 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:38.954048    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d69c8569-7g8xq: secret "image-registry-tls" not found
May 26 23:44:38.954419 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:38.954105    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls podName:5c562bb9-35d6-43d8-9208-35fac148e6ec nodeName:}" failed. No retries permitted until 2026-05-26 23:44:46.954088784 +0000 UTC m=+51.130119934 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls") pod "image-registry-d69c8569-7g8xq" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec") : secret "image-registry-tls" not found
May 26 23:44:38.954419 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:38.954190    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 23:44:38.954419 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:38.954225    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls podName:7f61ca38-a898-4dea-9951-8550a6fb2629 nodeName:}" failed. No retries permitted until 2026-05-26 23:44:46.954214264 +0000 UTC m=+51.130245404 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls") pod "dns-default-lzfn9" (UID: "7f61ca38-a898-4dea-9951-8550a6fb2629") : secret "dns-default-metrics-tls" not found
May 26 23:44:42.702059 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:42.701985    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2zdft" event={"ID":"2077a3a9-df5b-4a8c-8ed8-b7572ce31153","Type":"ContainerStarted","Data":"76ab864f0625baafc5ecf220155ef94f901149dba6c94b2182b202ef021e11e3"}
May 26 23:44:42.715738 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:42.715688    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2zdft" podStartSLOduration=32.642619834 podStartE2EDuration="36.715674586s" podCreationTimestamp="2026-05-26 23:44:06 +0000 UTC" firstStartedPulling="2026-05-26 23:44:38.334584864 +0000 UTC m=+42.510616003" lastFinishedPulling="2026-05-26 23:44:42.407639611 +0000 UTC m=+46.583670755" observedRunningTime="2026-05-26 23:44:42.715449697 +0000 UTC m=+46.891480859" watchObservedRunningTime="2026-05-26 23:44:42.715674586 +0000 UTC m=+46.891705745"
May 26 23:44:47.008400 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:47.008364    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:44:47.008400 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:47.008413    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:44:47.008823 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:47.008464    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:44:47.008823 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:47.008513    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 23:44:47.008823 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:47.008552    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 26 23:44:47.008823 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:47.008562    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d69c8569-7g8xq: secret "image-registry-tls" not found
May 26 23:44:47.008823 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:47.008571    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 23:44:47.008823 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:47.008579    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert podName:c09ef6c1-6d34-446d-ba8b-f0bcc511af50 nodeName:}" failed. No retries permitted until 2026-05-26 23:45:03.008563816 +0000 UTC m=+67.184594956 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert") pod "ingress-canary-9jd8w" (UID: "c09ef6c1-6d34-446d-ba8b-f0bcc511af50") : secret "canary-serving-cert" not found
May 26 23:44:47.008823 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:47.008597    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls podName:5c562bb9-35d6-43d8-9208-35fac148e6ec nodeName:}" failed. No retries permitted until 2026-05-26 23:45:03.008586894 +0000 UTC m=+67.184618035 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls") pod "image-registry-d69c8569-7g8xq" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec") : secret "image-registry-tls" not found
May 26 23:44:47.008823 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:44:47.008609    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls podName:7f61ca38-a898-4dea-9951-8550a6fb2629 nodeName:}" failed. No retries permitted until 2026-05-26 23:45:03.008603754 +0000 UTC m=+67.184634894 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls") pod "dns-default-lzfn9" (UID: "7f61ca38-a898-4dea-9951-8550a6fb2629") : secret "dns-default-metrics-tls" not found
May 26 23:44:55.666036 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:44:55.666009    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nh9zh"
May 26 23:45:01.107631 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:01.107591    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:45:01.108194 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:01.107641    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7m2ct\" (UniqueName: \"kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct\") pod \"network-check-target-fplbc\" (UID: \"969b31ff-aeec-4ea2-aaae-9675876698b0\") " pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:45:01.110038 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:01.110019    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 26 23:45:01.110113 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:01.110074    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 26 23:45:01.118796 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:01.118769    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 26 23:45:01.118918 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:01.118830    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs podName:6d68dfa4-c945-4139-b299-2664709a5651 nodeName:}" failed. No retries permitted until 2026-05-26 23:46:05.118812992 +0000 UTC m=+129.294844132 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs") pod "network-metrics-daemon-6mdld" (UID: "6d68dfa4-c945-4139-b299-2664709a5651") : secret "metrics-daemon-secret" not found
May 26 23:45:01.121040 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:01.121016    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 26 23:45:01.131999 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:01.131978    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m2ct\" (UniqueName: \"kubernetes.io/projected/969b31ff-aeec-4ea2-aaae-9675876698b0-kube-api-access-7m2ct\") pod \"network-check-target-fplbc\" (UID: \"969b31ff-aeec-4ea2-aaae-9675876698b0\") " pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:45:01.245141 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:01.245116    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-88rwp\""
May 26 23:45:01.253030 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:01.253011    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:45:01.382287 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:01.382215    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fplbc"]
May 26 23:45:01.385557 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:45:01.385520    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969b31ff_aeec_4ea2_aaae_9675876698b0.slice/crio-e1c597041223d804511bafb2d7a80ab8555fa59cf4139ecc7f1674b80ab31c15 WatchSource:0}: Error finding container e1c597041223d804511bafb2d7a80ab8555fa59cf4139ecc7f1674b80ab31c15: Status 404 returned error can't find the container with id e1c597041223d804511bafb2d7a80ab8555fa59cf4139ecc7f1674b80ab31c15
May 26 23:45:01.739475 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:01.739386    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fplbc" event={"ID":"969b31ff-aeec-4ea2-aaae-9675876698b0","Type":"ContainerStarted","Data":"e1c597041223d804511bafb2d7a80ab8555fa59cf4139ecc7f1674b80ab31c15"}
May 26 23:45:03.031642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:03.031602    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:45:03.032114 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:03.031651    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:45:03.032114 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:03.031703    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:45:03.032114 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:03.031774    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 23:45:03.032114 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:03.031810    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 26 23:45:03.032114 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:03.031825    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d69c8569-7g8xq: secret "image-registry-tls" not found
May 26 23:45:03.032114 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:03.031831    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 23:45:03.032114 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:03.031855    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert podName:c09ef6c1-6d34-446d-ba8b-f0bcc511af50 nodeName:}" failed. No retries permitted until 2026-05-26 23:45:35.031833417 +0000 UTC m=+99.207864560 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert") pod "ingress-canary-9jd8w" (UID: "c09ef6c1-6d34-446d-ba8b-f0bcc511af50") : secret "canary-serving-cert" not found
May 26 23:45:03.032114 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:03.031875    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls podName:5c562bb9-35d6-43d8-9208-35fac148e6ec nodeName:}" failed. No retries permitted until 2026-05-26 23:45:35.031864769 +0000 UTC m=+99.207895914 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls") pod "image-registry-d69c8569-7g8xq" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec") : secret "image-registry-tls" not found
May 26 23:45:03.032114 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:03.031889    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls podName:7f61ca38-a898-4dea-9951-8550a6fb2629 nodeName:}" failed. No retries permitted until 2026-05-26 23:45:35.031881398 +0000 UTC m=+99.207912547 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls") pod "dns-default-lzfn9" (UID: "7f61ca38-a898-4dea-9951-8550a6fb2629") : secret "dns-default-metrics-tls" not found
May 26 23:45:04.746409 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:04.746311    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fplbc" event={"ID":"969b31ff-aeec-4ea2-aaae-9675876698b0","Type":"ContainerStarted","Data":"1cf4f9eba4076eb797d0f05f6a097843d4e5799d080445c57b2fa03287b68e1f"}
May 26 23:45:04.746776 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:04.746447    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:45:04.761660 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:04.761620    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fplbc" podStartSLOduration=65.697304375 podStartE2EDuration="1m8.761607497s" podCreationTimestamp="2026-05-26 23:43:56 +0000 UTC" firstStartedPulling="2026-05-26 23:45:01.387463297 +0000 UTC m=+65.563494437" lastFinishedPulling="2026-05-26 23:45:04.451766416 +0000 UTC m=+68.627797559" observedRunningTime="2026-05-26 23:45:04.760657624 +0000 UTC m=+68.936688786" watchObservedRunningTime="2026-05-26 23:45:04.761607497 +0000 UTC m=+68.937638659"
May 26 23:45:35.055707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:35.055664    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls\") pod \"image-registry-d69c8569-7g8xq\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") " pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:45:35.055707 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:35.055713    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:45:35.056203 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:35.055753    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:45:35.056203 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:35.055822    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 26 23:45:35.056203 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:35.055838    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 23:45:35.056203 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:35.055838    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d69c8569-7g8xq: secret "image-registry-tls" not found
May 26 23:45:35.056203 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:35.055877    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 23:45:35.056203 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:35.055935    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls podName:5c562bb9-35d6-43d8-9208-35fac148e6ec nodeName:}" failed. No retries permitted until 2026-05-26 23:46:39.055917013 +0000 UTC m=+163.231948163 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls") pod "image-registry-d69c8569-7g8xq" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec") : secret "image-registry-tls" not found
May 26 23:45:35.056203 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:35.055952    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls podName:7f61ca38-a898-4dea-9951-8550a6fb2629 nodeName:}" failed. No retries permitted until 2026-05-26 23:46:39.055946374 +0000 UTC m=+163.231977514 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls") pod "dns-default-lzfn9" (UID: "7f61ca38-a898-4dea-9951-8550a6fb2629") : secret "dns-default-metrics-tls" not found
May 26 23:45:35.056203 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:45:35.055963    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert podName:c09ef6c1-6d34-446d-ba8b-f0bcc511af50 nodeName:}" failed. No retries permitted until 2026-05-26 23:46:39.055957661 +0000 UTC m=+163.231988801 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert") pod "ingress-canary-9jd8w" (UID: "c09ef6c1-6d34-446d-ba8b-f0bcc511af50") : secret "canary-serving-cert" not found
May 26 23:45:35.751188 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:45:35.751162    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fplbc"
May 26 23:46:00.732826 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.732794    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-kwxzl"]
May 26 23:46:00.735364 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.735348    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:00.737790 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.737768    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
May 26 23:46:00.737790 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.737781    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
May 26 23:46:00.737975 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.737768    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
May 26 23:46:00.738665 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.738644    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rkrdw\""
May 26 23:46:00.738762 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.738700    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
May 26 23:46:00.746086 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.746066    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
May 26 23:46:00.746734 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.746714    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-kwxzl"]
May 26 23:46:00.826739 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.826714    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9375de25-ec32-4041-9c93-f241a8bc94ca-config\") pod \"console-operator-76dff48d59-kwxzl\" (UID: \"9375de25-ec32-4041-9c93-f241a8bc94ca\") " pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:00.826831 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.826743    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl278\" (UniqueName: \"kubernetes.io/projected/9375de25-ec32-4041-9c93-f241a8bc94ca-kube-api-access-zl278\") pod \"console-operator-76dff48d59-kwxzl\" (UID: \"9375de25-ec32-4041-9c93-f241a8bc94ca\") " pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:00.826831 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.826766    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9375de25-ec32-4041-9c93-f241a8bc94ca-trusted-ca\") pod \"console-operator-76dff48d59-kwxzl\" (UID: \"9375de25-ec32-4041-9c93-f241a8bc94ca\") " pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:00.826831 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.826825    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9375de25-ec32-4041-9c93-f241a8bc94ca-serving-cert\") pod \"console-operator-76dff48d59-kwxzl\" (UID: \"9375de25-ec32-4041-9c93-f241a8bc94ca\") " pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:00.927435 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.927411    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9375de25-ec32-4041-9c93-f241a8bc94ca-config\") pod \"console-operator-76dff48d59-kwxzl\" (UID: \"9375de25-ec32-4041-9c93-f241a8bc94ca\") " pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:00.927518 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.927440    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zl278\" (UniqueName: \"kubernetes.io/projected/9375de25-ec32-4041-9c93-f241a8bc94ca-kube-api-access-zl278\") pod \"console-operator-76dff48d59-kwxzl\" (UID: \"9375de25-ec32-4041-9c93-f241a8bc94ca\") " pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:00.927518 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.927466    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9375de25-ec32-4041-9c93-f241a8bc94ca-trusted-ca\") pod \"console-operator-76dff48d59-kwxzl\" (UID: \"9375de25-ec32-4041-9c93-f241a8bc94ca\") " pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:00.927667 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.927650    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9375de25-ec32-4041-9c93-f241a8bc94ca-serving-cert\") pod \"console-operator-76dff48d59-kwxzl\" (UID: \"9375de25-ec32-4041-9c93-f241a8bc94ca\") " pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:00.928216 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.928191    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9375de25-ec32-4041-9c93-f241a8bc94ca-config\") pod \"console-operator-76dff48d59-kwxzl\" (UID: \"9375de25-ec32-4041-9c93-f241a8bc94ca\") " pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:00.928338 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.928322    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9375de25-ec32-4041-9c93-f241a8bc94ca-trusted-ca\") pod \"console-operator-76dff48d59-kwxzl\" (UID: \"9375de25-ec32-4041-9c93-f241a8bc94ca\") " pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:00.930062 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.930035    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9375de25-ec32-4041-9c93-f241a8bc94ca-serving-cert\") pod \"console-operator-76dff48d59-kwxzl\" (UID: \"9375de25-ec32-4041-9c93-f241a8bc94ca\") " pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:00.934767 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:00.934747    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl278\" (UniqueName: \"kubernetes.io/projected/9375de25-ec32-4041-9c93-f241a8bc94ca-kube-api-access-zl278\") pod \"console-operator-76dff48d59-kwxzl\" (UID: \"9375de25-ec32-4041-9c93-f241a8bc94ca\") " pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:01.045620 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:01.045547    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:01.161455 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:01.161427    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-kwxzl"]
May 26 23:46:01.165097 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:01.165069    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9375de25_ec32_4041_9c93_f241a8bc94ca.slice/crio-d6bde6d23af8057bf24497dd5e8e87ac916ffb335f59a2fb15fd52e22f0c53eb WatchSource:0}: Error finding container d6bde6d23af8057bf24497dd5e8e87ac916ffb335f59a2fb15fd52e22f0c53eb: Status 404 returned error can't find the container with id d6bde6d23af8057bf24497dd5e8e87ac916ffb335f59a2fb15fd52e22f0c53eb
May 26 23:46:01.854916 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:01.854859    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl" event={"ID":"9375de25-ec32-4041-9c93-f241a8bc94ca","Type":"ContainerStarted","Data":"d6bde6d23af8057bf24497dd5e8e87ac916ffb335f59a2fb15fd52e22f0c53eb"}
May 26 23:46:03.859798 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:03.859772    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/0.log"
May 26 23:46:03.860201 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:03.859811    2566 generic.go:358] "Generic (PLEG): container finished" podID="9375de25-ec32-4041-9c93-f241a8bc94ca" containerID="12bf8b58d810ba14f17493c7440504fe884978bd362a7b046e8f3f8c76d993e2" exitCode=255
May 26 23:46:03.860201 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:03.859870    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl" event={"ID":"9375de25-ec32-4041-9c93-f241a8bc94ca","Type":"ContainerDied","Data":"12bf8b58d810ba14f17493c7440504fe884978bd362a7b046e8f3f8c76d993e2"}
May 26 23:46:03.860201 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:03.860082    2566 scope.go:117] "RemoveContainer" containerID="12bf8b58d810ba14f17493c7440504fe884978bd362a7b046e8f3f8c76d993e2"
May 26 23:46:04.495744 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.495720    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2bg9t_f31218b0-fbd7-43cf-b279-47cc71a12652/dns-node-resolver/0.log"
May 26 23:46:04.735435 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.735396    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"]
May 26 23:46:04.738363 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.738347    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"
May 26 23:46:04.740604 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.740578    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
May 26 23:46:04.740604 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.740594    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
May 26 23:46:04.740870 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.740700    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
May 26 23:46:04.740980 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.740966    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
May 26 23:46:04.741609 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.741595    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-xb422\""
May 26 23:46:04.750888 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.749473    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"]
May 26 23:46:04.855195 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.855165    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6000982f-147b-41ee-ba1f-00ccc23543a2-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-d4prq\" (UID: \"6000982f-147b-41ee-ba1f-00ccc23543a2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"
May 26 23:46:04.855376 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.855201    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6000982f-147b-41ee-ba1f-00ccc23543a2-config\") pod \"kube-storage-version-migrator-operator-565644f49-d4prq\" (UID: \"6000982f-147b-41ee-ba1f-00ccc23543a2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"
May 26 23:46:04.855376 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.855304    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddpfw\" (UniqueName: \"kubernetes.io/projected/6000982f-147b-41ee-ba1f-00ccc23543a2-kube-api-access-ddpfw\") pod \"kube-storage-version-migrator-operator-565644f49-d4prq\" (UID: \"6000982f-147b-41ee-ba1f-00ccc23543a2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"
May 26 23:46:04.862672 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.862653    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 26 23:46:04.863087 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.863020    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/0.log"
May 26 23:46:04.863087 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.863062    2566 generic.go:358] "Generic (PLEG): container finished" podID="9375de25-ec32-4041-9c93-f241a8bc94ca" containerID="3348a85eb7221c96943801e1d9015d9c2835a9a3305cb53ebe637c1015808888" exitCode=255
May 26 23:46:04.863196 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.863107    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl" event={"ID":"9375de25-ec32-4041-9c93-f241a8bc94ca","Type":"ContainerDied","Data":"3348a85eb7221c96943801e1d9015d9c2835a9a3305cb53ebe637c1015808888"}
May 26 23:46:04.863196 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.863142    2566 scope.go:117] "RemoveContainer" containerID="12bf8b58d810ba14f17493c7440504fe884978bd362a7b046e8f3f8c76d993e2"
May 26 23:46:04.863394 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.863368    2566 scope.go:117] "RemoveContainer" containerID="3348a85eb7221c96943801e1d9015d9c2835a9a3305cb53ebe637c1015808888"
May 26 23:46:04.863620 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:04.863597    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-kwxzl_openshift-console-operator(9375de25-ec32-4041-9c93-f241a8bc94ca)\"" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl" podUID="9375de25-ec32-4041-9c93-f241a8bc94ca"
May 26 23:46:04.956608 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.956574    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ddpfw\" (UniqueName: \"kubernetes.io/projected/6000982f-147b-41ee-ba1f-00ccc23543a2-kube-api-access-ddpfw\") pod \"kube-storage-version-migrator-operator-565644f49-d4prq\" (UID: \"6000982f-147b-41ee-ba1f-00ccc23543a2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"
May 26 23:46:04.956785 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.956625    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6000982f-147b-41ee-ba1f-00ccc23543a2-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-d4prq\" (UID: \"6000982f-147b-41ee-ba1f-00ccc23543a2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"
May 26 23:46:04.956785 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.956643    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6000982f-147b-41ee-ba1f-00ccc23543a2-config\") pod \"kube-storage-version-migrator-operator-565644f49-d4prq\" (UID: \"6000982f-147b-41ee-ba1f-00ccc23543a2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"
May 26 23:46:04.957177 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.957160    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6000982f-147b-41ee-ba1f-00ccc23543a2-config\") pod \"kube-storage-version-migrator-operator-565644f49-d4prq\" (UID: \"6000982f-147b-41ee-ba1f-00ccc23543a2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"
May 26 23:46:04.958929 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.958885    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6000982f-147b-41ee-ba1f-00ccc23543a2-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-d4prq\" (UID: \"6000982f-147b-41ee-ba1f-00ccc23543a2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"
May 26 23:46:04.964275 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:04.964251    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddpfw\" (UniqueName: \"kubernetes.io/projected/6000982f-147b-41ee-ba1f-00ccc23543a2-kube-api-access-ddpfw\") pod \"kube-storage-version-migrator-operator-565644f49-d4prq\" (UID: \"6000982f-147b-41ee-ba1f-00ccc23543a2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"
May 26 23:46:05.046264 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:05.046204    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"
May 26 23:46:05.158427 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:05.158393    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:46:05.158562 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:05.158543    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 26 23:46:05.158628 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:05.158618    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs podName:6d68dfa4-c945-4139-b299-2664709a5651 nodeName:}" failed. No retries permitted until 2026-05-26 23:48:07.158596248 +0000 UTC m=+251.334627388 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs") pod "network-metrics-daemon-6mdld" (UID: "6d68dfa4-c945-4139-b299-2664709a5651") : secret "metrics-daemon-secret" not found
May 26 23:46:05.164756 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:05.164734    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq"]
May 26 23:46:05.167730 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:05.167708    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6000982f_147b_41ee_ba1f_00ccc23543a2.slice/crio-09ba80dcea0427ba929a4558824bfa75edddec389f6d71a09dd89e18b5de470d WatchSource:0}: Error finding container 09ba80dcea0427ba929a4558824bfa75edddec389f6d71a09dd89e18b5de470d: Status 404 returned error can't find the container with id 09ba80dcea0427ba929a4558824bfa75edddec389f6d71a09dd89e18b5de470d
May 26 23:46:05.866244 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:05.866220    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 26 23:46:05.866626 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:05.866544    2566 scope.go:117] "RemoveContainer" containerID="3348a85eb7221c96943801e1d9015d9c2835a9a3305cb53ebe637c1015808888"
May 26 23:46:05.866758 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:05.866739    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-kwxzl_openshift-console-operator(9375de25-ec32-4041-9c93-f241a8bc94ca)\"" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl" podUID="9375de25-ec32-4041-9c93-f241a8bc94ca"
May 26 23:46:05.867398 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:05.867370    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq" event={"ID":"6000982f-147b-41ee-ba1f-00ccc23543a2","Type":"ContainerStarted","Data":"09ba80dcea0427ba929a4558824bfa75edddec389f6d71a09dd89e18b5de470d"}
May 26 23:46:05.896011 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:05.895992    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vjw25_52bec009-08b7-4931-8c9f-50435f32eb50/node-ca/0.log"
May 26 23:46:07.872578 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:07.872538    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq" event={"ID":"6000982f-147b-41ee-ba1f-00ccc23543a2","Type":"ContainerStarted","Data":"cd9e030ebdd58d3feeb6b31aa324e0fbcd61bc052c65fbc127660d4c34372c8d"}
May 26 23:46:07.890180 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:07.888153    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq" podStartSLOduration=1.8169142169999999 podStartE2EDuration="3.888134746s" podCreationTimestamp="2026-05-26 23:46:04 +0000 UTC" firstStartedPulling="2026-05-26 23:46:05.16961825 +0000 UTC m=+129.345649393" lastFinishedPulling="2026-05-26 23:46:07.240838779 +0000 UTC m=+131.416869922" observedRunningTime="2026-05-26 23:46:07.886878463 +0000 UTC m=+132.062909626" watchObservedRunningTime="2026-05-26 23:46:07.888134746 +0000 UTC m=+132.064165910"
May 26 23:46:08.459865 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:08.459828    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-54hw7"]
May 26 23:46:08.463354 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:08.463329    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-54hw7"
May 26 23:46:08.465763 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:08.465737    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-dsb95\""
May 26 23:46:08.465862 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:08.465809    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
May 26 23:46:08.466592 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:08.466575    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
May 26 23:46:08.471108 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:08.471089    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-54hw7"]
May 26 23:46:08.484922 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:08.484883    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27lv8\" (UniqueName: \"kubernetes.io/projected/eea2ccfc-3469-4a45-b2dd-e099acb0bbd1-kube-api-access-27lv8\") pod \"migrator-b985f5d89-54hw7\" (UID: \"eea2ccfc-3469-4a45-b2dd-e099acb0bbd1\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-54hw7"
May 26 23:46:08.585791 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:08.585757    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-27lv8\" (UniqueName: \"kubernetes.io/projected/eea2ccfc-3469-4a45-b2dd-e099acb0bbd1-kube-api-access-27lv8\") pod \"migrator-b985f5d89-54hw7\" (UID: \"eea2ccfc-3469-4a45-b2dd-e099acb0bbd1\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-54hw7"
May 26 23:46:08.593290 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:08.593263    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27lv8\" (UniqueName: \"kubernetes.io/projected/eea2ccfc-3469-4a45-b2dd-e099acb0bbd1-kube-api-access-27lv8\") pod \"migrator-b985f5d89-54hw7\" (UID: \"eea2ccfc-3469-4a45-b2dd-e099acb0bbd1\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-54hw7"
May 26 23:46:08.773737 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:08.773657    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-54hw7"
May 26 23:46:08.892086 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:08.892062    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-54hw7"]
May 26 23:46:08.896022 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:08.895999    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeea2ccfc_3469_4a45_b2dd_e099acb0bbd1.slice/crio-fd42dc51b7105ecf1983d04dd286716e97933a6b449d2dc8629e3d7b3ec8dcd7 WatchSource:0}: Error finding container fd42dc51b7105ecf1983d04dd286716e97933a6b449d2dc8629e3d7b3ec8dcd7: Status 404 returned error can't find the container with id fd42dc51b7105ecf1983d04dd286716e97933a6b449d2dc8629e3d7b3ec8dcd7
May 26 23:46:09.877957 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:09.877918    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-54hw7" event={"ID":"eea2ccfc-3469-4a45-b2dd-e099acb0bbd1","Type":"ContainerStarted","Data":"fd42dc51b7105ecf1983d04dd286716e97933a6b449d2dc8629e3d7b3ec8dcd7"}
May 26 23:46:10.882264 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:10.882228    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-54hw7" event={"ID":"eea2ccfc-3469-4a45-b2dd-e099acb0bbd1","Type":"ContainerStarted","Data":"3bd9258e3649e5d4900bced367357b3803235e953a799d3727f246edefb7c66e"}
May 26 23:46:10.882625 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:10.882269    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-54hw7" event={"ID":"eea2ccfc-3469-4a45-b2dd-e099acb0bbd1","Type":"ContainerStarted","Data":"ffff2908c6241e8769bb360af4aa52b871dd666a6614967cabde9e727dd6f8e8"}
May 26 23:46:10.896978 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:10.896934    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-54hw7" podStartSLOduration=1.7414632129999998 podStartE2EDuration="2.89691922s" podCreationTimestamp="2026-05-26 23:46:08 +0000 UTC" firstStartedPulling="2026-05-26 23:46:08.897806287 +0000 UTC m=+133.073837431" lastFinishedPulling="2026-05-26 23:46:10.053262294 +0000 UTC m=+134.229293438" observedRunningTime="2026-05-26 23:46:10.895634996 +0000 UTC m=+135.071666158" watchObservedRunningTime="2026-05-26 23:46:10.89691922 +0000 UTC m=+135.072950376"
May 26 23:46:11.045780 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:11.045746    2566 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:11.045949 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:11.045786    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:11.046174 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:11.046156    2566 scope.go:117] "RemoveContainer" containerID="3348a85eb7221c96943801e1d9015d9c2835a9a3305cb53ebe637c1015808888"
May 26 23:46:11.046348 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:11.046329    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-kwxzl_openshift-console-operator(9375de25-ec32-4041-9c93-f241a8bc94ca)\"" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl" podUID="9375de25-ec32-4041-9c93-f241a8bc94ca"
May 26 23:46:14.028036 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.027998    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-9fh4q"]
May 26 23:46:14.030944 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.030922    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-9fh4q"
May 26 23:46:14.034838 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.034816    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
May 26 23:46:14.034943 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.034845    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
May 26 23:46:14.035767 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.035735    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
May 26 23:46:14.035870 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.035765    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
May 26 23:46:14.035870 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.035780    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-c8zbs\""
May 26 23:46:14.049577 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.049553    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-9fh4q"]
May 26 23:46:14.125855 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.125823    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpdp8\" (UniqueName: \"kubernetes.io/projected/20b6e215-dc0a-4c47-a22f-61e685bc3b55-kube-api-access-xpdp8\") pod \"service-ca-67d667c7d-9fh4q\" (UID: \"20b6e215-dc0a-4c47-a22f-61e685bc3b55\") " pod="openshift-service-ca/service-ca-67d667c7d-9fh4q"
May 26 23:46:14.126063 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.125889    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/20b6e215-dc0a-4c47-a22f-61e685bc3b55-signing-key\") pod \"service-ca-67d667c7d-9fh4q\" (UID: \"20b6e215-dc0a-4c47-a22f-61e685bc3b55\") " pod="openshift-service-ca/service-ca-67d667c7d-9fh4q"
May 26 23:46:14.126063 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.125945    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/20b6e215-dc0a-4c47-a22f-61e685bc3b55-signing-cabundle\") pod \"service-ca-67d667c7d-9fh4q\" (UID: \"20b6e215-dc0a-4c47-a22f-61e685bc3b55\") " pod="openshift-service-ca/service-ca-67d667c7d-9fh4q"
May 26 23:46:14.226227 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.226199    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xpdp8\" (UniqueName: \"kubernetes.io/projected/20b6e215-dc0a-4c47-a22f-61e685bc3b55-kube-api-access-xpdp8\") pod \"service-ca-67d667c7d-9fh4q\" (UID: \"20b6e215-dc0a-4c47-a22f-61e685bc3b55\") " pod="openshift-service-ca/service-ca-67d667c7d-9fh4q"
May 26 23:46:14.226327 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.226256    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/20b6e215-dc0a-4c47-a22f-61e685bc3b55-signing-key\") pod \"service-ca-67d667c7d-9fh4q\" (UID: \"20b6e215-dc0a-4c47-a22f-61e685bc3b55\") " pod="openshift-service-ca/service-ca-67d667c7d-9fh4q"
May 26 23:46:14.226327 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.226293    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/20b6e215-dc0a-4c47-a22f-61e685bc3b55-signing-cabundle\") pod \"service-ca-67d667c7d-9fh4q\" (UID: \"20b6e215-dc0a-4c47-a22f-61e685bc3b55\") " pod="openshift-service-ca/service-ca-67d667c7d-9fh4q"
May 26 23:46:14.226929 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.226892    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/20b6e215-dc0a-4c47-a22f-61e685bc3b55-signing-cabundle\") pod \"service-ca-67d667c7d-9fh4q\" (UID: \"20b6e215-dc0a-4c47-a22f-61e685bc3b55\") " pod="openshift-service-ca/service-ca-67d667c7d-9fh4q"
May 26 23:46:14.228751 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.228728    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/20b6e215-dc0a-4c47-a22f-61e685bc3b55-signing-key\") pod \"service-ca-67d667c7d-9fh4q\" (UID: \"20b6e215-dc0a-4c47-a22f-61e685bc3b55\") " pod="openshift-service-ca/service-ca-67d667c7d-9fh4q"
May 26 23:46:14.238547 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.238517    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpdp8\" (UniqueName: \"kubernetes.io/projected/20b6e215-dc0a-4c47-a22f-61e685bc3b55-kube-api-access-xpdp8\") pod \"service-ca-67d667c7d-9fh4q\" (UID: \"20b6e215-dc0a-4c47-a22f-61e685bc3b55\") " pod="openshift-service-ca/service-ca-67d667c7d-9fh4q"
May 26 23:46:14.339240 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.339212    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-9fh4q"
May 26 23:46:14.456255 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.456227    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-9fh4q"]
May 26 23:46:14.459339 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:14.459313    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20b6e215_dc0a_4c47_a22f_61e685bc3b55.slice/crio-f132951d29e05e193bae8d48042900e9d1290b3747b9cb31c2856ca33d43215c WatchSource:0}: Error finding container f132951d29e05e193bae8d48042900e9d1290b3747b9cb31c2856ca33d43215c: Status 404 returned error can't find the container with id f132951d29e05e193bae8d48042900e9d1290b3747b9cb31c2856ca33d43215c
May 26 23:46:14.892571 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:14.892531    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-9fh4q" event={"ID":"20b6e215-dc0a-4c47-a22f-61e685bc3b55","Type":"ContainerStarted","Data":"f132951d29e05e193bae8d48042900e9d1290b3747b9cb31c2856ca33d43215c"}
May 26 23:46:16.899438 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:16.899402    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-9fh4q" event={"ID":"20b6e215-dc0a-4c47-a22f-61e685bc3b55","Type":"ContainerStarted","Data":"75f90a42742c51c632ba6430a43b89d379418153fb9e75a19809d9cc9f9c7585"}
May 26 23:46:16.914082 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:16.914038    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-67d667c7d-9fh4q" podStartSLOduration=2.097074849 podStartE2EDuration="3.914026138s" podCreationTimestamp="2026-05-26 23:46:13 +0000 UTC" firstStartedPulling="2026-05-26 23:46:14.46118143 +0000 UTC m=+138.637212570" lastFinishedPulling="2026-05-26 23:46:16.278132705 +0000 UTC m=+140.454163859" observedRunningTime="2026-05-26 23:46:16.91362045 +0000 UTC m=+141.089651615" watchObservedRunningTime="2026-05-26 23:46:16.914026138 +0000 UTC m=+141.090057300"
May 26 23:46:22.429841 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:22.429808    2566 scope.go:117] "RemoveContainer" containerID="3348a85eb7221c96943801e1d9015d9c2835a9a3305cb53ebe637c1015808888"
May 26 23:46:22.915801 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:22.915771    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 26 23:46:22.915976 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:22.915857    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl" event={"ID":"9375de25-ec32-4041-9c93-f241a8bc94ca","Type":"ContainerStarted","Data":"fa3e4232b4ff888d23d8a378d60541e92a6b411a633aa2e1059f609ede3464c0"}
May 26 23:46:22.916229 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:22.916207    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:22.920685 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:22.920664    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl"
May 26 23:46:22.932191 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:22.932149    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-76dff48d59-kwxzl" podStartSLOduration=20.765796314 podStartE2EDuration="22.932137684s" podCreationTimestamp="2026-05-26 23:46:00 +0000 UTC" firstStartedPulling="2026-05-26 23:46:01.166955465 +0000 UTC m=+125.342986608" lastFinishedPulling="2026-05-26 23:46:03.333296835 +0000 UTC m=+127.509327978" observedRunningTime="2026-05-26 23:46:22.931171643 +0000 UTC m=+147.107202806" watchObservedRunningTime="2026-05-26 23:46:22.932137684 +0000 UTC m=+147.108168876"
May 26 23:46:34.149321 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:34.149276    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-d69c8569-7g8xq" podUID="5c562bb9-35d6-43d8-9208-35fac148e6ec"
May 26 23:46:34.162422 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:34.162395    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-lzfn9" podUID="7f61ca38-a898-4dea-9951-8550a6fb2629"
May 26 23:46:34.168546 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:34.168526    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-9jd8w" podUID="c09ef6c1-6d34-446d-ba8b-f0bcc511af50"
May 26 23:46:34.437855 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:34.437771    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6mdld" podUID="6d68dfa4-c945-4139-b299-2664709a5651"
May 26 23:46:34.943825 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:34.943791    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:46:34.944000 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:34.943791    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:46:34.944000 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:34.943801    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lzfn9"
May 26 23:46:36.721258 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.721221    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d69c8569-7g8xq"]
May 26 23:46:36.721632 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:36.721395    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-d69c8569-7g8xq" podUID="5c562bb9-35d6-43d8-9208-35fac148e6ec"
May 26 23:46:36.802865 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.802836    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-j244p"]
May 26 23:46:36.805594 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.805578    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-j244p"
May 26 23:46:36.807745 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.807722    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hfqm5\""
May 26 23:46:36.808609 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.808588    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
May 26 23:46:36.809267 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.809248    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-99c8764b-wrs4q"]
May 26 23:46:36.811077 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.811061    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.816666 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.816647    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-j244p"]
May 26 23:46:36.820877 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.820857    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-b8svw"]
May 26 23:46:36.822820 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.822802    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.825074 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.825055    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 26 23:46:36.825162 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.825095    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 26 23:46:36.825367 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.825352    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 26 23:46:36.825433 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.825409    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 26 23:46:36.825491 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.825474    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xdcqh\""
May 26 23:46:36.829486 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.829470    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-99c8764b-wrs4q"]
May 26 23:46:36.836852 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.836807    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b8svw"]
May 26 23:46:36.891590 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891555    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46c63e21-7e21-4033-96c3-697183ff9a0e-registry-tls\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.891708 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891595    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f3892b4-11de-40ef-8b92-a974a0082ecd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.891708 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891662    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f3892b4-11de-40ef-8b92-a974a0082ecd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.891708 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891690    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46c63e21-7e21-4033-96c3-697183ff9a0e-registry-certificates\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.891809 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891708    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46c63e21-7e21-4033-96c3-697183ff9a0e-trusted-ca\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.891809 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891743    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46c63e21-7e21-4033-96c3-697183ff9a0e-ca-trust-extracted\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.891809 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891779    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f3892b4-11de-40ef-8b92-a974a0082ecd-crio-socket\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.891809 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891798    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f3892b4-11de-40ef-8b92-a974a0082ecd-data-volume\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.891983 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891816    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46c63e21-7e21-4033-96c3-697183ff9a0e-installation-pull-secrets\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.891983 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891835    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46c63e21-7e21-4033-96c3-697183ff9a0e-bound-sa-token\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.891983 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891860    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv8qr\" (UniqueName: \"kubernetes.io/projected/46c63e21-7e21-4033-96c3-697183ff9a0e-kube-api-access-wv8qr\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.891983 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891884    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv8c8\" (UniqueName: \"kubernetes.io/projected/7f3892b4-11de-40ef-8b92-a974a0082ecd-kube-api-access-nv8c8\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.891983 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891944    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6e0c95d8-bccf-4800-8b47-0888e194da94-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-j244p\" (UID: \"6e0c95d8-bccf-4800-8b47-0888e194da94\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-j244p"
May 26 23:46:36.891983 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.891960    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46c63e21-7e21-4033-96c3-697183ff9a0e-image-registry-private-configuration\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.948440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.948414    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:46:36.952526 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.952508    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:46:36.992795 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.992734    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46c63e21-7e21-4033-96c3-697183ff9a0e-ca-trust-extracted\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.992795 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.992763    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f3892b4-11de-40ef-8b92-a974a0082ecd-crio-socket\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.992795 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.992778    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f3892b4-11de-40ef-8b92-a974a0082ecd-data-volume\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.993051 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.992798    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46c63e21-7e21-4033-96c3-697183ff9a0e-installation-pull-secrets\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.993051 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.992817    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46c63e21-7e21-4033-96c3-697183ff9a0e-bound-sa-token\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.993051 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.992843    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wv8qr\" (UniqueName: \"kubernetes.io/projected/46c63e21-7e21-4033-96c3-697183ff9a0e-kube-api-access-wv8qr\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.993051 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.992869    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nv8c8\" (UniqueName: \"kubernetes.io/projected/7f3892b4-11de-40ef-8b92-a974a0082ecd-kube-api-access-nv8c8\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.993051 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.992944    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f3892b4-11de-40ef-8b92-a974a0082ecd-crio-socket\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.993297 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.993066    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6e0c95d8-bccf-4800-8b47-0888e194da94-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-j244p\" (UID: \"6e0c95d8-bccf-4800-8b47-0888e194da94\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-j244p"
May 26 23:46:36.993297 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.993119    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46c63e21-7e21-4033-96c3-697183ff9a0e-image-registry-private-configuration\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.993297 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.993174    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46c63e21-7e21-4033-96c3-697183ff9a0e-registry-tls\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.993297 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.993188    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f3892b4-11de-40ef-8b92-a974a0082ecd-data-volume\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.993297 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.993208    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f3892b4-11de-40ef-8b92-a974a0082ecd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.993297 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.993215    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46c63e21-7e21-4033-96c3-697183ff9a0e-ca-trust-extracted\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.993592 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.993349    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f3892b4-11de-40ef-8b92-a974a0082ecd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.993592 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.993384    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46c63e21-7e21-4033-96c3-697183ff9a0e-registry-certificates\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.993592 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.993410    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46c63e21-7e21-4033-96c3-697183ff9a0e-trusted-ca\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.994148 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.994121    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f3892b4-11de-40ef-8b92-a974a0082ecd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.994469 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.994425    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46c63e21-7e21-4033-96c3-697183ff9a0e-registry-certificates\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.995491 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.995467    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46c63e21-7e21-4033-96c3-697183ff9a0e-trusted-ca\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.995736 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.995709    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46c63e21-7e21-4033-96c3-697183ff9a0e-installation-pull-secrets\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.996046 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.996019    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46c63e21-7e21-4033-96c3-697183ff9a0e-registry-tls\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.996137 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.996083    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f3892b4-11de-40ef-8b92-a974a0082ecd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:36.996323 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.996300    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46c63e21-7e21-4033-96c3-697183ff9a0e-image-registry-private-configuration\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:36.996704 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:36.996681    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6e0c95d8-bccf-4800-8b47-0888e194da94-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-j244p\" (UID: \"6e0c95d8-bccf-4800-8b47-0888e194da94\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-j244p"
May 26 23:46:37.013508 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.013476    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv8qr\" (UniqueName: \"kubernetes.io/projected/46c63e21-7e21-4033-96c3-697183ff9a0e-kube-api-access-wv8qr\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:37.013760 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.013741    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv8c8\" (UniqueName: \"kubernetes.io/projected/7f3892b4-11de-40ef-8b92-a974a0082ecd-kube-api-access-nv8c8\") pod \"insights-runtime-extractor-b8svw\" (UID: \"7f3892b4-11de-40ef-8b92-a974a0082ecd\") " pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:37.014945 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.014929    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46c63e21-7e21-4033-96c3-697183ff9a0e-bound-sa-token\") pod \"image-registry-99c8764b-wrs4q\" (UID: \"46c63e21-7e21-4033-96c3-697183ff9a0e\") " pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:37.093881 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.093843    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zchjw\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-kube-api-access-zchjw\") pod \"5c562bb9-35d6-43d8-9208-35fac148e6ec\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") "
May 26 23:46:37.093881 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.093880    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c562bb9-35d6-43d8-9208-35fac148e6ec-trusted-ca\") pod \"5c562bb9-35d6-43d8-9208-35fac148e6ec\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") "
May 26 23:46:37.094125 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.094033    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-bound-sa-token\") pod \"5c562bb9-35d6-43d8-9208-35fac148e6ec\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") "
May 26 23:46:37.094125 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.094116    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c562bb9-35d6-43d8-9208-35fac148e6ec-installation-pull-secrets\") pod \"5c562bb9-35d6-43d8-9208-35fac148e6ec\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") "
May 26 23:46:37.094225 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.094156    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c562bb9-35d6-43d8-9208-35fac148e6ec-image-registry-private-configuration\") pod \"5c562bb9-35d6-43d8-9208-35fac148e6ec\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") "
May 26 23:46:37.094225 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.094187    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c562bb9-35d6-43d8-9208-35fac148e6ec-ca-trust-extracted\") pod \"5c562bb9-35d6-43d8-9208-35fac148e6ec\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") "
May 26 23:46:37.094225 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.094216    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-certificates\") pod \"5c562bb9-35d6-43d8-9208-35fac148e6ec\" (UID: \"5c562bb9-35d6-43d8-9208-35fac148e6ec\") "
May 26 23:46:37.094459 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.094412    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c562bb9-35d6-43d8-9208-35fac148e6ec-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5c562bb9-35d6-43d8-9208-35fac148e6ec" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:46:37.094459 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.094443    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c562bb9-35d6-43d8-9208-35fac148e6ec-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5c562bb9-35d6-43d8-9208-35fac148e6ec" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 23:46:37.094591 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.094514    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c562bb9-35d6-43d8-9208-35fac148e6ec-ca-trust-extracted\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:46:37.094591 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.094535    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c562bb9-35d6-43d8-9208-35fac148e6ec-trusted-ca\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:46:37.094941 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.094890    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5c562bb9-35d6-43d8-9208-35fac148e6ec" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:46:37.096570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.096542    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c562bb9-35d6-43d8-9208-35fac148e6ec-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5c562bb9-35d6-43d8-9208-35fac148e6ec" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:46:37.096570 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.096559    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-kube-api-access-zchjw" (OuterVolumeSpecName: "kube-api-access-zchjw") pod "5c562bb9-35d6-43d8-9208-35fac148e6ec" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec"). InnerVolumeSpecName "kube-api-access-zchjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 23:46:37.096723 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.096581    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5c562bb9-35d6-43d8-9208-35fac148e6ec" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 23:46:37.096772 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.096751    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c562bb9-35d6-43d8-9208-35fac148e6ec-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5c562bb9-35d6-43d8-9208-35fac148e6ec" (UID: "5c562bb9-35d6-43d8-9208-35fac148e6ec"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:46:37.115919 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.115887    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-j244p"
May 26 23:46:37.123600 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.123584    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t75lz\""
May 26 23:46:37.131650 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.131631    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:37.131727 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.131706    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b8svw"
May 26 23:46:37.196284 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.196100    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-bound-sa-token\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:46:37.196284 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.196135    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c562bb9-35d6-43d8-9208-35fac148e6ec-installation-pull-secrets\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:46:37.196284 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.196152    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c562bb9-35d6-43d8-9208-35fac148e6ec-image-registry-private-configuration\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:46:37.196284 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.196169    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-certificates\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:46:37.196284 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.196183    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zchjw\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-kube-api-access-zchjw\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:46:37.260020 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:37.259975    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e0c95d8_bccf_4800_8b47_0888e194da94.slice/crio-b86735c9e5607eb0044c0c94e4084b11df6bf1545df48d5882f27d2173ff82f2 WatchSource:0}: Error finding container b86735c9e5607eb0044c0c94e4084b11df6bf1545df48d5882f27d2173ff82f2: Status 404 returned error can't find the container with id b86735c9e5607eb0044c0c94e4084b11df6bf1545df48d5882f27d2173ff82f2
May 26 23:46:37.260881 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.260802    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-j244p"]
May 26 23:46:37.277414 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.277388    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-99c8764b-wrs4q"]
May 26 23:46:37.280044 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:37.280020    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c63e21_7e21_4033_96c3_697183ff9a0e.slice/crio-1b9a4c33e778bc90a2c0872c1f72af9e7a56acd597650433ad72d12147844b7d WatchSource:0}: Error finding container 1b9a4c33e778bc90a2c0872c1f72af9e7a56acd597650433ad72d12147844b7d: Status 404 returned error can't find the container with id 1b9a4c33e778bc90a2c0872c1f72af9e7a56acd597650433ad72d12147844b7d
May 26 23:46:37.295463 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.295442    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b8svw"]
May 26 23:46:37.307855 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:37.307829    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f3892b4_11de_40ef_8b92_a974a0082ecd.slice/crio-d2a57f348db7171a7b6f67bdca7fc5c270a10ce3afffdab7fee3111f06e24bfc WatchSource:0}: Error finding container d2a57f348db7171a7b6f67bdca7fc5c270a10ce3afffdab7fee3111f06e24bfc: Status 404 returned error can't find the container with id d2a57f348db7171a7b6f67bdca7fc5c270a10ce3afffdab7fee3111f06e24bfc
May 26 23:46:37.952633 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.952600    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-99c8764b-wrs4q" event={"ID":"46c63e21-7e21-4033-96c3-697183ff9a0e","Type":"ContainerStarted","Data":"17338c72d0a4b92555b4944b7d2a54df84c586e585dafed9e5bdd427268ca7a2"}
May 26 23:46:37.953013 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.952640    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-99c8764b-wrs4q" event={"ID":"46c63e21-7e21-4033-96c3-697183ff9a0e","Type":"ContainerStarted","Data":"1b9a4c33e778bc90a2c0872c1f72af9e7a56acd597650433ad72d12147844b7d"}
May 26 23:46:37.953013 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.952668    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:46:37.953814 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.953783    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-j244p" event={"ID":"6e0c95d8-bccf-4800-8b47-0888e194da94","Type":"ContainerStarted","Data":"b86735c9e5607eb0044c0c94e4084b11df6bf1545df48d5882f27d2173ff82f2"}
May 26 23:46:37.955515 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.955495    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d69c8569-7g8xq"
May 26 23:46:37.955684 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.955656    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b8svw" event={"ID":"7f3892b4-11de-40ef-8b92-a974a0082ecd","Type":"ContainerStarted","Data":"e10515a933450b70bc4f3ad3b68cbc9392ea6abc04e5a24c19a70003af674d83"}
May 26 23:46:37.955784 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.955689    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b8svw" event={"ID":"7f3892b4-11de-40ef-8b92-a974a0082ecd","Type":"ContainerStarted","Data":"b5039d0516604ca9965b351ebd85607ab1626e9fdd0056e2c8eee4e4685cf3e8"}
May 26 23:46:37.955784 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.955705    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b8svw" event={"ID":"7f3892b4-11de-40ef-8b92-a974a0082ecd","Type":"ContainerStarted","Data":"d2a57f348db7171a7b6f67bdca7fc5c270a10ce3afffdab7fee3111f06e24bfc"}
May 26 23:46:37.971080 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.971043    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-99c8764b-wrs4q" podStartSLOduration=1.971030442 podStartE2EDuration="1.971030442s" podCreationTimestamp="2026-05-26 23:46:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 23:46:37.969453653 +0000 UTC m=+162.145484849" watchObservedRunningTime="2026-05-26 23:46:37.971030442 +0000 UTC m=+162.147061604"
May 26 23:46:37.994487 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.994464    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d69c8569-7g8xq"]
May 26 23:46:37.999217 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:37.999196    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d69c8569-7g8xq"]
May 26 23:46:38.104122 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:38.104054    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c562bb9-35d6-43d8-9208-35fac148e6ec-registry-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:46:38.430938 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:38.430852    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c562bb9-35d6-43d8-9208-35fac148e6ec" path="/var/lib/kubelet/pods/5c562bb9-35d6-43d8-9208-35fac148e6ec/volumes"
May 26 23:46:38.960265 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:38.960227    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-j244p" event={"ID":"6e0c95d8-bccf-4800-8b47-0888e194da94","Type":"ContainerStarted","Data":"3b3b5861f9bd1d95cc28f2fd8fe8ba249b869c3bb76d2dace8b73842592e2087"}
May 26 23:46:38.994928 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:38.994639    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-j244p" podStartSLOduration=2.014432003 podStartE2EDuration="2.994620676s" podCreationTimestamp="2026-05-26 23:46:36 +0000 UTC" firstStartedPulling="2026-05-26 23:46:37.26220027 +0000 UTC m=+161.438231411" lastFinishedPulling="2026-05-26 23:46:38.242388944 +0000 UTC m=+162.418420084" observedRunningTime="2026-05-26 23:46:38.993178813 +0000 UTC m=+163.169209975" watchObservedRunningTime="2026-05-26 23:46:38.994620676 +0000 UTC m=+163.170651839"
May 26 23:46:39.114179 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.114133    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:46:39.114356 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.114222    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:46:39.116926 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.116879    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f61ca38-a898-4dea-9951-8550a6fb2629-metrics-tls\") pod \"dns-default-lzfn9\" (UID: \"7f61ca38-a898-4dea-9951-8550a6fb2629\") " pod="openshift-dns/dns-default-lzfn9"
May 26 23:46:39.117064 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.117037    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c09ef6c1-6d34-446d-ba8b-f0bcc511af50-cert\") pod \"ingress-canary-9jd8w\" (UID: \"c09ef6c1-6d34-446d-ba8b-f0bcc511af50\") " pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:46:39.147809 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.147786    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8pnjl\""
May 26 23:46:39.147959 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.147814    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gx4rr\""
May 26 23:46:39.155964 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.155942    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9jd8w"
May 26 23:46:39.156073 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.156025    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lzfn9"
May 26 23:46:39.378679 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.378654    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lzfn9"]
May 26 23:46:39.382070 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:39.382045    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f61ca38_a898_4dea_9951_8550a6fb2629.slice/crio-b0777db12501eb7a7fde426f9c91b4db5a74ff78193328b34240ef3ee85bc6d0 WatchSource:0}: Error finding container b0777db12501eb7a7fde426f9c91b4db5a74ff78193328b34240ef3ee85bc6d0: Status 404 returned error can't find the container with id b0777db12501eb7a7fde426f9c91b4db5a74ff78193328b34240ef3ee85bc6d0
May 26 23:46:39.393211 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.393191    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9jd8w"]
May 26 23:46:39.395838 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:39.395817    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc09ef6c1_6d34_446d_ba8b_f0bcc511af50.slice/crio-6f4889a89234e34485f665e4c83ff0f4fd9d8a269c998417ee225ae78935a301 WatchSource:0}: Error finding container 6f4889a89234e34485f665e4c83ff0f4fd9d8a269c998417ee225ae78935a301: Status 404 returned error can't find the container with id 6f4889a89234e34485f665e4c83ff0f4fd9d8a269c998417ee225ae78935a301
May 26 23:46:39.966810 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.966771    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lzfn9" event={"ID":"7f61ca38-a898-4dea-9951-8550a6fb2629","Type":"ContainerStarted","Data":"b0777db12501eb7a7fde426f9c91b4db5a74ff78193328b34240ef3ee85bc6d0"}
May 26 23:46:39.967684 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.967663    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9jd8w" event={"ID":"c09ef6c1-6d34-446d-ba8b-f0bcc511af50","Type":"ContainerStarted","Data":"6f4889a89234e34485f665e4c83ff0f4fd9d8a269c998417ee225ae78935a301"}
May 26 23:46:39.969157 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.969131    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b8svw" event={"ID":"7f3892b4-11de-40ef-8b92-a974a0082ecd","Type":"ContainerStarted","Data":"f3df0079a1856e5cc8a9452553398b1ee9c1504aaf1d86c283bb81231d60ebdf"}
May 26 23:46:39.969391 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.969365    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-j244p"
May 26 23:46:39.973687 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.973669    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-j244p"
May 26 23:46:39.986923 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:39.986869    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-b8svw" podStartSLOduration=2.042494039 podStartE2EDuration="3.986854395s" podCreationTimestamp="2026-05-26 23:46:36 +0000 UTC" firstStartedPulling="2026-05-26 23:46:37.351480897 +0000 UTC m=+161.527512037" lastFinishedPulling="2026-05-26 23:46:39.295841251 +0000 UTC m=+163.471872393" observedRunningTime="2026-05-26 23:46:39.985215266 +0000 UTC m=+164.161246425" watchObservedRunningTime="2026-05-26 23:46:39.986854395 +0000 UTC m=+164.162885558"
May 26 23:46:40.268614 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.268526    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"]
May 26 23:46:40.272092 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.272060    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:40.274589 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.274563    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
May 26 23:46:40.275562 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.275529    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 26 23:46:40.275562 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.275549    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 26 23:46:40.275708 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.275700    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
May 26 23:46:40.275793 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.275753    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-6z7kl\""
May 26 23:46:40.275965 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.275942    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 26 23:46:40.279816 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.279777    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"]
May 26 23:46:40.423360 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.423099    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf6447a1-0f11-4d83-9518-853e223c171e-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-cw9c4\" (UID: \"cf6447a1-0f11-4d83-9518-853e223c171e\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:40.423360 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.423157    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf6447a1-0f11-4d83-9518-853e223c171e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-cw9c4\" (UID: \"cf6447a1-0f11-4d83-9518-853e223c171e\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:40.423360 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.423216    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf6447a1-0f11-4d83-9518-853e223c171e-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-cw9c4\" (UID: \"cf6447a1-0f11-4d83-9518-853e223c171e\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:40.423360 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.423248    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd9bm\" (UniqueName: \"kubernetes.io/projected/cf6447a1-0f11-4d83-9518-853e223c171e-kube-api-access-vd9bm\") pod \"prometheus-operator-7779bbdd78-cw9c4\" (UID: \"cf6447a1-0f11-4d83-9518-853e223c171e\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:40.523831 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.523751    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf6447a1-0f11-4d83-9518-853e223c171e-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-cw9c4\" (UID: \"cf6447a1-0f11-4d83-9518-853e223c171e\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:40.524010 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.523835    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf6447a1-0f11-4d83-9518-853e223c171e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-cw9c4\" (UID: \"cf6447a1-0f11-4d83-9518-853e223c171e\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:40.524010 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.523870    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf6447a1-0f11-4d83-9518-853e223c171e-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-cw9c4\" (UID: \"cf6447a1-0f11-4d83-9518-853e223c171e\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:40.524010 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:40.523882    2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
May 26 23:46:40.524010 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.523912    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vd9bm\" (UniqueName: \"kubernetes.io/projected/cf6447a1-0f11-4d83-9518-853e223c171e-kube-api-access-vd9bm\") pod \"prometheus-operator-7779bbdd78-cw9c4\" (UID: \"cf6447a1-0f11-4d83-9518-853e223c171e\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:40.524010 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:40.523974    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf6447a1-0f11-4d83-9518-853e223c171e-prometheus-operator-tls podName:cf6447a1-0f11-4d83-9518-853e223c171e nodeName:}" failed. No retries permitted until 2026-05-26 23:46:41.023949907 +0000 UTC m=+165.199981046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/cf6447a1-0f11-4d83-9518-853e223c171e-prometheus-operator-tls") pod "prometheus-operator-7779bbdd78-cw9c4" (UID: "cf6447a1-0f11-4d83-9518-853e223c171e") : secret "prometheus-operator-tls" not found
May 26 23:46:40.524860 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.524834    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf6447a1-0f11-4d83-9518-853e223c171e-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-cw9c4\" (UID: \"cf6447a1-0f11-4d83-9518-853e223c171e\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:40.527300 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.527275    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf6447a1-0f11-4d83-9518-853e223c171e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-cw9c4\" (UID: \"cf6447a1-0f11-4d83-9518-853e223c171e\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:40.532494 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.532470    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd9bm\" (UniqueName: \"kubernetes.io/projected/cf6447a1-0f11-4d83-9518-853e223c171e-kube-api-access-vd9bm\") pod \"prometheus-operator-7779bbdd78-cw9c4\" (UID: \"cf6447a1-0f11-4d83-9518-853e223c171e\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:40.973472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:40.973382    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lzfn9" event={"ID":"7f61ca38-a898-4dea-9951-8550a6fb2629","Type":"ContainerStarted","Data":"71c519e10bbf95409f7561e13ff817437a255582f67304bbdea0209a3d2a6d6e"}
May 26 23:46:41.029213 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:41.029181    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf6447a1-0f11-4d83-9518-853e223c171e-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-cw9c4\" (UID: \"cf6447a1-0f11-4d83-9518-853e223c171e\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:41.032334 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:41.032306    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf6447a1-0f11-4d83-9518-853e223c171e-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-cw9c4\" (UID: \"cf6447a1-0f11-4d83-9518-853e223c171e\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:41.185329 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:41.185297    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"
May 26 23:46:41.466589 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:41.466561    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4"]
May 26 23:46:41.469787 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:41.469762    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf6447a1_0f11_4d83_9518_853e223c171e.slice/crio-c1a9fb36f4a69c25b817be7a886b9dba9f86fbc4e0bd060edad180306af4459a WatchSource:0}: Error finding container c1a9fb36f4a69c25b817be7a886b9dba9f86fbc4e0bd060edad180306af4459a: Status 404 returned error can't find the container with id c1a9fb36f4a69c25b817be7a886b9dba9f86fbc4e0bd060edad180306af4459a
May 26 23:46:41.979706 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:41.979669    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lzfn9" event={"ID":"7f61ca38-a898-4dea-9951-8550a6fb2629","Type":"ContainerStarted","Data":"bd33b1212ca0ccb4d09210b6344fa3c9326ccf6ceffb5f48cfe4d573ab9b4896"}
May 26 23:46:41.980131 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:41.979826    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lzfn9"
May 26 23:46:41.981289 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:41.981265    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9jd8w" event={"ID":"c09ef6c1-6d34-446d-ba8b-f0bcc511af50","Type":"ContainerStarted","Data":"77cb23f9772b97d97a8d8ce073a8a8830a380e6eaa88954c1c8df148474f8575"}
May 26 23:46:41.982163 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:41.982142    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4" event={"ID":"cf6447a1-0f11-4d83-9518-853e223c171e","Type":"ContainerStarted","Data":"c1a9fb36f4a69c25b817be7a886b9dba9f86fbc4e0bd060edad180306af4459a"}
May 26 23:46:41.996046 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:41.996010    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-lzfn9" podStartSLOduration=129.582770286 podStartE2EDuration="2m10.996000304s" podCreationTimestamp="2026-05-26 23:44:31 +0000 UTC" firstStartedPulling="2026-05-26 23:46:39.383829479 +0000 UTC m=+163.559860619" lastFinishedPulling="2026-05-26 23:46:40.797059495 +0000 UTC m=+164.973090637" observedRunningTime="2026-05-26 23:46:41.995932051 +0000 UTC m=+166.171963212" watchObservedRunningTime="2026-05-26 23:46:41.996000304 +0000 UTC m=+166.172031465"
May 26 23:46:42.011192 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:42.011158    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9jd8w" podStartSLOduration=129.008439185 podStartE2EDuration="2m11.011147581s" podCreationTimestamp="2026-05-26 23:44:31 +0000 UTC" firstStartedPulling="2026-05-26 23:46:39.397556202 +0000 UTC m=+163.573587342" lastFinishedPulling="2026-05-26 23:46:41.400264575 +0000 UTC m=+165.576295738" observedRunningTime="2026-05-26 23:46:42.010170762 +0000 UTC m=+166.186201922" watchObservedRunningTime="2026-05-26 23:46:42.011147581 +0000 UTC m=+166.187178787"
May 26 23:46:42.987957 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:42.987886    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4" event={"ID":"cf6447a1-0f11-4d83-9518-853e223c171e","Type":"ContainerStarted","Data":"a2b6af2a504d11fefb816750aa04b8f56c82a679e5cc4d6aea9eb28187396c9a"}
May 26 23:46:42.987957 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:42.987958    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4" event={"ID":"cf6447a1-0f11-4d83-9518-853e223c171e","Type":"ContainerStarted","Data":"e94e9ecceed0bbd0e6a126c1f7b6bd5f67b5219ba7bf7b4f2bfa3bcdda77597a"}
May 26 23:46:43.004016 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:43.003965    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7779bbdd78-cw9c4" podStartSLOduration=1.635123427 podStartE2EDuration="3.003950516s" podCreationTimestamp="2026-05-26 23:46:40 +0000 UTC" firstStartedPulling="2026-05-26 23:46:41.471858866 +0000 UTC m=+165.647890006" lastFinishedPulling="2026-05-26 23:46:42.840685948 +0000 UTC m=+167.016717095" observedRunningTime="2026-05-26 23:46:43.003339699 +0000 UTC m=+167.179370859" watchObservedRunningTime="2026-05-26 23:46:43.003950516 +0000 UTC m=+167.179981679"
May 26 23:46:44.628658 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.628601    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"]
May 26 23:46:44.632211 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.632189    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-928r2"]
May 26 23:46:44.632395 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.632373    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.635092 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.635070    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
May 26 23:46:44.635223 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.635096    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-d64cd\""
May 26 23:46:44.635382 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.635357    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.636094 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.636071    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
May 26 23:46:44.636192 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.636101    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
May 26 23:46:44.637659 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.637642    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 26 23:46:44.637762 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.637741    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 26 23:46:44.638195 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.638175    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 26 23:46:44.638445 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.638425    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v42w5\""
May 26 23:46:44.642198 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.642178    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"]
May 26 23:46:44.759813 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.759777    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-accelerators-collector-config\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.759988 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.759826    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-textfile\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.759988 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.759880    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5172d29-f6c4-42e6-a5e7-845b3115ce06-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.759988 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.759946    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-root\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.759988 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.759975    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-tls\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.760191 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.760015    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-metrics-client-ca\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.760191 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.760050    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c5172d29-f6c4-42e6-a5e7-845b3115ce06-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.760191 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.760082    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5172d29-f6c4-42e6-a5e7-845b3115ce06-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.760191 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.760109    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.760191 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.760135    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-sys\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.760191 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.760164    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c5172d29-f6c4-42e6-a5e7-845b3115ce06-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.760191 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.760188    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-wtmp\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.760529 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.760236    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsrh2\" (UniqueName: \"kubernetes.io/projected/c5172d29-f6c4-42e6-a5e7-845b3115ce06-kube-api-access-bsrh2\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.760529 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.760287    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9mvc\" (UniqueName: \"kubernetes.io/projected/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-kube-api-access-v9mvc\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.760529 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.760320    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5172d29-f6c4-42e6-a5e7-845b3115ce06-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.860876 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.860838    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5172d29-f6c4-42e6-a5e7-845b3115ce06-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.861051 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.860892    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-accelerators-collector-config\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.861051 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.860939    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-textfile\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.861051 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.860969    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5172d29-f6c4-42e6-a5e7-845b3115ce06-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.861051 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.860995    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-root\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.861051 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.861020    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-tls\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.861051 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.861051    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-metrics-client-ca\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.861352 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.861085    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c5172d29-f6c4-42e6-a5e7-845b3115ce06-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.861352 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:44.861175    2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 26 23:46:44.861352 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:46:44.861292    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-tls podName:cd14e3a7-dd58-4b30-aa32-84deae62c9d5 nodeName:}" failed. No retries permitted until 2026-05-26 23:46:45.361262728 +0000 UTC m=+169.537293869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-tls") pod "node-exporter-928r2" (UID: "cd14e3a7-dd58-4b30-aa32-84deae62c9d5") : secret "node-exporter-tls" not found
May 26 23:46:44.861352 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.861325    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-textfile\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.861525 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.861329    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5172d29-f6c4-42e6-a5e7-845b3115ce06-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.861525 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.861417    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.861525 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.861450    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-sys\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.861525 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.861482    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c5172d29-f6c4-42e6-a5e7-845b3115ce06-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.861525 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.861506    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-wtmp\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.861726 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.861555    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bsrh2\" (UniqueName: \"kubernetes.io/projected/c5172d29-f6c4-42e6-a5e7-845b3115ce06-kube-api-access-bsrh2\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.861726 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.861603    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v9mvc\" (UniqueName: \"kubernetes.io/projected/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-kube-api-access-v9mvc\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.862118 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.861948    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5172d29-f6c4-42e6-a5e7-845b3115ce06-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.862118 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.862006    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-sys\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.862284 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.862181    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c5172d29-f6c4-42e6-a5e7-845b3115ce06-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.862346 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.862327    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-wtmp\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.862429 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.862409    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c5172d29-f6c4-42e6-a5e7-845b3115ce06-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.862524 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.862505    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-metrics-client-ca\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.862605 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.862585    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-root\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.862942 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.862873    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-accelerators-collector-config\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.863848 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.863820    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5172d29-f6c4-42e6-a5e7-845b3115ce06-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.863981 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.863921    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5172d29-f6c4-42e6-a5e7-845b3115ce06-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.865023 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.865005    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.870775 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.870745    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsrh2\" (UniqueName: \"kubernetes.io/projected/c5172d29-f6c4-42e6-a5e7-845b3115ce06-kube-api-access-bsrh2\") pod \"kube-state-metrics-54fd6bc5c4-pp9bb\" (UID: \"c5172d29-f6c4-42e6-a5e7-845b3115ce06\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:44.871375 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.871347    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9mvc\" (UniqueName: \"kubernetes.io/projected/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-kube-api-access-v9mvc\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:44.945457 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:44.945393    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"
May 26 23:46:45.079136 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:45.079101    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb"]
May 26 23:46:45.081983 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:45.081960    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5172d29_f6c4_42e6_a5e7_845b3115ce06.slice/crio-a9dd8b6c0493da11b252efa1fe77d9a40ddf1151e3132955a44344610b8e2060 WatchSource:0}: Error finding container a9dd8b6c0493da11b252efa1fe77d9a40ddf1151e3132955a44344610b8e2060: Status 404 returned error can't find the container with id a9dd8b6c0493da11b252efa1fe77d9a40ddf1151e3132955a44344610b8e2060
May 26 23:46:45.365355 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:45.365322    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-tls\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:45.367736 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:45.367709    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cd14e3a7-dd58-4b30-aa32-84deae62c9d5-node-exporter-tls\") pod \"node-exporter-928r2\" (UID: \"cd14e3a7-dd58-4b30-aa32-84deae62c9d5\") " pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:45.552044 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:45.552017    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-928r2"
May 26 23:46:45.562024 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:45.561985    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd14e3a7_dd58_4b30_aa32_84deae62c9d5.slice/crio-5f06f883fd1969599999390f1c8a86d59c81ba1926488eedecc6a35cc6ba3043 WatchSource:0}: Error finding container 5f06f883fd1969599999390f1c8a86d59c81ba1926488eedecc6a35cc6ba3043: Status 404 returned error can't find the container with id 5f06f883fd1969599999390f1c8a86d59c81ba1926488eedecc6a35cc6ba3043
May 26 23:46:45.998502 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:45.998463    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb" event={"ID":"c5172d29-f6c4-42e6-a5e7-845b3115ce06","Type":"ContainerStarted","Data":"a9dd8b6c0493da11b252efa1fe77d9a40ddf1151e3132955a44344610b8e2060"}
May 26 23:46:45.999565 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:45.999539    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-928r2" event={"ID":"cd14e3a7-dd58-4b30-aa32-84deae62c9d5","Type":"ContainerStarted","Data":"5f06f883fd1969599999390f1c8a86d59c81ba1926488eedecc6a35cc6ba3043"}
May 26 23:46:46.434118 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:46.434089    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:46:47.003817 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.003727    2566 generic.go:358] "Generic (PLEG): container finished" podID="cd14e3a7-dd58-4b30-aa32-84deae62c9d5" containerID="cf14778757a0b73eff1f1f9c864fb31ba02dc74ab411d449b6b32e62db0f99b9" exitCode=0
May 26 23:46:47.004268 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.003824    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-928r2" event={"ID":"cd14e3a7-dd58-4b30-aa32-84deae62c9d5","Type":"ContainerDied","Data":"cf14778757a0b73eff1f1f9c864fb31ba02dc74ab411d449b6b32e62db0f99b9"}
May 26 23:46:47.005742 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.005721    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb" event={"ID":"c5172d29-f6c4-42e6-a5e7-845b3115ce06","Type":"ContainerStarted","Data":"3b0c369a653c4a9f7a472b2a22cc16e05168f356e04c93a4f29e1b9ea4e046dc"}
May 26 23:46:47.005836 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.005750    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb" event={"ID":"c5172d29-f6c4-42e6-a5e7-845b3115ce06","Type":"ContainerStarted","Data":"80ce70ebe008f4a086bd4939654b830f5b1dff9e9c032fc7a3bc8ff6956c3cba"}
May 26 23:46:47.005836 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.005763    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb" event={"ID":"c5172d29-f6c4-42e6-a5e7-845b3115ce06","Type":"ContainerStarted","Data":"8b7d02ce6fb4cd50c3c9bda3730326468eb704851025d010a65d9a9f44c956ce"}
May 26 23:46:47.039534 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.039493    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-pp9bb" podStartSLOduration=1.547088426 podStartE2EDuration="3.039480254s" podCreationTimestamp="2026-05-26 23:46:44 +0000 UTC" firstStartedPulling="2026-05-26 23:46:45.084090579 +0000 UTC m=+169.260121737" lastFinishedPulling="2026-05-26 23:46:46.576482423 +0000 UTC m=+170.752513565" observedRunningTime="2026-05-26 23:46:47.038164064 +0000 UTC m=+171.214195224" watchObservedRunningTime="2026-05-26 23:46:47.039480254 +0000 UTC m=+171.215511415"
May 26 23:46:47.589523 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.589488    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-59cb967f8-vsddq"]
May 26 23:46:47.591977 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.591958    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.594248 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.594204    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
May 26 23:46:47.594357 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.594281    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
May 26 23:46:47.594357 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.594301    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
May 26 23:46:47.594506 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.594460    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
May 26 23:46:47.594748 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.594702    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
May 26 23:46:47.594748 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.594702    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2cmnm\""
May 26 23:46:47.594970 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.594838    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-f7a83q3p14frq\""
May 26 23:46:47.602937 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.602915    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-59cb967f8-vsddq"]
May 26 23:46:47.685514 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.685480    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c800763-6b48-4319-882c-ea1637f6b41b-metrics-client-ca\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.685649 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.685535    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.685649 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.685589    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmf5c\" (UniqueName: \"kubernetes.io/projected/0c800763-6b48-4319-882c-ea1637f6b41b-kube-api-access-lmf5c\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.685649 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.685636    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-tls\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.685765 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.685672    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-grpc-tls\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.685765 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.685688    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.685830 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.685768    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.685830 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.685794    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.786479 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.786451    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.786641 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.786484    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lmf5c\" (UniqueName: \"kubernetes.io/projected/0c800763-6b48-4319-882c-ea1637f6b41b-kube-api-access-lmf5c\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.786641 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.786508    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-tls\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.786641 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.786536    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-grpc-tls\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.786641 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.786562    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.786641 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.786623    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.786867 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.786794    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.786867 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.786845    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c800763-6b48-4319-882c-ea1637f6b41b-metrics-client-ca\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.787544 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.787523    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c800763-6b48-4319-882c-ea1637f6b41b-metrics-client-ca\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.789663 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.789636    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.789780 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.789688    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-grpc-tls\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.789780 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.789744    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-tls\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.789883 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.789744    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.789883 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.789849    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.789883 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.789849    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0c800763-6b48-4319-882c-ea1637f6b41b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.794094 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.794075    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmf5c\" (UniqueName: \"kubernetes.io/projected/0c800763-6b48-4319-882c-ea1637f6b41b-kube-api-access-lmf5c\") pod \"thanos-querier-59cb967f8-vsddq\" (UID: \"0c800763-6b48-4319-882c-ea1637f6b41b\") " pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:47.902097 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:47.902025    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:48.010378 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:48.010341    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-928r2" event={"ID":"cd14e3a7-dd58-4b30-aa32-84deae62c9d5","Type":"ContainerStarted","Data":"434108c98a91799794a5f787d092926eeed38eb33210e25a41a5e0b6f5117eaf"}
May 26 23:46:48.010720 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:48.010387    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-928r2" event={"ID":"cd14e3a7-dd58-4b30-aa32-84deae62c9d5","Type":"ContainerStarted","Data":"eec6681ad5a2f3219b2921b28dc39333eeea103b03c25a5fa431a1b537fe3c2b"}
May 26 23:46:48.025633 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:48.025600    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-59cb967f8-vsddq"]
May 26 23:46:48.029489 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:48.029443    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-928r2" podStartSLOduration=3.014829357 podStartE2EDuration="4.029426028s" podCreationTimestamp="2026-05-26 23:46:44 +0000 UTC" firstStartedPulling="2026-05-26 23:46:45.563935208 +0000 UTC m=+169.739966348" lastFinishedPulling="2026-05-26 23:46:46.578531876 +0000 UTC m=+170.754563019" observedRunningTime="2026-05-26 23:46:48.028376134 +0000 UTC m=+172.204407296" watchObservedRunningTime="2026-05-26 23:46:48.029426028 +0000 UTC m=+172.205457190"
May 26 23:46:48.031267 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:48.031242    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c800763_6b48_4319_882c_ea1637f6b41b.slice/crio-48591bd1bc03e49840c411876fde39546ada85e1d45fb58f7824f985b262ed08 WatchSource:0}: Error finding container 48591bd1bc03e49840c411876fde39546ada85e1d45fb58f7824f985b262ed08: Status 404 returned error can't find the container with id 48591bd1bc03e49840c411876fde39546ada85e1d45fb58f7824f985b262ed08
May 26 23:46:49.014855 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.014820    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq" event={"ID":"0c800763-6b48-4319-882c-ea1637f6b41b","Type":"ContainerStarted","Data":"48591bd1bc03e49840c411876fde39546ada85e1d45fb58f7824f985b262ed08"}
May 26 23:46:49.018704 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.018685    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6f796f9c6-2bvsh"]
May 26 23:46:49.021378 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.021360    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.024829 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.024798    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
May 26 23:46:49.024953 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.024870    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dd7m4\""
May 26 23:46:49.024953 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.024814    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2cd7a807o550u\""
May 26 23:46:49.025077 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.024978    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
May 26 23:46:49.025077 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.024871    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 26 23:46:49.025077 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.024798    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
May 26 23:46:49.032242 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.032220    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f796f9c6-2bvsh"]
May 26 23:46:49.098104 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.098075    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83683ed4-eb7a-41cc-8820-b576c4692611-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.098259 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.098128    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/83683ed4-eb7a-41cc-8820-b576c4692611-audit-log\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.098259 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.098175    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83683ed4-eb7a-41cc-8820-b576c4692611-client-ca-bundle\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.098367 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.098297    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjfvg\" (UniqueName: \"kubernetes.io/projected/83683ed4-eb7a-41cc-8820-b576c4692611-kube-api-access-gjfvg\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.098434 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.098405    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/83683ed4-eb7a-41cc-8820-b576c4692611-secret-metrics-server-tls\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.098498 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.098483    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/83683ed4-eb7a-41cc-8820-b576c4692611-metrics-server-audit-profiles\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.098552 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.098536    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/83683ed4-eb7a-41cc-8820-b576c4692611-secret-metrics-server-client-certs\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.199276 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.199245    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83683ed4-eb7a-41cc-8820-b576c4692611-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.199451 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.199291    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/83683ed4-eb7a-41cc-8820-b576c4692611-audit-log\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.199451 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.199328    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83683ed4-eb7a-41cc-8820-b576c4692611-client-ca-bundle\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.199451 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.199345    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gjfvg\" (UniqueName: \"kubernetes.io/projected/83683ed4-eb7a-41cc-8820-b576c4692611-kube-api-access-gjfvg\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.199451 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.199370    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/83683ed4-eb7a-41cc-8820-b576c4692611-secret-metrics-server-tls\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.199451 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.199416    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/83683ed4-eb7a-41cc-8820-b576c4692611-metrics-server-audit-profiles\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.199833 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.199804    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/83683ed4-eb7a-41cc-8820-b576c4692611-audit-log\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.199930 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.199806    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/83683ed4-eb7a-41cc-8820-b576c4692611-secret-metrics-server-client-certs\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.200086 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.200066    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83683ed4-eb7a-41cc-8820-b576c4692611-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.200642 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.200616    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/83683ed4-eb7a-41cc-8820-b576c4692611-metrics-server-audit-profiles\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.202365 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.202346    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/83683ed4-eb7a-41cc-8820-b576c4692611-secret-metrics-server-tls\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.202459 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.202445    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83683ed4-eb7a-41cc-8820-b576c4692611-client-ca-bundle\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.202575 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.202555    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/83683ed4-eb7a-41cc-8820-b576c4692611-secret-metrics-server-client-certs\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.207369 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.207350    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjfvg\" (UniqueName: \"kubernetes.io/projected/83683ed4-eb7a-41cc-8820-b576c4692611-kube-api-access-gjfvg\") pod \"metrics-server-6f796f9c6-2bvsh\" (UID: \"83683ed4-eb7a-41cc-8820-b576c4692611\") " pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.332961 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.332922    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:46:49.406886 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.406850    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-x964q"]
May 26 23:46:49.414440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.414414    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-x964q"]
May 26 23:46:49.414625 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.414554    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-x964q"
May 26 23:46:49.417487 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.417015    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-mx774\""
May 26 23:46:49.417487 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.417014    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
May 26 23:46:49.502643 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.502607    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d68b3b9b-2da4-4eeb-b951-d50657d5b23c-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-x964q\" (UID: \"d68b3b9b-2da4-4eeb-b951-d50657d5b23c\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-x964q"
May 26 23:46:49.603686 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.603609    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d68b3b9b-2da4-4eeb-b951-d50657d5b23c-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-x964q\" (UID: \"d68b3b9b-2da4-4eeb-b951-d50657d5b23c\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-x964q"
May 26 23:46:49.606078 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.606049    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d68b3b9b-2da4-4eeb-b951-d50657d5b23c-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-x964q\" (UID: \"d68b3b9b-2da4-4eeb-b951-d50657d5b23c\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-x964q"
May 26 23:46:49.727732 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.727696    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-x964q"
May 26 23:46:49.742170 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.742130    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f796f9c6-2bvsh"]
May 26 23:46:49.744172 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:49.744139    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83683ed4_eb7a_41cc_8820_b576c4692611.slice/crio-8d81cf8c7a4cc769956178e0fa0c4e1debead1eee51291437b9204e99217883b WatchSource:0}: Error finding container 8d81cf8c7a4cc769956178e0fa0c4e1debead1eee51291437b9204e99217883b: Status 404 returned error can't find the container with id 8d81cf8c7a4cc769956178e0fa0c4e1debead1eee51291437b9204e99217883b
May 26 23:46:49.875980 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:49.875950    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-x964q"]
May 26 23:46:49.880659 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:49.880635    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd68b3b9b_2da4_4eeb_b951_d50657d5b23c.slice/crio-92462eb3c913d21e9a1ed4be5cf44ad15036c83976696a1036b5a5f3a168bffd WatchSource:0}: Error finding container 92462eb3c913d21e9a1ed4be5cf44ad15036c83976696a1036b5a5f3a168bffd: Status 404 returned error can't find the container with id 92462eb3c913d21e9a1ed4be5cf44ad15036c83976696a1036b5a5f3a168bffd
May 26 23:46:50.018290 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:50.018256    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-x964q" event={"ID":"d68b3b9b-2da4-4eeb-b951-d50657d5b23c","Type":"ContainerStarted","Data":"92462eb3c913d21e9a1ed4be5cf44ad15036c83976696a1036b5a5f3a168bffd"}
May 26 23:46:50.019257 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:50.019225    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh" event={"ID":"83683ed4-eb7a-41cc-8820-b576c4692611","Type":"ContainerStarted","Data":"8d81cf8c7a4cc769956178e0fa0c4e1debead1eee51291437b9204e99217883b"}
May 26 23:46:50.020871 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:50.020851    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq" event={"ID":"0c800763-6b48-4319-882c-ea1637f6b41b","Type":"ContainerStarted","Data":"30c4cd9331c802f468f207974ef291c770b574d3d8d5e5d3464284555dfcc975"}
May 26 23:46:50.020978 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:50.020877    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq" event={"ID":"0c800763-6b48-4319-882c-ea1637f6b41b","Type":"ContainerStarted","Data":"60e1b3c4c4a9b7e62f307d3fb355c782031e61e0ed5d5b433a9eb782bc7b2173"}
May 26 23:46:50.020978 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:50.020892    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq" event={"ID":"0c800763-6b48-4319-882c-ea1637f6b41b","Type":"ContainerStarted","Data":"fe8072f12e19ee873bd682dd9199362ca1f21d066ff05b12085f0a2e0bf1b148"}
May 26 23:46:51.027957 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:51.027915    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq" event={"ID":"0c800763-6b48-4319-882c-ea1637f6b41b","Type":"ContainerStarted","Data":"2ac3b5fc79ae0315c2310866a9d06089b31cf6cd086ad1bc5fb6b5f482d4137d"}
May 26 23:46:51.027957 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:51.027966    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq" event={"ID":"0c800763-6b48-4319-882c-ea1637f6b41b","Type":"ContainerStarted","Data":"2f3b007db810c1ebe1ef77e4fbd25fa8a9bb329923fe7eef4f370e0d5d7165c7"}
May 26 23:46:51.027957 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:51.027981    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq" event={"ID":"0c800763-6b48-4319-882c-ea1637f6b41b","Type":"ContainerStarted","Data":"fc7c9c8a39af10805420b0692d81fc5b85fb2ed56e70148e7c3dd9b900688832"}
May 26 23:46:51.028797 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:51.028775    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:51.053186 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:51.053129    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq" podStartSLOduration=1.5346456769999999 podStartE2EDuration="4.053112485s" podCreationTimestamp="2026-05-26 23:46:47 +0000 UTC" firstStartedPulling="2026-05-26 23:46:48.03308224 +0000 UTC m=+172.209113379" lastFinishedPulling="2026-05-26 23:46:50.551549044 +0000 UTC m=+174.727580187" observedRunningTime="2026-05-26 23:46:51.051788981 +0000 UTC m=+175.227820151" watchObservedRunningTime="2026-05-26 23:46:51.053112485 +0000 UTC m=+175.229143646"
May 26 23:46:51.990518 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:51.990489    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lzfn9"
May 26 23:46:52.032296 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.032215    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-x964q" event={"ID":"d68b3b9b-2da4-4eeb-b951-d50657d5b23c","Type":"ContainerStarted","Data":"c1f76a35be771d6af307de5cb934eceb7b80ada4fd444e58f3dc8d33d5307f1c"}
May 26 23:46:52.032696 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.032560    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-x964q"
May 26 23:46:52.033833 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.033805    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh" event={"ID":"83683ed4-eb7a-41cc-8820-b576c4692611","Type":"ContainerStarted","Data":"02da45106e013c76070e66d142f8eba0c5025bd82ec4d9b7b4d278a5e08428a4"}
May 26 23:46:52.037911 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.037875    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-x964q"
May 26 23:46:52.047038 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.047005    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-665c9c8584-x964q" podStartSLOduration=1.148642706 podStartE2EDuration="3.046993219s" podCreationTimestamp="2026-05-26 23:46:49 +0000 UTC" firstStartedPulling="2026-05-26 23:46:49.882392653 +0000 UTC m=+174.058423792" lastFinishedPulling="2026-05-26 23:46:51.780743159 +0000 UTC m=+175.956774305" observedRunningTime="2026-05-26 23:46:52.046592406 +0000 UTC m=+176.222623569" watchObservedRunningTime="2026-05-26 23:46:52.046993219 +0000 UTC m=+176.223024378"
May 26 23:46:52.075439 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.075274    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh" podStartSLOduration=1.044587472 podStartE2EDuration="3.07525674s" podCreationTimestamp="2026-05-26 23:46:49 +0000 UTC" firstStartedPulling="2026-05-26 23:46:49.747522072 +0000 UTC m=+173.923553218" lastFinishedPulling="2026-05-26 23:46:51.778191331 +0000 UTC m=+175.954222486" observedRunningTime="2026-05-26 23:46:52.074161608 +0000 UTC m=+176.250192769" watchObservedRunningTime="2026-05-26 23:46:52.07525674 +0000 UTC m=+176.251287902"
May 26 23:46:52.987738 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.987703    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fd87d9cbb-lhftj"]
May 26 23:46:52.990968 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.990952    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:52.993322 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.993302    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 26 23:46:52.994308 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.994284    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 26 23:46:52.994411 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.994309    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 26 23:46:52.994411 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.994368    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 26 23:46:52.994411 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.994378    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 26 23:46:52.994564 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.994481    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 26 23:46:52.994564 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.994518    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 26 23:46:52.994705 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.994691    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qkjcw\""
May 26 23:46:52.998207 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:52.998185    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
May 26 23:46:53.000162 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.000139    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fd87d9cbb-lhftj"]
May 26 23:46:53.139492 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.139455    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/173609e0-4434-4c50-8331-6a3316ef3092-console-serving-cert\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.140026 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.139585    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/173609e0-4434-4c50-8331-6a3316ef3092-console-oauth-config\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.140026 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.139710    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-oauth-serving-cert\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.140026 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.139993    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-service-ca\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.140169 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.140032    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-console-config\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.140169 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.140077    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb7lg\" (UniqueName: \"kubernetes.io/projected/173609e0-4434-4c50-8331-6a3316ef3092-kube-api-access-cb7lg\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.140169 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.140098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-trusted-ca-bundle\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.240855 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.240775    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-service-ca\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.240855 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.240821    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-console-config\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.240855 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.240849    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cb7lg\" (UniqueName: \"kubernetes.io/projected/173609e0-4434-4c50-8331-6a3316ef3092-kube-api-access-cb7lg\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.241086 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.240877    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-trusted-ca-bundle\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.241086 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.240932    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/173609e0-4434-4c50-8331-6a3316ef3092-console-serving-cert\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.241086 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.240974    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/173609e0-4434-4c50-8331-6a3316ef3092-console-oauth-config\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.241086 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.241016    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-oauth-serving-cert\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.241542 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.241516    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-console-config\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.241631 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.241549    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-service-ca\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.241683 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.241647    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-trusted-ca-bundle\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.241918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.241879    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-oauth-serving-cert\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.243521 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.243499    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/173609e0-4434-4c50-8331-6a3316ef3092-console-oauth-config\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.243598 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.243560    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/173609e0-4434-4c50-8331-6a3316ef3092-console-serving-cert\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.248330 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.248309    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb7lg\" (UniqueName: \"kubernetes.io/projected/173609e0-4434-4c50-8331-6a3316ef3092-kube-api-access-cb7lg\") pod \"console-7fd87d9cbb-lhftj\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") " pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.302032 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.302005    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:46:53.437292 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:53.437257    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fd87d9cbb-lhftj"]
May 26 23:46:53.439753 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:46:53.439724    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod173609e0_4434_4c50_8331_6a3316ef3092.slice/crio-dfd54d6efa43fbd17b3f1b4eddfc7695f64bb67459bc74b6412a5828dd44db03 WatchSource:0}: Error finding container dfd54d6efa43fbd17b3f1b4eddfc7695f64bb67459bc74b6412a5828dd44db03: Status 404 returned error can't find the container with id dfd54d6efa43fbd17b3f1b4eddfc7695f64bb67459bc74b6412a5828dd44db03
May 26 23:46:54.041289 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:54.041222    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fd87d9cbb-lhftj" event={"ID":"173609e0-4434-4c50-8331-6a3316ef3092","Type":"ContainerStarted","Data":"dfd54d6efa43fbd17b3f1b4eddfc7695f64bb67459bc74b6412a5828dd44db03"}
May 26 23:46:56.047766 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:56.047736    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fd87d9cbb-lhftj" event={"ID":"173609e0-4434-4c50-8331-6a3316ef3092","Type":"ContainerStarted","Data":"d9a9d865213076040c49be4d7e5cb22a4591f9dee8d7db2072050a6cf80a87a3"}
May 26 23:46:57.135823 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:57.135792    2566 patch_prober.go:28] interesting pod/image-registry-99c8764b-wrs4q container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 26 23:46:57.136251 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:57.135843    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-99c8764b-wrs4q" podUID="46c63e21-7e21-4033-96c3-697183ff9a0e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 26 23:46:58.044052 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:58.044025    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-59cb967f8-vsddq"
May 26 23:46:58.066999 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:58.066947    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7fd87d9cbb-lhftj" podStartSLOduration=3.547286185 podStartE2EDuration="6.066934005s" podCreationTimestamp="2026-05-26 23:46:52 +0000 UTC" firstStartedPulling="2026-05-26 23:46:53.441725492 +0000 UTC m=+177.617756644" lastFinishedPulling="2026-05-26 23:46:55.961373322 +0000 UTC m=+180.137404464" observedRunningTime="2026-05-26 23:46:56.066790877 +0000 UTC m=+180.242822038" watchObservedRunningTime="2026-05-26 23:46:58.066934005 +0000 UTC m=+182.242965166"
May 26 23:46:58.964808 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:46:58.964777    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-99c8764b-wrs4q"
May 26 23:47:03.302308 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:47:03.302276    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:47:03.302703 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:47:03.302317    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:47:03.306919 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:47:03.306877    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:47:04.074123 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:47:04.074093    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:47:09.333361 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:47:09.333325    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:47:09.333361 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:47:09.333369    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:47:29.338398 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:47:29.338370    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:47:29.342391 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:47:29.342368    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6f796f9c6-2bvsh"
May 26 23:47:33.150840 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:47:33.150804    2566 generic.go:358] "Generic (PLEG): container finished" podID="6000982f-147b-41ee-ba1f-00ccc23543a2" containerID="cd9e030ebdd58d3feeb6b31aa324e0fbcd61bc052c65fbc127660d4c34372c8d" exitCode=0
May 26 23:47:33.151321 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:47:33.150885    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq" event={"ID":"6000982f-147b-41ee-ba1f-00ccc23543a2","Type":"ContainerDied","Data":"cd9e030ebdd58d3feeb6b31aa324e0fbcd61bc052c65fbc127660d4c34372c8d"}
May 26 23:47:33.151321 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:47:33.151274    2566 scope.go:117] "RemoveContainer" containerID="cd9e030ebdd58d3feeb6b31aa324e0fbcd61bc052c65fbc127660d4c34372c8d"
May 26 23:47:34.155301 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:47:34.155268    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-d4prq" event={"ID":"6000982f-147b-41ee-ba1f-00ccc23543a2","Type":"ContainerStarted","Data":"22be99b6090d3ddff6419b98d6f4e8b3c963be104c2f082b6c18d7123d0eeac0"}
May 26 23:48:07.179043 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:07.179001    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:48:07.181392 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:07.181368    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d68dfa4-c945-4139-b299-2664709a5651-metrics-certs\") pod \"network-metrics-daemon-6mdld\" (UID: \"6d68dfa4-c945-4139-b299-2664709a5651\") " pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:48:07.438410 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:07.438331    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9n4z9\""
May 26 23:48:07.446401 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:07.446373    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mdld"
May 26 23:48:07.569654 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:07.569545    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6mdld"]
May 26 23:48:07.572209 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:48:07.572180    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d68dfa4_c945_4139_b299_2664709a5651.slice/crio-f2434df10c39cf374f4548da60b103102efa29232a7961a991dc6e5f657ec03d WatchSource:0}: Error finding container f2434df10c39cf374f4548da60b103102efa29232a7961a991dc6e5f657ec03d: Status 404 returned error can't find the container with id f2434df10c39cf374f4548da60b103102efa29232a7961a991dc6e5f657ec03d
May 26 23:48:08.253347 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:08.253306    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6mdld" event={"ID":"6d68dfa4-c945-4139-b299-2664709a5651","Type":"ContainerStarted","Data":"f2434df10c39cf374f4548da60b103102efa29232a7961a991dc6e5f657ec03d"}
May 26 23:48:09.258634 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:09.258051    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6mdld" event={"ID":"6d68dfa4-c945-4139-b299-2664709a5651","Type":"ContainerStarted","Data":"2afcd4efa39b70f8038eb4f49835e01e8bdf4a546463ae9f7db5e929aabff4f8"}
May 26 23:48:09.258634 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:09.258095    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6mdld" event={"ID":"6d68dfa4-c945-4139-b299-2664709a5651","Type":"ContainerStarted","Data":"e7cca909055aff10c43999fb48236f105f7553aae34d0791310a6ddf4c552fc3"}
May 26 23:48:09.274494 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:09.274442    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6mdld" podStartSLOduration=252.207701707 podStartE2EDuration="4m13.27442334s" podCreationTimestamp="2026-05-26 23:43:56 +0000 UTC" firstStartedPulling="2026-05-26 23:48:07.574104334 +0000 UTC m=+251.750135473" lastFinishedPulling="2026-05-26 23:48:08.640825951 +0000 UTC m=+252.816857106" observedRunningTime="2026-05-26 23:48:09.273162945 +0000 UTC m=+253.449194120" watchObservedRunningTime="2026-05-26 23:48:09.27442334 +0000 UTC m=+253.450454501"
May 26 23:48:12.522753 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.522722    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-86bc846b86-rjbkq"]
May 26 23:48:12.525959 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.525935    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.537315 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.537290    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86bc846b86-rjbkq"]
May 26 23:48:12.626181 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.626149    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-oauth-serving-cert\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.626376 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.626195    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-config\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.626376 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.626254    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g2lc\" (UniqueName: \"kubernetes.io/projected/e29c2f95-35c0-4f0f-9c42-b80b96befc38-kube-api-access-6g2lc\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.626376 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.626329    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-trusted-ca-bundle\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.626376 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.626355    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-serving-cert\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.626530 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.626387    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-oauth-config\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.626530 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.626405    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-service-ca\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.727782 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.727742    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6g2lc\" (UniqueName: \"kubernetes.io/projected/e29c2f95-35c0-4f0f-9c42-b80b96befc38-kube-api-access-6g2lc\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.727961 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.727799    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-trusted-ca-bundle\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.727961 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.727830    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-serving-cert\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.727961 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.727867    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-oauth-config\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.727961 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.727889    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-service-ca\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.728187 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.727970    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-oauth-serving-cert\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.728187 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.728011    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-config\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.728747 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.728664    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-service-ca\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.728747 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.728682    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-trusted-ca-bundle\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.728747 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.728692    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-config\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.729018 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.728944    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-oauth-serving-cert\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.730537 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.730508    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-serving-cert\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.730630 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.730553    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-oauth-config\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.735410 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.735392    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g2lc\" (UniqueName: \"kubernetes.io/projected/e29c2f95-35c0-4f0f-9c42-b80b96befc38-kube-api-access-6g2lc\") pod \"console-86bc846b86-rjbkq\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") " pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.836609 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.836564    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:12.992583 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:12.991491    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86bc846b86-rjbkq"]
May 26 23:48:12.993947 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:48:12.993889    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode29c2f95_35c0_4f0f_9c42_b80b96befc38.slice/crio-f79337e6f18a2b781a496186ce23767c192e7cfdaf068a1c3e57b38862d6292e WatchSource:0}: Error finding container f79337e6f18a2b781a496186ce23767c192e7cfdaf068a1c3e57b38862d6292e: Status 404 returned error can't find the container with id f79337e6f18a2b781a496186ce23767c192e7cfdaf068a1c3e57b38862d6292e
May 26 23:48:13.270694 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:13.270602    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bc846b86-rjbkq" event={"ID":"e29c2f95-35c0-4f0f-9c42-b80b96befc38","Type":"ContainerStarted","Data":"75204464e9d50c8dbd5b3b078e2a940dcd5a0f215049b72fa271455cbde71c51"}
May 26 23:48:13.270694 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:13.270640    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bc846b86-rjbkq" event={"ID":"e29c2f95-35c0-4f0f-9c42-b80b96befc38","Type":"ContainerStarted","Data":"f79337e6f18a2b781a496186ce23767c192e7cfdaf068a1c3e57b38862d6292e"}
May 26 23:48:13.286942 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:13.286862    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-86bc846b86-rjbkq" podStartSLOduration=1.28684202 podStartE2EDuration="1.28684202s" podCreationTimestamp="2026-05-26 23:48:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 23:48:13.286101437 +0000 UTC m=+257.462132611" watchObservedRunningTime="2026-05-26 23:48:13.28684202 +0000 UTC m=+257.462873183"
May 26 23:48:22.837637 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:22.837584    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:22.837637 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:22.837640    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:22.842349 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:22.842325    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:23.301915 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:23.301867    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:48:23.344887 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:23.344855    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fd87d9cbb-lhftj"]
May 26 23:48:48.366804 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.366758    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7fd87d9cbb-lhftj" podUID="173609e0-4434-4c50-8331-6a3316ef3092" containerName="console" containerID="cri-o://d9a9d865213076040c49be4d7e5cb22a4591f9dee8d7db2072050a6cf80a87a3" gracePeriod=15
May 26 23:48:48.601619 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.601588    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7fd87d9cbb-lhftj_173609e0-4434-4c50-8331-6a3316ef3092/console/0.log"
May 26 23:48:48.601745 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.601649    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:48:48.627985 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.627874    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/173609e0-4434-4c50-8331-6a3316ef3092-console-serving-cert\") pod \"173609e0-4434-4c50-8331-6a3316ef3092\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") "
May 26 23:48:48.627985 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.627965    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/173609e0-4434-4c50-8331-6a3316ef3092-console-oauth-config\") pod \"173609e0-4434-4c50-8331-6a3316ef3092\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") "
May 26 23:48:48.627985 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.627985    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-trusted-ca-bundle\") pod \"173609e0-4434-4c50-8331-6a3316ef3092\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") "
May 26 23:48:48.628253 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.628004    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-console-config\") pod \"173609e0-4434-4c50-8331-6a3316ef3092\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") "
May 26 23:48:48.628253 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.628027    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cb7lg\" (UniqueName: \"kubernetes.io/projected/173609e0-4434-4c50-8331-6a3316ef3092-kube-api-access-cb7lg\") pod \"173609e0-4434-4c50-8331-6a3316ef3092\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") "
May 26 23:48:48.628253 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.628055    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-service-ca\") pod \"173609e0-4434-4c50-8331-6a3316ef3092\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") "
May 26 23:48:48.628253 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.628123    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-oauth-serving-cert\") pod \"173609e0-4434-4c50-8331-6a3316ef3092\" (UID: \"173609e0-4434-4c50-8331-6a3316ef3092\") "
May 26 23:48:48.628854 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.628593    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "173609e0-4434-4c50-8331-6a3316ef3092" (UID: "173609e0-4434-4c50-8331-6a3316ef3092"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:48:48.628854 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.628569    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-console-config" (OuterVolumeSpecName: "console-config") pod "173609e0-4434-4c50-8331-6a3316ef3092" (UID: "173609e0-4434-4c50-8331-6a3316ef3092"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:48:48.628854 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.628704    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-service-ca" (OuterVolumeSpecName: "service-ca") pod "173609e0-4434-4c50-8331-6a3316ef3092" (UID: "173609e0-4434-4c50-8331-6a3316ef3092"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:48:48.628854 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.628719    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "173609e0-4434-4c50-8331-6a3316ef3092" (UID: "173609e0-4434-4c50-8331-6a3316ef3092"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:48:48.630435 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.630404    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/173609e0-4434-4c50-8331-6a3316ef3092-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "173609e0-4434-4c50-8331-6a3316ef3092" (UID: "173609e0-4434-4c50-8331-6a3316ef3092"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:48:48.630540 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.630472    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/173609e0-4434-4c50-8331-6a3316ef3092-kube-api-access-cb7lg" (OuterVolumeSpecName: "kube-api-access-cb7lg") pod "173609e0-4434-4c50-8331-6a3316ef3092" (UID: "173609e0-4434-4c50-8331-6a3316ef3092"). InnerVolumeSpecName "kube-api-access-cb7lg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 23:48:48.630540 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.630496    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/173609e0-4434-4c50-8331-6a3316ef3092-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "173609e0-4434-4c50-8331-6a3316ef3092" (UID: "173609e0-4434-4c50-8331-6a3316ef3092"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:48:48.729560 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.729510    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-oauth-serving-cert\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:48:48.729560 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.729556    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/173609e0-4434-4c50-8331-6a3316ef3092-console-serving-cert\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:48:48.729560 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.729566    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/173609e0-4434-4c50-8331-6a3316ef3092-console-oauth-config\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:48:48.729560 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.729576    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-trusted-ca-bundle\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:48:48.729828 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.729585    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-console-config\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:48:48.729828 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.729594    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb7lg\" (UniqueName: \"kubernetes.io/projected/173609e0-4434-4c50-8331-6a3316ef3092-kube-api-access-cb7lg\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:48:48.729828 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:48.729603    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/173609e0-4434-4c50-8331-6a3316ef3092-service-ca\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:48:49.375223 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:49.375195    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7fd87d9cbb-lhftj_173609e0-4434-4c50-8331-6a3316ef3092/console/0.log"
May 26 23:48:49.375613 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:49.375236    2566 generic.go:358] "Generic (PLEG): container finished" podID="173609e0-4434-4c50-8331-6a3316ef3092" containerID="d9a9d865213076040c49be4d7e5cb22a4591f9dee8d7db2072050a6cf80a87a3" exitCode=2
May 26 23:48:49.375613 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:49.375269    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fd87d9cbb-lhftj" event={"ID":"173609e0-4434-4c50-8331-6a3316ef3092","Type":"ContainerDied","Data":"d9a9d865213076040c49be4d7e5cb22a4591f9dee8d7db2072050a6cf80a87a3"}
May 26 23:48:49.375613 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:49.375317    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fd87d9cbb-lhftj"
May 26 23:48:49.375613 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:49.375331    2566 scope.go:117] "RemoveContainer" containerID="d9a9d865213076040c49be4d7e5cb22a4591f9dee8d7db2072050a6cf80a87a3"
May 26 23:48:49.375613 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:49.375317    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fd87d9cbb-lhftj" event={"ID":"173609e0-4434-4c50-8331-6a3316ef3092","Type":"ContainerDied","Data":"dfd54d6efa43fbd17b3f1b4eddfc7695f64bb67459bc74b6412a5828dd44db03"}
May 26 23:48:49.384175 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:49.384158    2566 scope.go:117] "RemoveContainer" containerID="d9a9d865213076040c49be4d7e5cb22a4591f9dee8d7db2072050a6cf80a87a3"
May 26 23:48:49.384459 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:48:49.384441    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9a9d865213076040c49be4d7e5cb22a4591f9dee8d7db2072050a6cf80a87a3\": container with ID starting with d9a9d865213076040c49be4d7e5cb22a4591f9dee8d7db2072050a6cf80a87a3 not found: ID does not exist" containerID="d9a9d865213076040c49be4d7e5cb22a4591f9dee8d7db2072050a6cf80a87a3"
May 26 23:48:49.384515 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:49.384467    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9a9d865213076040c49be4d7e5cb22a4591f9dee8d7db2072050a6cf80a87a3"} err="failed to get container status \"d9a9d865213076040c49be4d7e5cb22a4591f9dee8d7db2072050a6cf80a87a3\": rpc error: code = NotFound desc = could not find container \"d9a9d865213076040c49be4d7e5cb22a4591f9dee8d7db2072050a6cf80a87a3\": container with ID starting with d9a9d865213076040c49be4d7e5cb22a4591f9dee8d7db2072050a6cf80a87a3 not found: ID does not exist"
May 26 23:48:49.396671 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:49.396638    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fd87d9cbb-lhftj"]
May 26 23:48:49.399636 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:49.399612    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7fd87d9cbb-lhftj"]
May 26 23:48:50.431573 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:50.431540    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="173609e0-4434-4c50-8331-6a3316ef3092" path="/var/lib/kubelet/pods/173609e0-4434-4c50-8331-6a3316ef3092/volumes"
May 26 23:48:56.311452 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:56.311417    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 26 23:48:56.312520 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:56.312497    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 26 23:48:56.325332 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:48:56.325309    2566 kubelet.go:1628] "Image garbage collection succeeded"
May 26 23:49:23.042981 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.037250    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6887db57c9-8xscn"]
May 26 23:49:23.042981 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.037927    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="173609e0-4434-4c50-8331-6a3316ef3092" containerName="console"
May 26 23:49:23.042981 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.037947    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="173609e0-4434-4c50-8331-6a3316ef3092" containerName="console"
May 26 23:49:23.042981 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.038080    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="173609e0-4434-4c50-8331-6a3316ef3092" containerName="console"
May 26 23:49:23.042981 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.041775    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.052508 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.052453    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6887db57c9-8xscn"]
May 26 23:49:23.111150 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.111108    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-config\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.111150 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.111149    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lhrb\" (UniqueName: \"kubernetes.io/projected/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-kube-api-access-7lhrb\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.111368 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.111183    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-oauth-serving-cert\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.111368 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.111278    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-serving-cert\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.111368 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.111319    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-oauth-config\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.111368 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.111341    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-service-ca\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.111519 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.111394    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-trusted-ca-bundle\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.211994 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.211962    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-service-ca\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.212218 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.212009    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-trusted-ca-bundle\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.212218 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.212077    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-config\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.212218 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.212101    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7lhrb\" (UniqueName: \"kubernetes.io/projected/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-kube-api-access-7lhrb\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.212218 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.212134    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-oauth-serving-cert\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.212218 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.212188    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-serving-cert\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.212466 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.212221    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-oauth-config\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.212755 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.212727    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-service-ca\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.212856 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.212835    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-config\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.212939 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.212878    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-oauth-serving-cert\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.213169 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.212952    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-trusted-ca-bundle\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.214836 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.214809    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-oauth-config\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.214986 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.214933    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-serving-cert\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.220102 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.220082    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lhrb\" (UniqueName: \"kubernetes.io/projected/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-kube-api-access-7lhrb\") pod \"console-6887db57c9-8xscn\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") " pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.353210 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.353169    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:23.479304 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.479151    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6887db57c9-8xscn"]
May 26 23:49:23.482072 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:49:23.482047    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf6846d_7c01_4b3f_bbca_08cae7d9f07a.slice/crio-fc7ec36ffa16afc2a7d73df654b439e13b855d1b45e3fde64cf474e86b1f40bf WatchSource:0}: Error finding container fc7ec36ffa16afc2a7d73df654b439e13b855d1b45e3fde64cf474e86b1f40bf: Status 404 returned error can't find the container with id fc7ec36ffa16afc2a7d73df654b439e13b855d1b45e3fde64cf474e86b1f40bf
May 26 23:49:23.484168 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:23.484147    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 26 23:49:24.471551 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:24.471515    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6887db57c9-8xscn" event={"ID":"acf6846d-7c01-4b3f-bbca-08cae7d9f07a","Type":"ContainerStarted","Data":"136e67c114e1dcff788334b77e85beb93bd43d7796a0457bd4916cfe50cc5ffd"}
May 26 23:49:24.471551 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:24.471552    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6887db57c9-8xscn" event={"ID":"acf6846d-7c01-4b3f-bbca-08cae7d9f07a","Type":"ContainerStarted","Data":"fc7ec36ffa16afc2a7d73df654b439e13b855d1b45e3fde64cf474e86b1f40bf"}
May 26 23:49:24.489232 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:24.489184    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6887db57c9-8xscn" podStartSLOduration=1.489166445 podStartE2EDuration="1.489166445s" podCreationTimestamp="2026-05-26 23:49:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 23:49:24.487581184 +0000 UTC m=+328.663612346" watchObservedRunningTime="2026-05-26 23:49:24.489166445 +0000 UTC m=+328.665197608"
May 26 23:49:33.353840 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:33.353781    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:33.353840 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:33.353854    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:33.358645 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:33.358623    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:33.500392 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:33.500356    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:49:33.540751 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:33.540713    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86bc846b86-rjbkq"]
May 26 23:49:58.561309 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.561243    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-86bc846b86-rjbkq" podUID="e29c2f95-35c0-4f0f-9c42-b80b96befc38" containerName="console" containerID="cri-o://75204464e9d50c8dbd5b3b078e2a940dcd5a0f215049b72fa271455cbde71c51" gracePeriod=15
May 26 23:49:58.799181 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.799161    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-86bc846b86-rjbkq_e29c2f95-35c0-4f0f-9c42-b80b96befc38/console/0.log"
May 26 23:49:58.799281 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.799223    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:49:58.908687 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.908608    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-service-ca\") pod \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") "
May 26 23:49:58.908687 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.908642    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-oauth-serving-cert\") pod \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") "
May 26 23:49:58.908687 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.908668    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6g2lc\" (UniqueName: \"kubernetes.io/projected/e29c2f95-35c0-4f0f-9c42-b80b96befc38-kube-api-access-6g2lc\") pod \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") "
May 26 23:49:58.908980 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.908692    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-trusted-ca-bundle\") pod \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") "
May 26 23:49:58.908980 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.908869    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-serving-cert\") pod \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") "
May 26 23:49:58.908980 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.908930    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-oauth-config\") pod \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") "
May 26 23:49:58.909136 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.909013    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-config\") pod \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\" (UID: \"e29c2f95-35c0-4f0f-9c42-b80b96befc38\") "
May 26 23:49:58.909189 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.909127    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-service-ca" (OuterVolumeSpecName: "service-ca") pod "e29c2f95-35c0-4f0f-9c42-b80b96befc38" (UID: "e29c2f95-35c0-4f0f-9c42-b80b96befc38"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:49:58.909189 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.909133    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e29c2f95-35c0-4f0f-9c42-b80b96befc38" (UID: "e29c2f95-35c0-4f0f-9c42-b80b96befc38"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:49:58.909294 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.909276    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-service-ca\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:49:58.909347 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.909329    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-oauth-serving-cert\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:49:58.909389 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.909350    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e29c2f95-35c0-4f0f-9c42-b80b96befc38" (UID: "e29c2f95-35c0-4f0f-9c42-b80b96befc38"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:49:58.909425 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.909416    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-config" (OuterVolumeSpecName: "console-config") pod "e29c2f95-35c0-4f0f-9c42-b80b96befc38" (UID: "e29c2f95-35c0-4f0f-9c42-b80b96befc38"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:49:58.911176 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.911135    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e29c2f95-35c0-4f0f-9c42-b80b96befc38" (UID: "e29c2f95-35c0-4f0f-9c42-b80b96befc38"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:49:58.911296 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.911277    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e29c2f95-35c0-4f0f-9c42-b80b96befc38" (UID: "e29c2f95-35c0-4f0f-9c42-b80b96befc38"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:49:58.911341 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:58.911315    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e29c2f95-35c0-4f0f-9c42-b80b96befc38-kube-api-access-6g2lc" (OuterVolumeSpecName: "kube-api-access-6g2lc") pod "e29c2f95-35c0-4f0f-9c42-b80b96befc38" (UID: "e29c2f95-35c0-4f0f-9c42-b80b96befc38"). InnerVolumeSpecName "kube-api-access-6g2lc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 23:49:59.009997 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.009967    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-config\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:49:59.009997 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.009989    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g2lc\" (UniqueName: \"kubernetes.io/projected/e29c2f95-35c0-4f0f-9c42-b80b96befc38-kube-api-access-6g2lc\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:49:59.009997 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.010000    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c2f95-35c0-4f0f-9c42-b80b96befc38-trusted-ca-bundle\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:49:59.010198 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.010009    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-serving-cert\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:49:59.010198 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.010017    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e29c2f95-35c0-4f0f-9c42-b80b96befc38-console-oauth-config\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:49:59.568072 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.568043    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-86bc846b86-rjbkq_e29c2f95-35c0-4f0f-9c42-b80b96befc38/console/0.log"
May 26 23:49:59.568491 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.568087    2566 generic.go:358] "Generic (PLEG): container finished" podID="e29c2f95-35c0-4f0f-9c42-b80b96befc38" containerID="75204464e9d50c8dbd5b3b078e2a940dcd5a0f215049b72fa271455cbde71c51" exitCode=2
May 26 23:49:59.568491 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.568149    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bc846b86-rjbkq"
May 26 23:49:59.568491 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.568181    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bc846b86-rjbkq" event={"ID":"e29c2f95-35c0-4f0f-9c42-b80b96befc38","Type":"ContainerDied","Data":"75204464e9d50c8dbd5b3b078e2a940dcd5a0f215049b72fa271455cbde71c51"}
May 26 23:49:59.568491 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.568226    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bc846b86-rjbkq" event={"ID":"e29c2f95-35c0-4f0f-9c42-b80b96befc38","Type":"ContainerDied","Data":"f79337e6f18a2b781a496186ce23767c192e7cfdaf068a1c3e57b38862d6292e"}
May 26 23:49:59.568491 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.568242    2566 scope.go:117] "RemoveContainer" containerID="75204464e9d50c8dbd5b3b078e2a940dcd5a0f215049b72fa271455cbde71c51"
May 26 23:49:59.576616 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.576600    2566 scope.go:117] "RemoveContainer" containerID="75204464e9d50c8dbd5b3b078e2a940dcd5a0f215049b72fa271455cbde71c51"
May 26 23:49:59.576854 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:49:59.576838    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75204464e9d50c8dbd5b3b078e2a940dcd5a0f215049b72fa271455cbde71c51\": container with ID starting with 75204464e9d50c8dbd5b3b078e2a940dcd5a0f215049b72fa271455cbde71c51 not found: ID does not exist" containerID="75204464e9d50c8dbd5b3b078e2a940dcd5a0f215049b72fa271455cbde71c51"
May 26 23:49:59.576916 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.576862    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75204464e9d50c8dbd5b3b078e2a940dcd5a0f215049b72fa271455cbde71c51"} err="failed to get container status \"75204464e9d50c8dbd5b3b078e2a940dcd5a0f215049b72fa271455cbde71c51\": rpc error: code = NotFound desc = could not find container \"75204464e9d50c8dbd5b3b078e2a940dcd5a0f215049b72fa271455cbde71c51\": container with ID starting with 75204464e9d50c8dbd5b3b078e2a940dcd5a0f215049b72fa271455cbde71c51 not found: ID does not exist"
May 26 23:49:59.588528 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.588508    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86bc846b86-rjbkq"]
May 26 23:49:59.592880 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:49:59.592861    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86bc846b86-rjbkq"]
May 26 23:50:00.432005 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:50:00.431971    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e29c2f95-35c0-4f0f-9c42-b80b96befc38" path="/var/lib/kubelet/pods/e29c2f95-35c0-4f0f-9c42-b80b96befc38/volumes"
May 26 23:51:26.512798 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.512722    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"]
May 26 23:51:26.513262 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.513061    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e29c2f95-35c0-4f0f-9c42-b80b96befc38" containerName="console"
May 26 23:51:26.513262 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.513072    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e29c2f95-35c0-4f0f-9c42-b80b96befc38" containerName="console"
May 26 23:51:26.513262 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.513139    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="e29c2f95-35c0-4f0f-9c42-b80b96befc38" containerName="console"
May 26 23:51:26.515768 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.515752    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"
May 26 23:51:26.518306 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.518286    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 26 23:51:26.518306 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.518303    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 26 23:51:26.519150 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.519137    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-tdlpw\""
May 26 23:51:26.523102 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.523083    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"]
May 26 23:51:26.545967 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.545945    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czsbs\" (UniqueName: \"kubernetes.io/projected/2071980f-5f2e-4da2-8b5e-c63d65d7ae7c-kube-api-access-czsbs\") pod \"feast-operator-controller-manager-5f95b9f996-nf85z\" (UID: \"2071980f-5f2e-4da2-8b5e-c63d65d7ae7c\") " pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"
May 26 23:51:26.646547 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.646506    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-czsbs\" (UniqueName: \"kubernetes.io/projected/2071980f-5f2e-4da2-8b5e-c63d65d7ae7c-kube-api-access-czsbs\") pod \"feast-operator-controller-manager-5f95b9f996-nf85z\" (UID: \"2071980f-5f2e-4da2-8b5e-c63d65d7ae7c\") " pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"
May 26 23:51:26.654539 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.654520    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czsbs\" (UniqueName: \"kubernetes.io/projected/2071980f-5f2e-4da2-8b5e-c63d65d7ae7c-kube-api-access-czsbs\") pod \"feast-operator-controller-manager-5f95b9f996-nf85z\" (UID: \"2071980f-5f2e-4da2-8b5e-c63d65d7ae7c\") " pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"
May 26 23:51:26.826563 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.826537    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"
May 26 23:51:26.946613 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:26.946473    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"]
May 26 23:51:26.949346 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:51:26.949320    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2071980f_5f2e_4da2_8b5e_c63d65d7ae7c.slice/crio-6655efe2b74fe3fffdd476567bbe63d87e519e85533c5b1732b15c986ae78191 WatchSource:0}: Error finding container 6655efe2b74fe3fffdd476567bbe63d87e519e85533c5b1732b15c986ae78191: Status 404 returned error can't find the container with id 6655efe2b74fe3fffdd476567bbe63d87e519e85533c5b1732b15c986ae78191
May 26 23:51:27.813196 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:27.813149    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z" event={"ID":"2071980f-5f2e-4da2-8b5e-c63d65d7ae7c","Type":"ContainerStarted","Data":"6655efe2b74fe3fffdd476567bbe63d87e519e85533c5b1732b15c986ae78191"}
May 26 23:51:29.821310 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:29.821265    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z" event={"ID":"2071980f-5f2e-4da2-8b5e-c63d65d7ae7c","Type":"ContainerStarted","Data":"2b4322aa713f18f157bc1c50564fc026191bc4c21c87c0688a3679308bef9220"}
May 26 23:51:29.821716 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:29.821409    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"
May 26 23:51:29.836245 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:29.836200    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z" podStartSLOduration=1.6118755359999999 podStartE2EDuration="3.836186772s" podCreationTimestamp="2026-05-26 23:51:26 +0000 UTC" firstStartedPulling="2026-05-26 23:51:26.951133057 +0000 UTC m=+451.127164211" lastFinishedPulling="2026-05-26 23:51:29.175444308 +0000 UTC m=+453.351475447" observedRunningTime="2026-05-26 23:51:29.834800107 +0000 UTC m=+454.010831269" watchObservedRunningTime="2026-05-26 23:51:29.836186772 +0000 UTC m=+454.012217934"
May 26 23:51:40.827153 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:51:40.827123    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"
May 26 23:52:09.282614 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.282576    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"]
May 26 23:52:09.287968 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.287949    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.290356 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.290331    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 26 23:52:09.290454 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.290352    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 26 23:52:09.290454 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.290397    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 26 23:52:09.290651 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.290625    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 26 23:52:09.290767 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.290634    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 26 23:52:09.290767 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.290688    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 26 23:52:09.291146 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.291126    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-r6nqr\""
May 26 23:52:09.291231 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.291135    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 26 23:52:09.296187 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.296164    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"]
May 26 23:52:09.402949 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.402891    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e465be87-74ef-4ddf-b5d1-e6b382877c6e-feast-data\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.403132 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.402972    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg29q\" (UniqueName: \"kubernetes.io/projected/e465be87-74ef-4ddf-b5d1-e6b382877c6e-kube-api-access-wg29q\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.403132 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.403011    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-registry-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.403132 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.403031    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-offline-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.403288 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.403137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-ui-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.403288 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.403187    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-online-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.504660 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.504624    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-registry-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.504660 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.504660    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-offline-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.504930 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.504686    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-ui-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.504930 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.504710    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-online-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.504930 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.504751    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e465be87-74ef-4ddf-b5d1-e6b382877c6e-feast-data\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.504930 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:52:09.504793    2566 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 26 23:52:09.504930 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:52:09.504866    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-offline-tls podName:e465be87-74ef-4ddf-b5d1-e6b382877c6e nodeName:}" failed. No retries permitted until 2026-05-26 23:52:10.004844659 +0000 UTC m=+494.180875815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-offline-tls") pod "feast-simple-feast-setup-8b4b44f9d-f9854" (UID: "e465be87-74ef-4ddf-b5d1-e6b382877c6e") : secret "feast-simple-feast-setup-offline-tls" not found
May 26 23:52:09.505227 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.504798    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wg29q\" (UniqueName: \"kubernetes.io/projected/e465be87-74ef-4ddf-b5d1-e6b382877c6e-kube-api-access-wg29q\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.505270 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.505234    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e465be87-74ef-4ddf-b5d1-e6b382877c6e-feast-data\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.507412 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.507386    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-registry-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.507567 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.507524    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-ui-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.507637 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.507544    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-online-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:09.513785 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:09.513765    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg29q\" (UniqueName: \"kubernetes.io/projected/e465be87-74ef-4ddf-b5d1-e6b382877c6e-kube-api-access-wg29q\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:10.009073 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:10.009035    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-offline-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:10.011718 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:10.011687    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-offline-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-f9854\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:10.209579 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:10.209544    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:10.339474 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:10.339449    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"]
May 26 23:52:10.341667 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:52:10.341635    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode465be87_74ef_4ddf_b5d1_e6b382877c6e.slice/crio-bd8c5666cbe2bc0140f25f08f8f15ad979a0c8463324737e0d18222bb7dd00e3 WatchSource:0}: Error finding container bd8c5666cbe2bc0140f25f08f8f15ad979a0c8463324737e0d18222bb7dd00e3: Status 404 returned error can't find the container with id bd8c5666cbe2bc0140f25f08f8f15ad979a0c8463324737e0d18222bb7dd00e3
May 26 23:52:10.939104 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:10.939063    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerStarted","Data":"bd8c5666cbe2bc0140f25f08f8f15ad979a0c8463324737e0d18222bb7dd00e3"}
May 26 23:52:41.037959 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:41.037919    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerStarted","Data":"ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b"}
May 26 23:52:45.050876 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:45.050841    2566 generic.go:358] "Generic (PLEG): container finished" podID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerID="ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b" exitCode=0
May 26 23:52:45.051271 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:45.050920    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerDied","Data":"ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b"}
May 26 23:52:46.056265 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:46.056230    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerStarted","Data":"aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e"}
May 26 23:52:51.072978 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:51.072943    2566 generic.go:358] "Generic (PLEG): container finished" podID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerID="aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e" exitCode=0
May 26 23:52:51.072978 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:51.072980    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerDied","Data":"aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e"}
May 26 23:52:52.080989 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:52.080946    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerStarted","Data":"c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8"}
May 26 23:52:52.081551 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:52.081002    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerStarted","Data":"dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c"}
May 26 23:52:52.081551 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:52.081018    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerStarted","Data":"97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938"}
May 26 23:52:52.081551 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:52.081031    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerStarted","Data":"9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4"}
May 26 23:52:52.104613 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:52.104534    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podStartSLOduration=13.396049854 podStartE2EDuration="43.104513484s" podCreationTimestamp="2026-05-26 23:52:09 +0000 UTC" firstStartedPulling="2026-05-26 23:52:10.343333141 +0000 UTC m=+494.519364281" lastFinishedPulling="2026-05-26 23:52:40.051796768 +0000 UTC m=+524.227827911" observedRunningTime="2026-05-26 23:52:52.103561011 +0000 UTC m=+536.279592196" watchObservedRunningTime="2026-05-26 23:52:52.104513484 +0000 UTC m=+536.280544646"
May 26 23:52:52.210725 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:52.210525    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:52.210725 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:52.210660    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:52.210725 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:52.210688    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:52.210725 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:52.210701    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:52.213062 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:52.212923    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.25:6571: connect: connection refused"
May 26 23:52:52.213062 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:52.212969    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.25:8816: connect: connection refused"
May 26 23:52:52.213062 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:52.212987    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused"
May 26 23:52:52.213062 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:52.212999    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused"
May 26 23:52:55.210959 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:55.210916    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused"
May 26 23:52:55.210959 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:55.210937    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused"
May 26 23:52:55.211512 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:55.210882    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.25:8816: connect: connection refused"
May 26 23:52:55.211512 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:55.211066    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.25:6571: connect: connection refused"
May 26 23:52:58.211070 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:58.211041    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:58.211585 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:58.211277    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:58.211585 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:58.211409    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:58.211780 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:58.211761    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:58.211852 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:58.211791    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:58.211852 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:58.211805    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:58.211996 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:58.211940    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:58.212032 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:58.212000    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:58.212246 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:58.212229    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:58.215625 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:58.215608    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:59.106213 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:59.106180    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:52:59.109249 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:52:59.109226    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:53:34.846966 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:53:34.846927    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6887db57c9-8xscn"]
May 26 23:53:56.336073 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:53:56.336047    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 26 23:53:56.339363 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:53:56.339334    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 26 23:53:59.868207 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:53:59.868137    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-6887db57c9-8xscn" podUID="acf6846d-7c01-4b3f-bbca-08cae7d9f07a" containerName="console" containerID="cri-o://136e67c114e1dcff788334b77e85beb93bd43d7796a0457bd4916cfe50cc5ffd" gracePeriod=15
May 26 23:54:00.110256 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.110229    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6887db57c9-8xscn_acf6846d-7c01-4b3f-bbca-08cae7d9f07a/console/0.log"
May 26 23:54:00.110389 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.110293    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:54:00.167298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.167203    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-oauth-config\") pod \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") "
May 26 23:54:00.167298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.167244    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-trusted-ca-bundle\") pod \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") "
May 26 23:54:00.167298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.167296    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lhrb\" (UniqueName: \"kubernetes.io/projected/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-kube-api-access-7lhrb\") pod \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") "
May 26 23:54:00.167573 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.167378    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-config\") pod \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") "
May 26 23:54:00.167573 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.167415    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-oauth-serving-cert\") pod \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") "
May 26 23:54:00.167573 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.167441    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-serving-cert\") pod \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") "
May 26 23:54:00.167573 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.167465    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-service-ca\") pod \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\" (UID: \"acf6846d-7c01-4b3f-bbca-08cae7d9f07a\") "
May 26 23:54:00.167757 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.167729    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "acf6846d-7c01-4b3f-bbca-08cae7d9f07a" (UID: "acf6846d-7c01-4b3f-bbca-08cae7d9f07a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:54:00.167925 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.167801    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "acf6846d-7c01-4b3f-bbca-08cae7d9f07a" (UID: "acf6846d-7c01-4b3f-bbca-08cae7d9f07a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:54:00.168026 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.167816    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-config" (OuterVolumeSpecName: "console-config") pod "acf6846d-7c01-4b3f-bbca-08cae7d9f07a" (UID: "acf6846d-7c01-4b3f-bbca-08cae7d9f07a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:54:00.168026 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.167989    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-service-ca" (OuterVolumeSpecName: "service-ca") pod "acf6846d-7c01-4b3f-bbca-08cae7d9f07a" (UID: "acf6846d-7c01-4b3f-bbca-08cae7d9f07a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:54:00.169885 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.169866    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "acf6846d-7c01-4b3f-bbca-08cae7d9f07a" (UID: "acf6846d-7c01-4b3f-bbca-08cae7d9f07a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:54:00.170136 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.170119    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-kube-api-access-7lhrb" (OuterVolumeSpecName: "kube-api-access-7lhrb") pod "acf6846d-7c01-4b3f-bbca-08cae7d9f07a" (UID: "acf6846d-7c01-4b3f-bbca-08cae7d9f07a"). InnerVolumeSpecName "kube-api-access-7lhrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 23:54:00.170186 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.170134    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "acf6846d-7c01-4b3f-bbca-08cae7d9f07a" (UID: "acf6846d-7c01-4b3f-bbca-08cae7d9f07a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:54:00.269020 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.268978    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-oauth-serving-cert\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:54:00.269020 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.269010    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-serving-cert\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:54:00.269020 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.269020    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-service-ca\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:54:00.269020 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.269030    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-oauth-config\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:54:00.269281 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.269038    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-trusted-ca-bundle\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:54:00.269281 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.269047    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lhrb\" (UniqueName: \"kubernetes.io/projected/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-kube-api-access-7lhrb\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:54:00.269281 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.269057    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acf6846d-7c01-4b3f-bbca-08cae7d9f07a-console-config\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:54:00.288643 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.288615    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6887db57c9-8xscn_acf6846d-7c01-4b3f-bbca-08cae7d9f07a/console/0.log"
May 26 23:54:00.288814 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.288656    2566 generic.go:358] "Generic (PLEG): container finished" podID="acf6846d-7c01-4b3f-bbca-08cae7d9f07a" containerID="136e67c114e1dcff788334b77e85beb93bd43d7796a0457bd4916cfe50cc5ffd" exitCode=2
May 26 23:54:00.288814 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.288722    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6887db57c9-8xscn" event={"ID":"acf6846d-7c01-4b3f-bbca-08cae7d9f07a","Type":"ContainerDied","Data":"136e67c114e1dcff788334b77e85beb93bd43d7796a0457bd4916cfe50cc5ffd"}
May 26 23:54:00.288814 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.288736    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6887db57c9-8xscn"
May 26 23:54:00.288814 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.288750    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6887db57c9-8xscn" event={"ID":"acf6846d-7c01-4b3f-bbca-08cae7d9f07a","Type":"ContainerDied","Data":"fc7ec36ffa16afc2a7d73df654b439e13b855d1b45e3fde64cf474e86b1f40bf"}
May 26 23:54:00.288814 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.288764    2566 scope.go:117] "RemoveContainer" containerID="136e67c114e1dcff788334b77e85beb93bd43d7796a0457bd4916cfe50cc5ffd"
May 26 23:54:00.297424 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.297398    2566 scope.go:117] "RemoveContainer" containerID="136e67c114e1dcff788334b77e85beb93bd43d7796a0457bd4916cfe50cc5ffd"
May 26 23:54:00.297687 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:54:00.297669    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"136e67c114e1dcff788334b77e85beb93bd43d7796a0457bd4916cfe50cc5ffd\": container with ID starting with 136e67c114e1dcff788334b77e85beb93bd43d7796a0457bd4916cfe50cc5ffd not found: ID does not exist" containerID="136e67c114e1dcff788334b77e85beb93bd43d7796a0457bd4916cfe50cc5ffd"
May 26 23:54:00.297739 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.297697    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"136e67c114e1dcff788334b77e85beb93bd43d7796a0457bd4916cfe50cc5ffd"} err="failed to get container status \"136e67c114e1dcff788334b77e85beb93bd43d7796a0457bd4916cfe50cc5ffd\": rpc error: code = NotFound desc = could not find container \"136e67c114e1dcff788334b77e85beb93bd43d7796a0457bd4916cfe50cc5ffd\": container with ID starting with 136e67c114e1dcff788334b77e85beb93bd43d7796a0457bd4916cfe50cc5ffd not found: ID does not exist"
May 26 23:54:00.309085 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.309055    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6887db57c9-8xscn"]
May 26 23:54:00.311733 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.311708    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6887db57c9-8xscn"]
May 26 23:54:00.432472 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.432385    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acf6846d-7c01-4b3f-bbca-08cae7d9f07a" path="/var/lib/kubelet/pods/acf6846d-7c01-4b3f-bbca-08cae7d9f07a/volumes"
May 26 23:54:00.445684 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.445642    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"]
May 26 23:54:00.446020 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.445991    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="registry" containerID="cri-o://9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4" gracePeriod=30
May 26 23:54:00.446219 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.446062    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="online" containerID="cri-o://97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938" gracePeriod=30
May 26 23:54:00.446336 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.446063    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="offline" containerID="cri-o://dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c" gracePeriod=30
May 26 23:54:00.446336 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.446063    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="ui" containerID="cri-o://c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8" gracePeriod=30
May 26 23:54:00.818130 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.818089    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"]
May 26 23:54:00.818620 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.818596    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf6846d-7c01-4b3f-bbca-08cae7d9f07a" containerName="console"
May 26 23:54:00.818713 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.818623    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf6846d-7c01-4b3f-bbca-08cae7d9f07a" containerName="console"
May 26 23:54:00.818769 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.818725    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="acf6846d-7c01-4b3f-bbca-08cae7d9f07a" containerName="console"
May 26 23:54:00.824118 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.824095    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.826643 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.826623    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-dmrlj\""
May 26 23:54:00.833279 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.833252    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"]
May 26 23:54:00.873626 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.873583    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-online-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.873626 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.873639    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-offline-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.874176 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.873671    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-registry-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.874176 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.873751    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fee6e4ad-903d-414e-9835-2ae9a8b5997d-feast-data\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.874176 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.873820    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-ui-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.874176 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.873944    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qljt\" (UniqueName: \"kubernetes.io/projected/fee6e4ad-903d-414e-9835-2ae9a8b5997d-kube-api-access-8qljt\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.975082 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.975039    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-registry-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.975210 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.975091    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fee6e4ad-903d-414e-9835-2ae9a8b5997d-feast-data\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.975210 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.975124    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-ui-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.975210 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.975166    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8qljt\" (UniqueName: \"kubernetes.io/projected/fee6e4ad-903d-414e-9835-2ae9a8b5997d-kube-api-access-8qljt\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.975210 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.975204    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-online-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.975433 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.975229    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-offline-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.975433 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:54:00.975273    2566 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 26 23:54:00.975433 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:54:00.975361    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-ui-tls podName:fee6e4ad-903d-414e-9835-2ae9a8b5997d nodeName:}" failed. No retries permitted until 2026-05-26 23:54:01.475340784 +0000 UTC m=+605.651371943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-ui-tls") pod "feast-simple-feast-setup-8b4b44f9d-mtrlx" (UID: "fee6e4ad-903d-414e-9835-2ae9a8b5997d") : secret "feast-simple-feast-setup-ui-tls" not found
May 26 23:54:00.975602 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.975566    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fee6e4ad-903d-414e-9835-2ae9a8b5997d-feast-data\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.977946 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.977886    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-offline-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.978283 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.978260    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-online-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.978358 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.978316    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-registry-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:00.987667 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:00.987643    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qljt\" (UniqueName: \"kubernetes.io/projected/fee6e4ad-903d-414e-9835-2ae9a8b5997d-kube-api-access-8qljt\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:01.294837 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:01.294805    2566 generic.go:358] "Generic (PLEG): container finished" podID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerID="c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8" exitCode=0
May 26 23:54:01.294837 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:01.294830    2566 generic.go:358] "Generic (PLEG): container finished" podID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerID="97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938" exitCode=0
May 26 23:54:01.295035 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:01.294875    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerDied","Data":"c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8"}
May 26 23:54:01.295035 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:01.294921    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerDied","Data":"97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938"}
May 26 23:54:01.480553 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:01.480470    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-ui-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:01.483025 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:01.482997    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-ui-tls\") pod \"feast-simple-feast-setup-8b4b44f9d-mtrlx\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") " pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:01.736159 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:01.736068    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:01.865922 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:01.865862    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"]
May 26 23:54:02.299522 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:02.299469    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerStarted","Data":"81efbc2f3ad82296999d81ab4200c1f38b0957aa213bf80ff51dbf70d932674a"}
May 26 23:54:02.299522 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:02.299526    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerStarted","Data":"a8a3df4e4eab1e693caac5156472f5ccc45103cab04ee36a00d8a4a2c9dec6c7"}
May 26 23:54:07.314842 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:07.314810    2566 generic.go:358] "Generic (PLEG): container finished" podID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerID="81efbc2f3ad82296999d81ab4200c1f38b0957aa213bf80ff51dbf70d932674a" exitCode=0
May 26 23:54:07.315274 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:07.314880    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerDied","Data":"81efbc2f3ad82296999d81ab4200c1f38b0957aa213bf80ff51dbf70d932674a"}
May 26 23:54:08.211985 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:08.211938    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused"
May 26 23:54:08.320189 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:08.320151    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerStarted","Data":"950b02316a54b76ebd96fec75aa432c41155b34d4fec18f86ba78db954f849e7"}
May 26 23:54:09.107108 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:09.107001    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused"
May 26 23:54:13.335940 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:13.335886    2566 generic.go:358] "Generic (PLEG): container finished" podID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerID="950b02316a54b76ebd96fec75aa432c41155b34d4fec18f86ba78db954f849e7" exitCode=0
May 26 23:54:13.336328 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:13.335964    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerDied","Data":"950b02316a54b76ebd96fec75aa432c41155b34d4fec18f86ba78db954f849e7"}
May 26 23:54:14.344762 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:14.344719    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerStarted","Data":"8e3f8093eaac7ef6b3885029360016654146f254711284e58acf07073e6fdce0"}
May 26 23:54:14.345272 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:14.344779    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerStarted","Data":"78e71c08a856e2ff5c6fc38f435b1627213d81684491a6b1c73fbe7ff60dc9fe"}
May 26 23:54:14.345272 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:14.344796    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerStarted","Data":"f319a23da81199fe474c0d1dc4e7c76e8b2cc9e91bad53dae880eafe2d42d8e5"}
May 26 23:54:14.345272 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:14.344809    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerStarted","Data":"b40b880ec209da8b139217fd88b365da658b4a75e5598798c936f171f150ca6c"}
May 26 23:54:14.368679 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:14.368592    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podStartSLOduration=14.368570584 podStartE2EDuration="14.368570584s" podCreationTimestamp="2026-05-26 23:54:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 23:54:14.366264731 +0000 UTC m=+618.542295906" watchObservedRunningTime="2026-05-26 23:54:14.368570584 +0000 UTC m=+618.544601747"
May 26 23:54:16.736500 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:16.736450    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:16.736500 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:16.736498    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:16.737070 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:16.736513    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:16.737070 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:16.736526    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:16.738751 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:16.738717    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused"
May 26 23:54:16.739016 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:16.738734    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused"
May 26 23:54:16.739195 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:16.738753    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused"
May 26 23:54:16.739349 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:16.738784    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused"
May 26 23:54:18.212202 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:18.212131    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused"
May 26 23:54:19.106500 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:19.106450    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused"
May 26 23:54:19.737493 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:19.737463    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:19.737853 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:19.737560    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:19.737853 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:19.737780    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:19.738201 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:19.738164    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:19.738385 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:19.738362    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:19.738471 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:19.738389    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:19.738471 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:19.738449    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:19.738539 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:19.738497    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:19.738599 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:19.738581    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:19.741971 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:19.741951    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:20.367758 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:20.367728    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:20.370756 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:20.370734    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:54:28.212239 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:28.212190    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused"
May 26 23:54:28.214714 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:28.212331    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:54:29.107042 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:29.106995    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused"
May 26 23:54:29.107244 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:29.107129    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:54:31.103276 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.103252    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:54:31.251961 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.251865    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-offline-tls\") pod \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") "
May 26 23:54:31.251961 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.251922    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-ui-tls\") pod \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") "
May 26 23:54:31.251961 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.251961    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wg29q\" (UniqueName: \"kubernetes.io/projected/e465be87-74ef-4ddf-b5d1-e6b382877c6e-kube-api-access-wg29q\") pod \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") "
May 26 23:54:31.252226 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.251979    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-online-tls\") pod \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") "
May 26 23:54:31.252226 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.252014    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e465be87-74ef-4ddf-b5d1-e6b382877c6e-feast-data\") pod \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") "
May 26 23:54:31.252226 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.252036    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-registry-tls\") pod \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\" (UID: \"e465be87-74ef-4ddf-b5d1-e6b382877c6e\") "
May 26 23:54:31.252664 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.252561    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e465be87-74ef-4ddf-b5d1-e6b382877c6e-feast-data" (OuterVolumeSpecName: "feast-data") pod "e465be87-74ef-4ddf-b5d1-e6b382877c6e" (UID: "e465be87-74ef-4ddf-b5d1-e6b382877c6e"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 23:54:31.254412 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.254379    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e465be87-74ef-4ddf-b5d1-e6b382877c6e" (UID: "e465be87-74ef-4ddf-b5d1-e6b382877c6e"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:54:31.254412 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.254395    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e465be87-74ef-4ddf-b5d1-e6b382877c6e" (UID: "e465be87-74ef-4ddf-b5d1-e6b382877c6e"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:54:31.254557 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.254471    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-online-tls" (OuterVolumeSpecName: "online-tls") pod "e465be87-74ef-4ddf-b5d1-e6b382877c6e" (UID: "e465be87-74ef-4ddf-b5d1-e6b382877c6e"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:54:31.254701 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.254672    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e465be87-74ef-4ddf-b5d1-e6b382877c6e" (UID: "e465be87-74ef-4ddf-b5d1-e6b382877c6e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:54:31.254789 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.254704    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e465be87-74ef-4ddf-b5d1-e6b382877c6e-kube-api-access-wg29q" (OuterVolumeSpecName: "kube-api-access-wg29q") pod "e465be87-74ef-4ddf-b5d1-e6b382877c6e" (UID: "e465be87-74ef-4ddf-b5d1-e6b382877c6e"). InnerVolumeSpecName "kube-api-access-wg29q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 23:54:31.353364 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.353332    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wg29q\" (UniqueName: \"kubernetes.io/projected/e465be87-74ef-4ddf-b5d1-e6b382877c6e-kube-api-access-wg29q\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:54:31.353364 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.353362    2566 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-online-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:54:31.353364 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.353372    2566 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e465be87-74ef-4ddf-b5d1-e6b382877c6e-feast-data\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:54:31.353568 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.353380    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-registry-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:54:31.353568 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.353388    2566 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-offline-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:54:31.353568 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.353398    2566 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e465be87-74ef-4ddf-b5d1-e6b382877c6e-ui-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:54:31.401616 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.401585    2566 generic.go:358] "Generic (PLEG): container finished" podID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerID="dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c" exitCode=137
May 26 23:54:31.401616 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.401611    2566 generic.go:358] "Generic (PLEG): container finished" podID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerID="9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4" exitCode=137
May 26 23:54:31.401770 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.401652    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"
May 26 23:54:31.401770 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.401672    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerDied","Data":"dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c"}
May 26 23:54:31.401770 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.401705    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerDied","Data":"9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4"}
May 26 23:54:31.401770 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.401717    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854" event={"ID":"e465be87-74ef-4ddf-b5d1-e6b382877c6e","Type":"ContainerDied","Data":"bd8c5666cbe2bc0140f25f08f8f15ad979a0c8463324737e0d18222bb7dd00e3"}
May 26 23:54:31.401770 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.401731    2566 scope.go:117] "RemoveContainer" containerID="c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8"
May 26 23:54:31.414973 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.414947    2566 scope.go:117] "RemoveContainer" containerID="dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c"
May 26 23:54:31.422566 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.422548    2566 scope.go:117] "RemoveContainer" containerID="97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938"
May 26 23:54:31.427000 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.426974    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"]
May 26 23:54:31.431610 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.431509    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8b4b44f9d-f9854"]
May 26 23:54:31.431678 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.431620    2566 scope.go:117] "RemoveContainer" containerID="9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4"
May 26 23:54:31.438462 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.438440    2566 scope.go:117] "RemoveContainer" containerID="aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e"
May 26 23:54:31.445116 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.445099    2566 scope.go:117] "RemoveContainer" containerID="ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b"
May 26 23:54:31.453582 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.453399    2566 scope.go:117] "RemoveContainer" containerID="c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8"
May 26 23:54:31.453699 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:54:31.453678    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8\": container with ID starting with c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8 not found: ID does not exist" containerID="c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8"
May 26 23:54:31.453757 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.453707    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8"} err="failed to get container status \"c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8\": rpc error: code = NotFound desc = could not find container \"c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8\": container with ID starting with c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8 not found: ID does not exist"
May 26 23:54:31.453757 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.453726    2566 scope.go:117] "RemoveContainer" containerID="dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c"
May 26 23:54:31.454013 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:54:31.453994    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c\": container with ID starting with dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c not found: ID does not exist" containerID="dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c"
May 26 23:54:31.454080 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.454032    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c"} err="failed to get container status \"dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c\": rpc error: code = NotFound desc = could not find container \"dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c\": container with ID starting with dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c not found: ID does not exist"
May 26 23:54:31.454080 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.454049    2566 scope.go:117] "RemoveContainer" containerID="97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938"
May 26 23:54:31.454282 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:54:31.454267    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938\": container with ID starting with 97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938 not found: ID does not exist" containerID="97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938"
May 26 23:54:31.454336 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.454287    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938"} err="failed to get container status \"97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938\": rpc error: code = NotFound desc = could not find container \"97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938\": container with ID starting with 97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938 not found: ID does not exist"
May 26 23:54:31.454336 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.454301    2566 scope.go:117] "RemoveContainer" containerID="9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4"
May 26 23:54:31.454551 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:54:31.454535    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4\": container with ID starting with 9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4 not found: ID does not exist" containerID="9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4"
May 26 23:54:31.454612 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.454555    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4"} err="failed to get container status \"9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4\": rpc error: code = NotFound desc = could not find container \"9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4\": container with ID starting with 9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4 not found: ID does not exist"
May 26 23:54:31.454612 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.454568    2566 scope.go:117] "RemoveContainer" containerID="aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e"
May 26 23:54:31.454770 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:54:31.454755    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e\": container with ID starting with aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e not found: ID does not exist" containerID="aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e"
May 26 23:54:31.454817 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.454772    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e"} err="failed to get container status \"aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e\": rpc error: code = NotFound desc = could not find container \"aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e\": container with ID starting with aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e not found: ID does not exist"
May 26 23:54:31.454817 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.454784    2566 scope.go:117] "RemoveContainer" containerID="ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b"
May 26 23:54:31.455028 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:54:31.455012    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b\": container with ID starting with ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b not found: ID does not exist" containerID="ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b"
May 26 23:54:31.455081 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.455031    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b"} err="failed to get container status \"ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b\": rpc error: code = NotFound desc = could not find container \"ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b\": container with ID starting with ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b not found: ID does not exist"
May 26 23:54:31.455081 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.455044    2566 scope.go:117] "RemoveContainer" containerID="c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8"
May 26 23:54:31.455257 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.455236    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8"} err="failed to get container status \"c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8\": rpc error: code = NotFound desc = could not find container \"c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8\": container with ID starting with c7ef16cb5398c86214204c7cdb4d9ed9f3802e6453595d23af3bf8d0e1be53a8 not found: ID does not exist"
May 26 23:54:31.455327 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.455259    2566 scope.go:117] "RemoveContainer" containerID="dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c"
May 26 23:54:31.455522 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.455506    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c"} err="failed to get container status \"dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c\": rpc error: code = NotFound desc = could not find container \"dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c\": container with ID starting with dfeef9833bfc8719e985819cada08677ec41ffbfb6e412ece0fa986cfc89f39c not found: ID does not exist"
May 26 23:54:31.455571 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.455523    2566 scope.go:117] "RemoveContainer" containerID="97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938"
May 26 23:54:31.455721 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.455705    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938"} err="failed to get container status \"97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938\": rpc error: code = NotFound desc = could not find container \"97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938\": container with ID starting with 97d4f8f19eee9f20ee78a3bd7639c08a9beab26c7be602260bbb8793934bd938 not found: ID does not exist"
May 26 23:54:31.455770 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.455723    2566 scope.go:117] "RemoveContainer" containerID="9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4"
May 26 23:54:31.455992 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.455969    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4"} err="failed to get container status \"9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4\": rpc error: code = NotFound desc = could not find container \"9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4\": container with ID starting with 9d6ec2cca04a854fb1e339cf409ac56a18a97fd1a296bd6ab10390f2a91b8df4 not found: ID does not exist"
May 26 23:54:31.456084 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.456001    2566 scope.go:117] "RemoveContainer" containerID="aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e"
May 26 23:54:31.456210 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.456193    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e"} err="failed to get container status \"aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e\": rpc error: code = NotFound desc = could not find container \"aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e\": container with ID starting with aeb0f12566d490e1efab54b894fb8ed2209707638ba70c3d25987cb5aec2386e not found: ID does not exist"
May 26 23:54:31.456210 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.456209    2566 scope.go:117] "RemoveContainer" containerID="ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b"
May 26 23:54:31.456454 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:31.456432    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b"} err="failed to get container status \"ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b\": rpc error: code = NotFound desc = could not find container \"ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b\": container with ID starting with ebba11907fb3aa8a73d991ef95268907db706dc0db9781ce1a24649f8cb4062b not found: ID does not exist"
May 26 23:54:32.431194 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:54:32.431165    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" path="/var/lib/kubelet/pods/e465be87-74ef-4ddf-b5d1-e6b382877c6e/volumes"
May 26 23:55:22.513659 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.513616    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"]
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514067    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="feast-apply"
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514085    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="feast-apply"
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514105    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="feast-init"
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514113    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="feast-init"
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514123    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="ui"
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514132    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="ui"
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514154    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="registry"
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514163    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="registry"
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514171    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="online"
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514178    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="online"
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514187    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="offline"
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514194    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="offline"
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514291    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="registry"
May 26 23:55:22.514298 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514303    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="offline"
May 26 23:55:22.515026 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514318    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="online"
May 26 23:55:22.515026 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.514329    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="e465be87-74ef-4ddf-b5d1-e6b382877c6e" containerName="ui"
May 26 23:55:22.517994 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.517974    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.520229 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.520204    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 26 23:55:22.520229 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.520222    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
May 26 23:55:22.520395 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.520257    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-s6xwq\""
May 26 23:55:22.520546 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.520527    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 26 23:55:22.520689 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.520667    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 26 23:55:22.520795 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.520716    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 26 23:55:22.520795 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.520583    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 26 23:55:22.520930 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.520583    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 26 23:55:22.528128 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.528108    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"]
May 26 23:55:22.681849 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.681819    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-offline-tls\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.681849 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.681856    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d40095a-243e-456f-b9e3-08b07a964ad8-feast-data\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.682119 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.681936    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-ui-tls\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.682119 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.681958    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d40095a-243e-456f-b9e3-08b07a964ad8-registry-tls\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.682119 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.681990    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-online-tls\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.682119 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.682030    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm4nx\" (UniqueName: \"kubernetes.io/projected/7d40095a-243e-456f-b9e3-08b07a964ad8-kube-api-access-zm4nx\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.783110 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.783017    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-offline-tls\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.783110 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.783054    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d40095a-243e-456f-b9e3-08b07a964ad8-feast-data\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.783330 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:55:22.783153    2566 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 26 23:55:22.783330 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:55:22.783216    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-offline-tls podName:7d40095a-243e-456f-b9e3-08b07a964ad8 nodeName:}" failed. No retries permitted until 2026-05-26 23:55:23.283199736 +0000 UTC m=+687.459230880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-offline-tls") pod "feast-simple-feast-remote-setup-59f9866457-jtdhv" (UID: "7d40095a-243e-456f-b9e3-08b07a964ad8") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 26 23:55:22.783330 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.783236    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-ui-tls\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.783330 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.783258    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d40095a-243e-456f-b9e3-08b07a964ad8-registry-tls\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.783544 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.783381    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d40095a-243e-456f-b9e3-08b07a964ad8-feast-data\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.783544 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.783395    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-online-tls\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.783544 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.783461    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zm4nx\" (UniqueName: \"kubernetes.io/projected/7d40095a-243e-456f-b9e3-08b07a964ad8-kube-api-access-zm4nx\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.783841 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.783821    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d40095a-243e-456f-b9e3-08b07a964ad8-registry-tls\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.785765 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.785748    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-ui-tls\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.785856 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.785797    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-online-tls\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:22.791542 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:22.791518    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm4nx\" (UniqueName: \"kubernetes.io/projected/7d40095a-243e-456f-b9e3-08b07a964ad8-kube-api-access-zm4nx\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:23.288735 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:23.288696    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-offline-tls\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:23.291207 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:23.291179    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-offline-tls\") pod \"feast-simple-feast-remote-setup-59f9866457-jtdhv\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:23.429821 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:23.429788    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:23.567937 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:23.566977    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"]
May 26 23:55:23.573996 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:23.573976    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 26 23:55:24.550645 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:24.550604    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" event={"ID":"7d40095a-243e-456f-b9e3-08b07a964ad8","Type":"ContainerStarted","Data":"838a4851e7f54ffb93bf522d70dff845ef263347cf17082d0d9af39dc7b6d4d3"}
May 26 23:55:24.550825 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:24.550655    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" event={"ID":"7d40095a-243e-456f-b9e3-08b07a964ad8","Type":"ContainerStarted","Data":"9d4619ba7fb5c68e88468f6ffddc10a7c910f75127bf9bb194fb77e5b28f67aa"}
May 26 23:55:28.564453 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:28.564425    2566 generic.go:358] "Generic (PLEG): container finished" podID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerID="838a4851e7f54ffb93bf522d70dff845ef263347cf17082d0d9af39dc7b6d4d3" exitCode=0
May 26 23:55:28.564831 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:28.564499    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" event={"ID":"7d40095a-243e-456f-b9e3-08b07a964ad8","Type":"ContainerDied","Data":"838a4851e7f54ffb93bf522d70dff845ef263347cf17082d0d9af39dc7b6d4d3"}
May 26 23:55:29.569567 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:29.569529    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" event={"ID":"7d40095a-243e-456f-b9e3-08b07a964ad8","Type":"ContainerStarted","Data":"1c6a6b6a598c4ce1752286fdf841eb432a68717040d96c61299586b6607aaee9"}
May 26 23:55:34.585741 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:34.585704    2566 generic.go:358] "Generic (PLEG): container finished" podID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerID="1c6a6b6a598c4ce1752286fdf841eb432a68717040d96c61299586b6607aaee9" exitCode=0
May 26 23:55:34.586140 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:34.585777    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" event={"ID":"7d40095a-243e-456f-b9e3-08b07a964ad8","Type":"ContainerDied","Data":"1c6a6b6a598c4ce1752286fdf841eb432a68717040d96c61299586b6607aaee9"}
May 26 23:55:35.592332 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:35.592290    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" event={"ID":"7d40095a-243e-456f-b9e3-08b07a964ad8","Type":"ContainerStarted","Data":"fa889b7e351b6cae3124ead6ed0be7cd3a741ed2be4d93cd4883b5541063de3a"}
May 26 23:55:35.592703 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:35.592345    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" event={"ID":"7d40095a-243e-456f-b9e3-08b07a964ad8","Type":"ContainerStarted","Data":"ceba3e9ccbfacb370781973a71faa843a11109965a0ea46327c855bbdeecac03"}
May 26 23:55:35.592703 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:35.592360    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" event={"ID":"7d40095a-243e-456f-b9e3-08b07a964ad8","Type":"ContainerStarted","Data":"c3c679ad594304653830e6de21eb7922f6f18af183c68c1c0b59ebc9f2f3ca5f"}
May 26 23:55:35.615582 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:35.615522    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" podStartSLOduration=13.615500692 podStartE2EDuration="13.615500692s" podCreationTimestamp="2026-05-26 23:55:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 23:55:35.613568543 +0000 UTC m=+699.789599706" watchObservedRunningTime="2026-05-26 23:55:35.615500692 +0000 UTC m=+699.791531855"
May 26 23:55:38.432175 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:38.432119    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused"
May 26 23:55:38.432650 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:38.432141    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 26 23:55:38.432650 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:38.432288    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.27:8816: connect: connection refused"
May 26 23:55:38.434924 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:38.434849    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:38.434924 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:38.434884    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:38.437926 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:38.435139    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:41.431481 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:41.431451    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:41.431942 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:41.431624    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:41.432178 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:41.432149    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:41.432294 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:41.432186    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:41.432363 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:41.432308    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:41.432582 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:41.432566    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:41.435538 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:41.435521    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:41.611969 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:41.611939    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:55:41.615067 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:55:41.615048    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:56:43.636319 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:43.636288    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"]
May 26 23:56:43.636848 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:43.636701    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="online" containerID="cri-o://c3c679ad594304653830e6de21eb7922f6f18af183c68c1c0b59ebc9f2f3ca5f" gracePeriod=30
May 26 23:56:43.636848 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:43.636810    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="offline" containerID="cri-o://ceba3e9ccbfacb370781973a71faa843a11109965a0ea46327c855bbdeecac03" gracePeriod=30
May 26 23:56:43.637022 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:43.636955    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="ui" containerID="cri-o://fa889b7e351b6cae3124ead6ed0be7cd3a741ed2be4d93cd4883b5541063de3a" gracePeriod=30
May 26 23:56:43.771143 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:43.771106    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"]
May 26 23:56:43.771687 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:43.771454    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="registry" containerID="cri-o://b40b880ec209da8b139217fd88b365da658b4a75e5598798c936f171f150ca6c" gracePeriod=30
May 26 23:56:43.771687 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:43.771511    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="offline" containerID="cri-o://78e71c08a856e2ff5c6fc38f435b1627213d81684491a6b1c73fbe7ff60dc9fe" gracePeriod=30
May 26 23:56:43.771687 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:43.771528    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="online" containerID="cri-o://f319a23da81199fe474c0d1dc4e7c76e8b2cc9e91bad53dae880eafe2d42d8e5" gracePeriod=30
May 26 23:56:43.771687 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:43.771525    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="ui" containerID="cri-o://8e3f8093eaac7ef6b3885029360016654146f254711284e58acf07073e6fdce0" gracePeriod=30
May 26 23:56:44.801545 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:44.801512    2566 generic.go:358] "Generic (PLEG): container finished" podID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerID="8e3f8093eaac7ef6b3885029360016654146f254711284e58acf07073e6fdce0" exitCode=0
May 26 23:56:44.801545 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:44.801540    2566 generic.go:358] "Generic (PLEG): container finished" podID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerID="f319a23da81199fe474c0d1dc4e7c76e8b2cc9e91bad53dae880eafe2d42d8e5" exitCode=0
May 26 23:56:44.802021 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:44.801587    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerDied","Data":"8e3f8093eaac7ef6b3885029360016654146f254711284e58acf07073e6fdce0"}
May 26 23:56:44.802021 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:44.801629    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerDied","Data":"f319a23da81199fe474c0d1dc4e7c76e8b2cc9e91bad53dae880eafe2d42d8e5"}
May 26 23:56:44.803917 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:44.803884    2566 generic.go:358] "Generic (PLEG): container finished" podID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerID="fa889b7e351b6cae3124ead6ed0be7cd3a741ed2be4d93cd4883b5541063de3a" exitCode=0
May 26 23:56:44.803917 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:44.803915    2566 generic.go:358] "Generic (PLEG): container finished" podID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerID="c3c679ad594304653830e6de21eb7922f6f18af183c68c1c0b59ebc9f2f3ca5f" exitCode=0
May 26 23:56:44.804059 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:44.803926    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" event={"ID":"7d40095a-243e-456f-b9e3-08b07a964ad8","Type":"ContainerDied","Data":"fa889b7e351b6cae3124ead6ed0be7cd3a741ed2be4d93cd4883b5541063de3a"}
May 26 23:56:44.804059 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:44.803953    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" event={"ID":"7d40095a-243e-456f-b9e3-08b07a964ad8","Type":"ContainerDied","Data":"c3c679ad594304653830e6de21eb7922f6f18af183c68c1c0b59ebc9f2f3ca5f"}
May 26 23:56:49.738403 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:49.738359    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused"
May 26 23:56:50.368381 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:50.368339    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused"
May 26 23:56:51.432918 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:51.432857    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused"
May 26 23:56:51.613111 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:51.613067    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 26 23:56:59.738979 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:56:59.738935    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused"
May 26 23:57:00.367943 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:00.367862    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused"
May 26 23:57:01.432548 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:01.432506    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused"
May 26 23:57:01.613027 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:01.612992    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 26 23:57:09.739305 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:09.739198    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused"
May 26 23:57:09.739867 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:09.739376    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:57:10.368356 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:10.368314    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused"
May 26 23:57:10.368539 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:10.368463    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:57:11.432708 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:11.432665    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused"
May 26 23:57:11.433146 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:11.432793    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:57:11.613097 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:11.613053    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 26 23:57:11.613269 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:11.613259    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:57:13.888100 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:13.888067    2566 generic.go:358] "Generic (PLEG): container finished" podID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerID="78e71c08a856e2ff5c6fc38f435b1627213d81684491a6b1c73fbe7ff60dc9fe" exitCode=137
May 26 23:57:13.888100 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:13.888097    2566 generic.go:358] "Generic (PLEG): container finished" podID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerID="b40b880ec209da8b139217fd88b365da658b4a75e5598798c936f171f150ca6c" exitCode=137
May 26 23:57:13.888531 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:13.888148    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerDied","Data":"78e71c08a856e2ff5c6fc38f435b1627213d81684491a6b1c73fbe7ff60dc9fe"}
May 26 23:57:13.888531 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:13.888188    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerDied","Data":"b40b880ec209da8b139217fd88b365da658b4a75e5598798c936f171f150ca6c"}
May 26 23:57:13.890336 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:13.890316    2566 generic.go:358] "Generic (PLEG): container finished" podID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerID="ceba3e9ccbfacb370781973a71faa843a11109965a0ea46327c855bbdeecac03" exitCode=137
May 26 23:57:13.890440 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:13.890351    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" event={"ID":"7d40095a-243e-456f-b9e3-08b07a964ad8","Type":"ContainerDied","Data":"ceba3e9ccbfacb370781973a71faa843a11109965a0ea46327c855bbdeecac03"}
May 26 23:57:14.284760 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.284729    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:57:14.316514 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.316486    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-online-tls\") pod \"7d40095a-243e-456f-b9e3-08b07a964ad8\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") "
May 26 23:57:14.316683 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.316548    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-ui-tls\") pod \"7d40095a-243e-456f-b9e3-08b07a964ad8\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") "
May 26 23:57:14.316683 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.316585    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d40095a-243e-456f-b9e3-08b07a964ad8-registry-tls\") pod \"7d40095a-243e-456f-b9e3-08b07a964ad8\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") "
May 26 23:57:14.316683 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.316629    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-offline-tls\") pod \"7d40095a-243e-456f-b9e3-08b07a964ad8\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") "
May 26 23:57:14.316683 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.316655    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d40095a-243e-456f-b9e3-08b07a964ad8-feast-data\") pod \"7d40095a-243e-456f-b9e3-08b07a964ad8\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") "
May 26 23:57:14.316880 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.316692    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zm4nx\" (UniqueName: \"kubernetes.io/projected/7d40095a-243e-456f-b9e3-08b07a964ad8-kube-api-access-zm4nx\") pod \"7d40095a-243e-456f-b9e3-08b07a964ad8\" (UID: \"7d40095a-243e-456f-b9e3-08b07a964ad8\") "
May 26 23:57:14.317354 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.317182    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d40095a-243e-456f-b9e3-08b07a964ad8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7d40095a-243e-456f-b9e3-08b07a964ad8" (UID: "7d40095a-243e-456f-b9e3-08b07a964ad8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 23:57:14.317805 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.317758    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d40095a-243e-456f-b9e3-08b07a964ad8-feast-data" (OuterVolumeSpecName: "feast-data") pod "7d40095a-243e-456f-b9e3-08b07a964ad8" (UID: "7d40095a-243e-456f-b9e3-08b07a964ad8"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 23:57:14.319668 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.319638    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d40095a-243e-456f-b9e3-08b07a964ad8-kube-api-access-zm4nx" (OuterVolumeSpecName: "kube-api-access-zm4nx") pod "7d40095a-243e-456f-b9e3-08b07a964ad8" (UID: "7d40095a-243e-456f-b9e3-08b07a964ad8"). InnerVolumeSpecName "kube-api-access-zm4nx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 23:57:14.319887 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.319861    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7d40095a-243e-456f-b9e3-08b07a964ad8" (UID: "7d40095a-243e-456f-b9e3-08b07a964ad8"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:57:14.320054 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.320025    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-online-tls" (OuterVolumeSpecName: "online-tls") pod "7d40095a-243e-456f-b9e3-08b07a964ad8" (UID: "7d40095a-243e-456f-b9e3-08b07a964ad8"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:57:14.320626 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.320600    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "7d40095a-243e-456f-b9e3-08b07a964ad8" (UID: "7d40095a-243e-456f-b9e3-08b07a964ad8"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:57:14.417637 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.417614    2566 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-offline-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:57:14.417637 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.417638    2566 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d40095a-243e-456f-b9e3-08b07a964ad8-feast-data\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:57:14.417775 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.417648    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zm4nx\" (UniqueName: \"kubernetes.io/projected/7d40095a-243e-456f-b9e3-08b07a964ad8-kube-api-access-zm4nx\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:57:14.417775 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.417657    2566 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-online-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:57:14.417775 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.417667    2566 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d40095a-243e-456f-b9e3-08b07a964ad8-ui-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:57:14.417775 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.417675    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7d40095a-243e-456f-b9e3-08b07a964ad8-registry-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:57:14.422992 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.422971    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:57:14.518735 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.518708    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-online-tls\") pod \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") "
May 26 23:57:14.518921 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.518765    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qljt\" (UniqueName: \"kubernetes.io/projected/fee6e4ad-903d-414e-9835-2ae9a8b5997d-kube-api-access-8qljt\") pod \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") "
May 26 23:57:14.518921 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.518793    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fee6e4ad-903d-414e-9835-2ae9a8b5997d-feast-data\") pod \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") "
May 26 23:57:14.518921 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.518826    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-ui-tls\") pod \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") "
May 26 23:57:14.518921 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.518847    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-registry-tls\") pod \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") "
May 26 23:57:14.519139 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.518937    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-offline-tls\") pod \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\" (UID: \"fee6e4ad-903d-414e-9835-2ae9a8b5997d\") "
May 26 23:57:14.519955 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.519869    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fee6e4ad-903d-414e-9835-2ae9a8b5997d-feast-data" (OuterVolumeSpecName: "feast-data") pod "fee6e4ad-903d-414e-9835-2ae9a8b5997d" (UID: "fee6e4ad-903d-414e-9835-2ae9a8b5997d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 23:57:14.521212 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.521190    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "fee6e4ad-903d-414e-9835-2ae9a8b5997d" (UID: "fee6e4ad-903d-414e-9835-2ae9a8b5997d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:57:14.521312 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.521211    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-online-tls" (OuterVolumeSpecName: "online-tls") pod "fee6e4ad-903d-414e-9835-2ae9a8b5997d" (UID: "fee6e4ad-903d-414e-9835-2ae9a8b5997d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:57:14.521312 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.521258    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fee6e4ad-903d-414e-9835-2ae9a8b5997d" (UID: "fee6e4ad-903d-414e-9835-2ae9a8b5997d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:57:14.521583 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.521564    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "fee6e4ad-903d-414e-9835-2ae9a8b5997d" (UID: "fee6e4ad-903d-414e-9835-2ae9a8b5997d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:57:14.521630 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.521564    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fee6e4ad-903d-414e-9835-2ae9a8b5997d-kube-api-access-8qljt" (OuterVolumeSpecName: "kube-api-access-8qljt") pod "fee6e4ad-903d-414e-9835-2ae9a8b5997d" (UID: "fee6e4ad-903d-414e-9835-2ae9a8b5997d"). InnerVolumeSpecName "kube-api-access-8qljt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 23:57:14.621038 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.621009    2566 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-online-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:57:14.621038 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.621033    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qljt\" (UniqueName: \"kubernetes.io/projected/fee6e4ad-903d-414e-9835-2ae9a8b5997d-kube-api-access-8qljt\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:57:14.621038 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.621044    2566 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fee6e4ad-903d-414e-9835-2ae9a8b5997d-feast-data\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:57:14.621265 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.621053    2566 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-ui-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:57:14.621265 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.621061    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-registry-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:57:14.621265 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.621069    2566 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fee6e4ad-903d-414e-9835-2ae9a8b5997d-offline-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:57:14.895747 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.895712    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx" event={"ID":"fee6e4ad-903d-414e-9835-2ae9a8b5997d","Type":"ContainerDied","Data":"a8a3df4e4eab1e693caac5156472f5ccc45103cab04ee36a00d8a4a2c9dec6c7"}
May 26 23:57:14.896230 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.895722    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"
May 26 23:57:14.896230 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.895761    2566 scope.go:117] "RemoveContainer" containerID="8e3f8093eaac7ef6b3885029360016654146f254711284e58acf07073e6fdce0"
May 26 23:57:14.898157 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.898126    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv" event={"ID":"7d40095a-243e-456f-b9e3-08b07a964ad8","Type":"ContainerDied","Data":"9d4619ba7fb5c68e88468f6ffddc10a7c910f75127bf9bb194fb77e5b28f67aa"}
May 26 23:57:14.898157 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.898144    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"
May 26 23:57:14.905603 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.905584    2566 scope.go:117] "RemoveContainer" containerID="78e71c08a856e2ff5c6fc38f435b1627213d81684491a6b1c73fbe7ff60dc9fe"
May 26 23:57:14.913560 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.913543    2566 scope.go:117] "RemoveContainer" containerID="f319a23da81199fe474c0d1dc4e7c76e8b2cc9e91bad53dae880eafe2d42d8e5"
May 26 23:57:14.918785 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.918758    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"]
May 26 23:57:14.922028 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.922010    2566 scope.go:117] "RemoveContainer" containerID="b40b880ec209da8b139217fd88b365da658b4a75e5598798c936f171f150ca6c"
May 26 23:57:14.922683 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.922661    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-59f9866457-jtdhv"]
May 26 23:57:14.929405 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.929385    2566 scope.go:117] "RemoveContainer" containerID="950b02316a54b76ebd96fec75aa432c41155b34d4fec18f86ba78db954f849e7"
May 26 23:57:14.932717 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.932693    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"]
May 26 23:57:14.936353 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.936330    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8b4b44f9d-mtrlx"]
May 26 23:57:14.938144 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.938128    2566 scope.go:117] "RemoveContainer" containerID="81efbc2f3ad82296999d81ab4200c1f38b0957aa213bf80ff51dbf70d932674a"
May 26 23:57:14.945374 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.945358    2566 scope.go:117] "RemoveContainer" containerID="fa889b7e351b6cae3124ead6ed0be7cd3a741ed2be4d93cd4883b5541063de3a"
May 26 23:57:14.952487 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.952470    2566 scope.go:117] "RemoveContainer" containerID="ceba3e9ccbfacb370781973a71faa843a11109965a0ea46327c855bbdeecac03"
May 26 23:57:14.959614 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.959599    2566 scope.go:117] "RemoveContainer" containerID="c3c679ad594304653830e6de21eb7922f6f18af183c68c1c0b59ebc9f2f3ca5f"
May 26 23:57:14.966334 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.966319    2566 scope.go:117] "RemoveContainer" containerID="1c6a6b6a598c4ce1752286fdf841eb432a68717040d96c61299586b6607aaee9"
May 26 23:57:14.972856 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:14.972840    2566 scope.go:117] "RemoveContainer" containerID="838a4851e7f54ffb93bf522d70dff845ef263347cf17082d0d9af39dc7b6d4d3"
May 26 23:57:16.431213 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:16.431182    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" path="/var/lib/kubelet/pods/7d40095a-243e-456f-b9e3-08b07a964ad8/volumes"
May 26 23:57:16.431719 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:16.431705    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" path="/var/lib/kubelet/pods/fee6e4ad-903d-414e-9835-2ae9a8b5997d/volumes"
May 26 23:57:26.469358 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469316    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-7sght"]
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469635    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="offline"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469648    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="offline"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469662    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="feast-apply"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469669    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="feast-apply"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469680    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="ui"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469686    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="ui"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469695    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="registry"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469700    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="registry"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469705    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="feast-apply"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469711    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="feast-apply"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469717    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="online"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469722    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="online"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469730    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="feast-init"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469741    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="feast-init"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469748    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="feast-init"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469752    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="feast-init"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469759    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="online"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469764    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="online"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469769    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="offline"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469774    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="offline"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469780    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="ui"
May 26 23:57:26.469802 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469784    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="ui"
May 26 23:57:26.470619 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469835    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="online"
May 26 23:57:26.470619 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469845    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="ui"
May 26 23:57:26.470619 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469853    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="offline"
May 26 23:57:26.470619 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469860    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="offline"
May 26 23:57:26.470619 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469866    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="registry"
May 26 23:57:26.470619 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469871    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="7d40095a-243e-456f-b9e3-08b07a964ad8" containerName="ui"
May 26 23:57:26.470619 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.469877    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="fee6e4ad-903d-414e-9835-2ae9a8b5997d" containerName="online"
May 26 23:57:26.474766 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.474746    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-7sght"
May 26 23:57:26.477087 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.477063    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 26 23:57:26.477202 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.477063    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-q2hb2\""
May 26 23:57:26.477202 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.477138    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\""
May 26 23:57:26.477843 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.477830    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 26 23:57:26.480995 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.480973    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-7sght"]
May 26 23:57:26.515636 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.515610    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l2cs\" (UniqueName: \"kubernetes.io/projected/d39c69e7-cad2-44df-b4ab-29c782936457-kube-api-access-5l2cs\") pod \"postgres-7c7656cc6d-7sght\" (UID: \"d39c69e7-cad2-44df-b4ab-29c782936457\") " pod="test-ns-feast/postgres-7c7656cc6d-7sght"
May 26 23:57:26.515801 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.515685    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/d39c69e7-cad2-44df-b4ab-29c782936457-postgresdata\") pod \"postgres-7c7656cc6d-7sght\" (UID: \"d39c69e7-cad2-44df-b4ab-29c782936457\") " pod="test-ns-feast/postgres-7c7656cc6d-7sght"
May 26 23:57:26.617133 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.617099    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/d39c69e7-cad2-44df-b4ab-29c782936457-postgresdata\") pod \"postgres-7c7656cc6d-7sght\" (UID: \"d39c69e7-cad2-44df-b4ab-29c782936457\") " pod="test-ns-feast/postgres-7c7656cc6d-7sght"
May 26 23:57:26.617296 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.617176    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5l2cs\" (UniqueName: \"kubernetes.io/projected/d39c69e7-cad2-44df-b4ab-29c782936457-kube-api-access-5l2cs\") pod \"postgres-7c7656cc6d-7sght\" (UID: \"d39c69e7-cad2-44df-b4ab-29c782936457\") " pod="test-ns-feast/postgres-7c7656cc6d-7sght"
May 26 23:57:26.617459 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.617438    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/d39c69e7-cad2-44df-b4ab-29c782936457-postgresdata\") pod \"postgres-7c7656cc6d-7sght\" (UID: \"d39c69e7-cad2-44df-b4ab-29c782936457\") " pod="test-ns-feast/postgres-7c7656cc6d-7sght"
May 26 23:57:26.625608 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.625583    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l2cs\" (UniqueName: \"kubernetes.io/projected/d39c69e7-cad2-44df-b4ab-29c782936457-kube-api-access-5l2cs\") pod \"postgres-7c7656cc6d-7sght\" (UID: \"d39c69e7-cad2-44df-b4ab-29c782936457\") " pod="test-ns-feast/postgres-7c7656cc6d-7sght"
May 26 23:57:26.798648 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.798539    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-7sght"
May 26 23:57:26.923282 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.923255    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-7sght"]
May 26 23:57:26.925781 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:57:26.925752    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd39c69e7_cad2_44df_b4ab_29c782936457.slice/crio-c89fe813fc6846d10aefa14963b80b48d8c1678da85376d0fef49623680d6fad WatchSource:0}: Error finding container c89fe813fc6846d10aefa14963b80b48d8c1678da85376d0fef49623680d6fad: Status 404 returned error can't find the container with id c89fe813fc6846d10aefa14963b80b48d8c1678da85376d0fef49623680d6fad
May 26 23:57:26.934967 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:26.934941    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-7sght" event={"ID":"d39c69e7-cad2-44df-b4ab-29c782936457","Type":"ContainerStarted","Data":"c89fe813fc6846d10aefa14963b80b48d8c1678da85376d0fef49623680d6fad"}
May 26 23:57:32.955800 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:32.955763    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-7sght" event={"ID":"d39c69e7-cad2-44df-b4ab-29c782936457","Type":"ContainerStarted","Data":"e829c815053b25ce9bd65e13016026b6eba26ba223ac6633afe05b7842202d1c"}
May 26 23:57:32.970737 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:32.970693    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/postgres-7c7656cc6d-7sght" podStartSLOduration=1.3575598979999999 podStartE2EDuration="6.970679808s" podCreationTimestamp="2026-05-26 23:57:26 +0000 UTC" firstStartedPulling="2026-05-26 23:57:26.927321391 +0000 UTC m=+811.103352531" lastFinishedPulling="2026-05-26 23:57:32.54044129 +0000 UTC m=+816.716472441" observedRunningTime="2026-05-26 23:57:32.968796005 +0000 UTC m=+817.144827169" watchObservedRunningTime="2026-05-26 23:57:32.970679808 +0000 UTC m=+817.146711020"
May 26 23:57:37.208718 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.208669    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"]
May 26 23:57:37.215036 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.215015    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.217257 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.217224    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\""
May 26 23:57:37.217499 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.217477    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-bd8gf\""
May 26 23:57:37.217658 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.217491    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 26 23:57:37.217658 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.217500    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\""
May 26 23:57:37.217824 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.217502    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\""
May 26 23:57:37.230984 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.230957    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"]
May 26 23:57:37.297078 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.297053    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-registry-tls\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.297201 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.297084    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/473f3614-7fe1-45dd-a3e6-aa578a304599-feast-data\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.297201 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.297118    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-online-tls\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.297201 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.297152    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpvtk\" (UniqueName: \"kubernetes.io/projected/473f3614-7fe1-45dd-a3e6-aa578a304599-kube-api-access-lpvtk\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.297324 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.297198    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-offline-tls\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.398358 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.398323    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-registry-tls\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.398358 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.398359    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/473f3614-7fe1-45dd-a3e6-aa578a304599-feast-data\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.398558 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.398382    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-online-tls\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.398558 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.398402    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lpvtk\" (UniqueName: \"kubernetes.io/projected/473f3614-7fe1-45dd-a3e6-aa578a304599-kube-api-access-lpvtk\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.398558 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.398460    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-offline-tls\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.398797 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.398775    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/473f3614-7fe1-45dd-a3e6-aa578a304599-feast-data\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.401002 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.400973    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-registry-tls\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.401113 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.401029    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-offline-tls\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.401157 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.401130    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-online-tls\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.406276 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.406254    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpvtk\" (UniqueName: \"kubernetes.io/projected/473f3614-7fe1-45dd-a3e6-aa578a304599-kube-api-access-lpvtk\") pod \"feast-credit-scoring-6bb4d9bdcd-tfz7n\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") " pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.533133 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.533054    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:37.659271 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.659243    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"]
May 26 23:57:37.663030 ip-10-0-136-129 kubenswrapper[2566]: W0526 23:57:37.662998    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod473f3614_7fe1_45dd_a3e6_aa578a304599.slice/crio-2e68b69df21cd2958bb34073dc1bc3d7b00e522abf1533c2241f93323c4a42d7 WatchSource:0}: Error finding container 2e68b69df21cd2958bb34073dc1bc3d7b00e522abf1533c2241f93323c4a42d7: Status 404 returned error can't find the container with id 2e68b69df21cd2958bb34073dc1bc3d7b00e522abf1533c2241f93323c4a42d7
May 26 23:57:37.971990 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.971947    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" event={"ID":"473f3614-7fe1-45dd-a3e6-aa578a304599","Type":"ContainerStarted","Data":"054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2"}
May 26 23:57:37.971990 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:37.971991    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" event={"ID":"473f3614-7fe1-45dd-a3e6-aa578a304599","Type":"ContainerStarted","Data":"2e68b69df21cd2958bb34073dc1bc3d7b00e522abf1533c2241f93323c4a42d7"}
May 26 23:57:38.976280 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:38.976201    2566 generic.go:358] "Generic (PLEG): container finished" podID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerID="054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2" exitCode=0
May 26 23:57:38.976612 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:38.976291    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" event={"ID":"473f3614-7fe1-45dd-a3e6-aa578a304599","Type":"ContainerDied","Data":"054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2"}
May 26 23:57:39.981878 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:39.981840    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" event={"ID":"473f3614-7fe1-45dd-a3e6-aa578a304599","Type":"ContainerStarted","Data":"b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b"}
May 26 23:57:46.005257 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:46.005220    2566 generic.go:358] "Generic (PLEG): container finished" podID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerID="b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b" exitCode=0
May 26 23:57:46.005736 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:46.005301    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" event={"ID":"473f3614-7fe1-45dd-a3e6-aa578a304599","Type":"ContainerDied","Data":"b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b"}
May 26 23:57:47.011830 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:47.011788    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" event={"ID":"473f3614-7fe1-45dd-a3e6-aa578a304599","Type":"ContainerStarted","Data":"4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d"}
May 26 23:57:47.012216 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:47.011841    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" event={"ID":"473f3614-7fe1-45dd-a3e6-aa578a304599","Type":"ContainerStarted","Data":"b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b"}
May 26 23:57:47.012216 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:47.011861    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" event={"ID":"473f3614-7fe1-45dd-a3e6-aa578a304599","Type":"ContainerStarted","Data":"26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1"}
May 26 23:57:47.037993 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:47.037932    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" podStartSLOduration=10.037912144 podStartE2EDuration="10.037912144s" podCreationTimestamp="2026-05-26 23:57:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 23:57:47.036271476 +0000 UTC m=+831.212302635" watchObservedRunningTime="2026-05-26 23:57:47.037912144 +0000 UTC m=+831.213943298"
May 26 23:57:47.534097 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:47.534049    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:47.534267 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:47.534112    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:47.534267 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:47.534128    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:49.534014 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:49.533970    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:49.534014 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:49.534020    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:49.534611 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:49.534035    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:49.535659 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:49.535620    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused"
May 26 23:57:49.535786 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:49.535653    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused"
May 26 23:57:49.535786 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:49.535688    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 26 23:57:52.533920 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:52.533859    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused"
May 26 23:57:52.534936 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:52.534888    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:52.535842 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:52.535823    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:52.539454 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:52.539431    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:53.045164 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:53.045136    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:55.533955 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:55.533888    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused"
May 26 23:57:58.534159 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:58.534122    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:57:58.534943 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:57:58.534917    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:58:56.357855 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:56.357828    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 26 23:58:56.361493 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:56.361475    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 26 23:58:58.336087 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:58.336053    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"]
May 26 23:58:58.336601 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:58.336394    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="registry" containerID="cri-o://26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1" gracePeriod=30
May 26 23:58:58.336601 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:58.336460    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="offline" containerID="cri-o://4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d" gracePeriod=30
May 26 23:58:58.336725 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:58.336557    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="online" containerID="cri-o://b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b" gracePeriod=30
May 26 23:58:58.351632 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:58.351607    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-7sght"]
May 26 23:58:58.351837 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:58.351805    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/postgres-7c7656cc6d-7sght" podUID="d39c69e7-cad2-44df-b4ab-29c782936457" containerName="postgres" containerID="cri-o://e829c815053b25ce9bd65e13016026b6eba26ba223ac6633afe05b7842202d1c" gracePeriod=30
May 26 23:58:58.586434 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:58.586371    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-7sght"
May 26 23:58:58.691496 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:58.691445    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l2cs\" (UniqueName: \"kubernetes.io/projected/d39c69e7-cad2-44df-b4ab-29c782936457-kube-api-access-5l2cs\") pod \"d39c69e7-cad2-44df-b4ab-29c782936457\" (UID: \"d39c69e7-cad2-44df-b4ab-29c782936457\") "
May 26 23:58:58.691710 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:58.691524    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/d39c69e7-cad2-44df-b4ab-29c782936457-postgresdata\") pod \"d39c69e7-cad2-44df-b4ab-29c782936457\" (UID: \"d39c69e7-cad2-44df-b4ab-29c782936457\") "
May 26 23:58:58.693736 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:58.693702    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d39c69e7-cad2-44df-b4ab-29c782936457-kube-api-access-5l2cs" (OuterVolumeSpecName: "kube-api-access-5l2cs") pod "d39c69e7-cad2-44df-b4ab-29c782936457" (UID: "d39c69e7-cad2-44df-b4ab-29c782936457"). InnerVolumeSpecName "kube-api-access-5l2cs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 23:58:58.726885 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:58.726841    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d39c69e7-cad2-44df-b4ab-29c782936457-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "d39c69e7-cad2-44df-b4ab-29c782936457" (UID: "d39c69e7-cad2-44df-b4ab-29c782936457"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 23:58:58.792063 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:58.792027    2566 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/d39c69e7-cad2-44df-b4ab-29c782936457-postgresdata\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:58:58.792063 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:58.792055    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l2cs\" (UniqueName: \"kubernetes.io/projected/d39c69e7-cad2-44df-b4ab-29c782936457-kube-api-access-5l2cs\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:58:59.240337 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:59.240303    2566 generic.go:358] "Generic (PLEG): container finished" podID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerID="b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b" exitCode=0
May 26 23:58:59.240481 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:59.240374    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" event={"ID":"473f3614-7fe1-45dd-a3e6-aa578a304599","Type":"ContainerDied","Data":"b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b"}
May 26 23:58:59.241373 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:59.241351    2566 generic.go:358] "Generic (PLEG): container finished" podID="d39c69e7-cad2-44df-b4ab-29c782936457" containerID="e829c815053b25ce9bd65e13016026b6eba26ba223ac6633afe05b7842202d1c" exitCode=0
May 26 23:58:59.241462 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:59.241385    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-7sght" event={"ID":"d39c69e7-cad2-44df-b4ab-29c782936457","Type":"ContainerDied","Data":"e829c815053b25ce9bd65e13016026b6eba26ba223ac6633afe05b7842202d1c"}
May 26 23:58:59.241462 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:59.241408    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-7sght"
May 26 23:58:59.241462 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:59.241420    2566 scope.go:117] "RemoveContainer" containerID="e829c815053b25ce9bd65e13016026b6eba26ba223ac6633afe05b7842202d1c"
May 26 23:58:59.241571 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:59.241408    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-7sght" event={"ID":"d39c69e7-cad2-44df-b4ab-29c782936457","Type":"ContainerDied","Data":"c89fe813fc6846d10aefa14963b80b48d8c1678da85376d0fef49623680d6fad"}
May 26 23:58:59.249674 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:59.249656    2566 scope.go:117] "RemoveContainer" containerID="e829c815053b25ce9bd65e13016026b6eba26ba223ac6633afe05b7842202d1c"
May 26 23:58:59.249936 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:58:59.249914    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e829c815053b25ce9bd65e13016026b6eba26ba223ac6633afe05b7842202d1c\": container with ID starting with e829c815053b25ce9bd65e13016026b6eba26ba223ac6633afe05b7842202d1c not found: ID does not exist" containerID="e829c815053b25ce9bd65e13016026b6eba26ba223ac6633afe05b7842202d1c"
May 26 23:58:59.249991 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:59.249947    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e829c815053b25ce9bd65e13016026b6eba26ba223ac6633afe05b7842202d1c"} err="failed to get container status \"e829c815053b25ce9bd65e13016026b6eba26ba223ac6633afe05b7842202d1c\": rpc error: code = NotFound desc = could not find container \"e829c815053b25ce9bd65e13016026b6eba26ba223ac6633afe05b7842202d1c\": container with ID starting with e829c815053b25ce9bd65e13016026b6eba26ba223ac6633afe05b7842202d1c not found: ID does not exist"
May 26 23:58:59.260606 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:59.260582    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-7sght"]
May 26 23:58:59.266170 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:58:59.266149    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-7sght"]
May 26 23:59:00.430711 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:00.430677    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d39c69e7-cad2-44df-b4ab-29c782936457" path="/var/lib/kubelet/pods/d39c69e7-cad2-44df-b4ab-29c782936457/volumes"
May 26 23:59:03.042149 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:03.042107    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 26 23:59:13.042478 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:13.042441    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 26 23:59:23.042432 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:23.042387    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 26 23:59:23.042856 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:23.042535    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:59:28.535209 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:28.535170    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused"
May 26 23:59:28.985217 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:28.985194    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:59:29.041605 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.041570    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-online-tls\") pod \"473f3614-7fe1-45dd-a3e6-aa578a304599\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") "
May 26 23:59:29.041777 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.041623    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpvtk\" (UniqueName: \"kubernetes.io/projected/473f3614-7fe1-45dd-a3e6-aa578a304599-kube-api-access-lpvtk\") pod \"473f3614-7fe1-45dd-a3e6-aa578a304599\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") "
May 26 23:59:29.041777 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.041671    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-registry-tls\") pod \"473f3614-7fe1-45dd-a3e6-aa578a304599\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") "
May 26 23:59:29.041777 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.041692    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-offline-tls\") pod \"473f3614-7fe1-45dd-a3e6-aa578a304599\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") "
May 26 23:59:29.041777 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.041715    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/473f3614-7fe1-45dd-a3e6-aa578a304599-feast-data\") pod \"473f3614-7fe1-45dd-a3e6-aa578a304599\" (UID: \"473f3614-7fe1-45dd-a3e6-aa578a304599\") "
May 26 23:59:29.043970 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.043939    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-online-tls" (OuterVolumeSpecName: "online-tls") pod "473f3614-7fe1-45dd-a3e6-aa578a304599" (UID: "473f3614-7fe1-45dd-a3e6-aa578a304599"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:59:29.044081 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.043936    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/473f3614-7fe1-45dd-a3e6-aa578a304599-kube-api-access-lpvtk" (OuterVolumeSpecName: "kube-api-access-lpvtk") pod "473f3614-7fe1-45dd-a3e6-aa578a304599" (UID: "473f3614-7fe1-45dd-a3e6-aa578a304599"). InnerVolumeSpecName "kube-api-access-lpvtk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 23:59:29.044081 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.043988    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "473f3614-7fe1-45dd-a3e6-aa578a304599" (UID: "473f3614-7fe1-45dd-a3e6-aa578a304599"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:59:29.044306 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.044289    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "473f3614-7fe1-45dd-a3e6-aa578a304599" (UID: "473f3614-7fe1-45dd-a3e6-aa578a304599"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 23:59:29.050626 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.050598    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/473f3614-7fe1-45dd-a3e6-aa578a304599-feast-data" (OuterVolumeSpecName: "feast-data") pod "473f3614-7fe1-45dd-a3e6-aa578a304599" (UID: "473f3614-7fe1-45dd-a3e6-aa578a304599"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 23:59:29.142627 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.142547    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpvtk\" (UniqueName: \"kubernetes.io/projected/473f3614-7fe1-45dd-a3e6-aa578a304599-kube-api-access-lpvtk\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:59:29.142627 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.142577    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-registry-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:59:29.142627 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.142588    2566 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-offline-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:59:29.142627 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.142597    2566 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/473f3614-7fe1-45dd-a3e6-aa578a304599-feast-data\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:59:29.142627 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.142606    2566 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/473f3614-7fe1-45dd-a3e6-aa578a304599-online-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 26 23:59:29.329743 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.329711    2566 generic.go:358] "Generic (PLEG): container finished" podID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerID="4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d" exitCode=137
May 26 23:59:29.329743 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.329737    2566 generic.go:358] "Generic (PLEG): container finished" podID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerID="26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1" exitCode=137
May 26 23:59:29.329965 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.329802    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"
May 26 23:59:29.329965 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.329800    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" event={"ID":"473f3614-7fe1-45dd-a3e6-aa578a304599","Type":"ContainerDied","Data":"4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d"}
May 26 23:59:29.329965 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.329845    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" event={"ID":"473f3614-7fe1-45dd-a3e6-aa578a304599","Type":"ContainerDied","Data":"26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1"}
May 26 23:59:29.329965 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.329857    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n" event={"ID":"473f3614-7fe1-45dd-a3e6-aa578a304599","Type":"ContainerDied","Data":"2e68b69df21cd2958bb34073dc1bc3d7b00e522abf1533c2241f93323c4a42d7"}
May 26 23:59:29.329965 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.329873    2566 scope.go:117] "RemoveContainer" containerID="4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d"
May 26 23:59:29.337935 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.337911    2566 scope.go:117] "RemoveContainer" containerID="b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b"
May 26 23:59:29.347088 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.346877    2566 scope.go:117] "RemoveContainer" containerID="26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1"
May 26 23:59:29.353500 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.353479    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"]
May 26 23:59:29.354988 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.354971    2566 scope.go:117] "RemoveContainer" containerID="b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b"
May 26 23:59:29.359075 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.358977    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-6bb4d9bdcd-tfz7n"]
May 26 23:59:29.362774 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.362753    2566 scope.go:117] "RemoveContainer" containerID="054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2"
May 26 23:59:29.369501 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.369485    2566 scope.go:117] "RemoveContainer" containerID="4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d"
May 26 23:59:29.369734 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:59:29.369715    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d\": container with ID starting with 4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d not found: ID does not exist" containerID="4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d"
May 26 23:59:29.369800 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.369746    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d"} err="failed to get container status \"4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d\": rpc error: code = NotFound desc = could not find container \"4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d\": container with ID starting with 4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d not found: ID does not exist"
May 26 23:59:29.369800 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.369773    2566 scope.go:117] "RemoveContainer" containerID="b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b"
May 26 23:59:29.370002 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:59:29.369985    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b\": container with ID starting with b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b not found: ID does not exist" containerID="b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b"
May 26 23:59:29.370048 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.370010    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b"} err="failed to get container status \"b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b\": rpc error: code = NotFound desc = could not find container \"b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b\": container with ID starting with b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b not found: ID does not exist"
May 26 23:59:29.370048 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.370026    2566 scope.go:117] "RemoveContainer" containerID="26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1"
May 26 23:59:29.370241 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:59:29.370225    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1\": container with ID starting with 26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1 not found: ID does not exist" containerID="26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1"
May 26 23:59:29.370284 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.370244    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1"} err="failed to get container status \"26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1\": rpc error: code = NotFound desc = could not find container \"26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1\": container with ID starting with 26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1 not found: ID does not exist"
May 26 23:59:29.370284 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.370256    2566 scope.go:117] "RemoveContainer" containerID="b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b"
May 26 23:59:29.370451 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:59:29.370434    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b\": container with ID starting with b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b not found: ID does not exist" containerID="b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b"
May 26 23:59:29.370513 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.370460    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b"} err="failed to get container status \"b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b\": rpc error: code = NotFound desc = could not find container \"b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b\": container with ID starting with b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b not found: ID does not exist"
May 26 23:59:29.370513 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.370482    2566 scope.go:117] "RemoveContainer" containerID="054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2"
May 26 23:59:29.370700 ip-10-0-136-129 kubenswrapper[2566]: E0526 23:59:29.370684    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2\": container with ID starting with 054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2 not found: ID does not exist" containerID="054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2"
May 26 23:59:29.370742 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.370704    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2"} err="failed to get container status \"054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2\": rpc error: code = NotFound desc = could not find container \"054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2\": container with ID starting with 054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2 not found: ID does not exist"
May 26 23:59:29.370742 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.370716    2566 scope.go:117] "RemoveContainer" containerID="4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d"
May 26 23:59:29.371041 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.371024    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d"} err="failed to get container status \"4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d\": rpc error: code = NotFound desc = could not find container \"4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d\": container with ID starting with 4f6387518dc1ed502ea146e78bf7dabb6ac1f0db34c774640aaf6fdd759ecc8d not found: ID does not exist"
May 26 23:59:29.371099 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.371043    2566 scope.go:117] "RemoveContainer" containerID="b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b"
May 26 23:59:29.371239 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.371223    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b"} err="failed to get container status \"b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b\": rpc error: code = NotFound desc = could not find container \"b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b\": container with ID starting with b7f5c801c119a9131181ce3e4bd5948298190edd757a3889c39973046e033d3b not found: ID does not exist"
May 26 23:59:29.371288 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.371239    2566 scope.go:117] "RemoveContainer" containerID="26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1"
May 26 23:59:29.371463 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.371447    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1"} err="failed to get container status \"26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1\": rpc error: code = NotFound desc = could not find container \"26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1\": container with ID starting with 26c46a0a2972112afe486c096d0904173ae4f2ba08bad66bad9185fe4e6709e1 not found: ID does not exist"
May 26 23:59:29.371510 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.371462    2566 scope.go:117] "RemoveContainer" containerID="b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b"
May 26 23:59:29.371664 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.371642    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b"} err="failed to get container status \"b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b\": rpc error: code = NotFound desc = could not find container \"b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b\": container with ID starting with b3d4715a1408cc6ffea2a16c5fa9383c8a6808bfcf1c1133c76fd7df6677c02b not found: ID does not exist"
May 26 23:59:29.371708 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.371666    2566 scope.go:117] "RemoveContainer" containerID="054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2"
May 26 23:59:29.371846 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:29.371819    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2"} err="failed to get container status \"054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2\": rpc error: code = NotFound desc = could not find container \"054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2\": container with ID starting with 054c9b322f46787d55546690138aa95d8251b25ac1a58a9607b4b84681d555c2 not found: ID does not exist"
May 26 23:59:30.430831 ip-10-0-136-129 kubenswrapper[2566]: I0526 23:59:30.430798    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" path="/var/lib/kubelet/pods/473f3614-7fe1-45dd-a3e6-aa578a304599/volumes"
May 27 00:00:00.135730 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.135686    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29664000-6fm6d"]
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136128    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="online"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136146    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="online"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136169    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="feast-apply"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136177    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="feast-apply"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136192    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d39c69e7-cad2-44df-b4ab-29c782936457" containerName="postgres"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136200    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d39c69e7-cad2-44df-b4ab-29c782936457" containerName="postgres"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136212    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="feast-init"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136220    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="feast-init"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136233    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="registry"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136242    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="registry"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136260    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="offline"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136268    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="offline"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136343    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="offline"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136355    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="online"
May 27 00:00:00.136365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136366    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="473f3614-7fe1-45dd-a3e6-aa578a304599" containerName="registry"
May 27 00:00:00.137150 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.136377    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="d39c69e7-cad2-44df-b4ab-29c782936457" containerName="postgres"
May 27 00:00:00.140980 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.140958    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29664000-6fm6d"
May 27 00:00:00.143494 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.143470    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\""
May 27 00:00:00.143589 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.143470    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-jbmxw\""
May 27 00:00:00.148044 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.148015    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29664000-6fm6d"]
May 27 00:00:00.307258 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.307222    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f3a1ac08-cc8a-4440-90f4-ac17b7148aa9-serviceca\") pod \"image-pruner-29664000-6fm6d\" (UID: \"f3a1ac08-cc8a-4440-90f4-ac17b7148aa9\") " pod="openshift-image-registry/image-pruner-29664000-6fm6d"
May 27 00:00:00.307398 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.307283    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h829m\" (UniqueName: \"kubernetes.io/projected/f3a1ac08-cc8a-4440-90f4-ac17b7148aa9-kube-api-access-h829m\") pod \"image-pruner-29664000-6fm6d\" (UID: \"f3a1ac08-cc8a-4440-90f4-ac17b7148aa9\") " pod="openshift-image-registry/image-pruner-29664000-6fm6d"
May 27 00:00:00.408307 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.408221    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f3a1ac08-cc8a-4440-90f4-ac17b7148aa9-serviceca\") pod \"image-pruner-29664000-6fm6d\" (UID: \"f3a1ac08-cc8a-4440-90f4-ac17b7148aa9\") " pod="openshift-image-registry/image-pruner-29664000-6fm6d"
May 27 00:00:00.408307 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.408264    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h829m\" (UniqueName: \"kubernetes.io/projected/f3a1ac08-cc8a-4440-90f4-ac17b7148aa9-kube-api-access-h829m\") pod \"image-pruner-29664000-6fm6d\" (UID: \"f3a1ac08-cc8a-4440-90f4-ac17b7148aa9\") " pod="openshift-image-registry/image-pruner-29664000-6fm6d"
May 27 00:00:00.409053 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.409027    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f3a1ac08-cc8a-4440-90f4-ac17b7148aa9-serviceca\") pod \"image-pruner-29664000-6fm6d\" (UID: \"f3a1ac08-cc8a-4440-90f4-ac17b7148aa9\") " pod="openshift-image-registry/image-pruner-29664000-6fm6d"
May 27 00:00:00.416609 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.416576    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h829m\" (UniqueName: \"kubernetes.io/projected/f3a1ac08-cc8a-4440-90f4-ac17b7148aa9-kube-api-access-h829m\") pod \"image-pruner-29664000-6fm6d\" (UID: \"f3a1ac08-cc8a-4440-90f4-ac17b7148aa9\") " pod="openshift-image-registry/image-pruner-29664000-6fm6d"
May 27 00:00:00.468512 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.468485    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29664000-6fm6d"
May 27 00:00:00.593149 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:00.593124    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29664000-6fm6d"]
May 27 00:00:00.595844 ip-10-0-136-129 kubenswrapper[2566]: W0527 00:00:00.595815    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a1ac08_cc8a_4440_90f4_ac17b7148aa9.slice/crio-be518cd61d912c37f175dc7c866a503923c3fca2008c978efc59449b0fdc43d9 WatchSource:0}: Error finding container be518cd61d912c37f175dc7c866a503923c3fca2008c978efc59449b0fdc43d9: Status 404 returned error can't find the container with id be518cd61d912c37f175dc7c866a503923c3fca2008c978efc59449b0fdc43d9
May 27 00:00:01.422517 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:01.422488    2566 generic.go:358] "Generic (PLEG): container finished" podID="f3a1ac08-cc8a-4440-90f4-ac17b7148aa9" containerID="b785510813aebfe1529c1e886a54370d0ed1ffcad8469186fc96b286ab8a005e" exitCode=0
May 27 00:00:01.422865 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:01.422553    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29664000-6fm6d" event={"ID":"f3a1ac08-cc8a-4440-90f4-ac17b7148aa9","Type":"ContainerDied","Data":"b785510813aebfe1529c1e886a54370d0ed1ffcad8469186fc96b286ab8a005e"}
May 27 00:00:01.422865 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:01.422576    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29664000-6fm6d" event={"ID":"f3a1ac08-cc8a-4440-90f4-ac17b7148aa9","Type":"ContainerStarted","Data":"be518cd61d912c37f175dc7c866a503923c3fca2008c978efc59449b0fdc43d9"}
May 27 00:00:02.552290 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:02.552264    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29664000-6fm6d"
May 27 00:00:02.631214 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:02.631177    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h829m\" (UniqueName: \"kubernetes.io/projected/f3a1ac08-cc8a-4440-90f4-ac17b7148aa9-kube-api-access-h829m\") pod \"f3a1ac08-cc8a-4440-90f4-ac17b7148aa9\" (UID: \"f3a1ac08-cc8a-4440-90f4-ac17b7148aa9\") "
May 27 00:00:02.631389 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:02.631236    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f3a1ac08-cc8a-4440-90f4-ac17b7148aa9-serviceca\") pod \"f3a1ac08-cc8a-4440-90f4-ac17b7148aa9\" (UID: \"f3a1ac08-cc8a-4440-90f4-ac17b7148aa9\") "
May 27 00:00:02.631665 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:02.631639    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3a1ac08-cc8a-4440-90f4-ac17b7148aa9-serviceca" (OuterVolumeSpecName: "serviceca") pod "f3a1ac08-cc8a-4440-90f4-ac17b7148aa9" (UID: "f3a1ac08-cc8a-4440-90f4-ac17b7148aa9"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 27 00:00:02.633504 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:02.633483    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a1ac08-cc8a-4440-90f4-ac17b7148aa9-kube-api-access-h829m" (OuterVolumeSpecName: "kube-api-access-h829m") pod "f3a1ac08-cc8a-4440-90f4-ac17b7148aa9" (UID: "f3a1ac08-cc8a-4440-90f4-ac17b7148aa9"). InnerVolumeSpecName "kube-api-access-h829m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 00:00:02.732270 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:02.732174    2566 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f3a1ac08-cc8a-4440-90f4-ac17b7148aa9-serviceca\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:00:02.732270 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:02.732214    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h829m\" (UniqueName: \"kubernetes.io/projected/f3a1ac08-cc8a-4440-90f4-ac17b7148aa9-kube-api-access-h829m\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:00:03.430234 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:03.430193    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29664000-6fm6d" event={"ID":"f3a1ac08-cc8a-4440-90f4-ac17b7148aa9","Type":"ContainerDied","Data":"be518cd61d912c37f175dc7c866a503923c3fca2008c978efc59449b0fdc43d9"}
May 27 00:00:03.430234 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:03.430229    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be518cd61d912c37f175dc7c866a503923c3fca2008c978efc59449b0fdc43d9"
May 27 00:00:03.430439 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:00:03.430247    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29664000-6fm6d"
May 27 00:01:00.371247 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.371201    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-854544c579-d5wmm"]
May 27 00:01:00.371699 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.371621    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3a1ac08-cc8a-4440-90f4-ac17b7148aa9" containerName="image-pruner"
May 27 00:01:00.371699 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.371638    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3a1ac08-cc8a-4440-90f4-ac17b7148aa9" containerName="image-pruner"
May 27 00:01:00.371783 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.371734    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="f3a1ac08-cc8a-4440-90f4-ac17b7148aa9" containerName="image-pruner"
May 27 00:01:00.374805 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.374787    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-d5wmm"
May 27 00:01:00.377503 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.377121    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\""
May 27 00:01:00.377503 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.377320    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-kpwtr\""
May 27 00:01:00.377990 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.377936    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\""
May 27 00:01:00.379133 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.379112    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-d5wmm"]
May 27 00:01:00.504924 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.504861    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqnxw\" (UniqueName: \"kubernetes.io/projected/91bd24e3-e31e-4bf2-931d-36d9dc671b9f-kube-api-access-xqnxw\") pod \"redis-854544c579-d5wmm\" (UID: \"91bd24e3-e31e-4bf2-931d-36d9dc671b9f\") " pod="test-ns-feast-rest/redis-854544c579-d5wmm"
May 27 00:01:00.605849 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.605809    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xqnxw\" (UniqueName: \"kubernetes.io/projected/91bd24e3-e31e-4bf2-931d-36d9dc671b9f-kube-api-access-xqnxw\") pod \"redis-854544c579-d5wmm\" (UID: \"91bd24e3-e31e-4bf2-931d-36d9dc671b9f\") " pod="test-ns-feast-rest/redis-854544c579-d5wmm"
May 27 00:01:00.614664 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.614637    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqnxw\" (UniqueName: \"kubernetes.io/projected/91bd24e3-e31e-4bf2-931d-36d9dc671b9f-kube-api-access-xqnxw\") pod \"redis-854544c579-d5wmm\" (UID: \"91bd24e3-e31e-4bf2-931d-36d9dc671b9f\") " pod="test-ns-feast-rest/redis-854544c579-d5wmm"
May 27 00:01:00.691198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.691124    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-d5wmm"
May 27 00:01:00.815536 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.815512    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-d5wmm"]
May 27 00:01:00.817653 ip-10-0-136-129 kubenswrapper[2566]: W0527 00:01:00.817621    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91bd24e3_e31e_4bf2_931d_36d9dc671b9f.slice/crio-3684f0f665511bd02b530ab7c383422af009690d991dc9355f34fb0844df0dc6 WatchSource:0}: Error finding container 3684f0f665511bd02b530ab7c383422af009690d991dc9355f34fb0844df0dc6: Status 404 returned error can't find the container with id 3684f0f665511bd02b530ab7c383422af009690d991dc9355f34fb0844df0dc6
May 27 00:01:00.819422 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:00.819403    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 27 00:01:01.598083 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:01.598043    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-d5wmm" event={"ID":"91bd24e3-e31e-4bf2-931d-36d9dc671b9f","Type":"ContainerStarted","Data":"3684f0f665511bd02b530ab7c383422af009690d991dc9355f34fb0844df0dc6"}
May 27 00:01:05.611215 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:05.611170    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-d5wmm" event={"ID":"91bd24e3-e31e-4bf2-931d-36d9dc671b9f","Type":"ContainerStarted","Data":"bdac3dee5a29414accc453ed6bce8bd8556745f1322a7863e5d980be0f74671c"}
May 27 00:01:05.625638 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:05.625591    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/redis-854544c579-d5wmm" podStartSLOduration=1.275709693 podStartE2EDuration="5.625576962s" podCreationTimestamp="2026-05-27 00:01:00 +0000 UTC" firstStartedPulling="2026-05-27 00:01:00.819527662 +0000 UTC m=+1024.995558802" lastFinishedPulling="2026-05-27 00:01:05.169394929 +0000 UTC m=+1029.345426071" observedRunningTime="2026-05-27 00:01:05.624399175 +0000 UTC m=+1029.800430350" watchObservedRunningTime="2026-05-27 00:01:05.625576962 +0000 UTC m=+1029.801608123"
May 27 00:01:10.939654 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:10.939614    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"]
May 27 00:01:10.942831 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:10.942814    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"
May 27 00:01:10.945095 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:10.945079    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\""
May 27 00:01:10.951875 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:10.951855    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"]
May 27 00:01:10.980309 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:10.980276    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zj6p\" (UniqueName: \"kubernetes.io/projected/b2c3f6cf-442d-4269-8fb1-41777cf894f0-kube-api-access-6zj6p\") pod \"postgres-7c7656cc6d-7mkhr\" (UID: \"b2c3f6cf-442d-4269-8fb1-41777cf894f0\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"
May 27 00:01:10.980462 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:10.980321    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/b2c3f6cf-442d-4269-8fb1-41777cf894f0-postgresdata\") pod \"postgres-7c7656cc6d-7mkhr\" (UID: \"b2c3f6cf-442d-4269-8fb1-41777cf894f0\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"
May 27 00:01:11.081669 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:11.081631    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/b2c3f6cf-442d-4269-8fb1-41777cf894f0-postgresdata\") pod \"postgres-7c7656cc6d-7mkhr\" (UID: \"b2c3f6cf-442d-4269-8fb1-41777cf894f0\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"
May 27 00:01:11.081850 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:11.081735    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6zj6p\" (UniqueName: \"kubernetes.io/projected/b2c3f6cf-442d-4269-8fb1-41777cf894f0-kube-api-access-6zj6p\") pod \"postgres-7c7656cc6d-7mkhr\" (UID: \"b2c3f6cf-442d-4269-8fb1-41777cf894f0\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"
May 27 00:01:11.082026 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:11.082004    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/b2c3f6cf-442d-4269-8fb1-41777cf894f0-postgresdata\") pod \"postgres-7c7656cc6d-7mkhr\" (UID: \"b2c3f6cf-442d-4269-8fb1-41777cf894f0\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"
May 27 00:01:11.090820 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:11.090800    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zj6p\" (UniqueName: \"kubernetes.io/projected/b2c3f6cf-442d-4269-8fb1-41777cf894f0-kube-api-access-6zj6p\") pod \"postgres-7c7656cc6d-7mkhr\" (UID: \"b2c3f6cf-442d-4269-8fb1-41777cf894f0\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"
May 27 00:01:11.254100 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:11.254020    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"
May 27 00:01:11.374921 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:11.374867    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"]
May 27 00:01:11.378413 ip-10-0-136-129 kubenswrapper[2566]: W0527 00:01:11.378381    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2c3f6cf_442d_4269_8fb1_41777cf894f0.slice/crio-0ec05d07281bb8878bcb678d0c94df92a5789df1f42744efff92d14292446ea0 WatchSource:0}: Error finding container 0ec05d07281bb8878bcb678d0c94df92a5789df1f42744efff92d14292446ea0: Status 404 returned error can't find the container with id 0ec05d07281bb8878bcb678d0c94df92a5789df1f42744efff92d14292446ea0
May 27 00:01:11.630891 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:11.630851    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr" event={"ID":"b2c3f6cf-442d-4269-8fb1-41777cf894f0","Type":"ContainerStarted","Data":"d4a5253a7313206e8b01262ae6cb62a8e574a8dcc1b9a978468271f3786bef0e"}
May 27 00:01:11.630891 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:11.630889    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr" event={"ID":"b2c3f6cf-442d-4269-8fb1-41777cf894f0","Type":"ContainerStarted","Data":"0ec05d07281bb8878bcb678d0c94df92a5789df1f42744efff92d14292446ea0"}
May 27 00:01:11.646995 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:11.646951    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr" podStartSLOduration=1.646935842 podStartE2EDuration="1.646935842s" podCreationTimestamp="2026-05-27 00:01:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 00:01:11.64483171 +0000 UTC m=+1035.820862872" watchObservedRunningTime="2026-05-27 00:01:11.646935842 +0000 UTC m=+1035.822967003"
May 27 00:01:16.610618 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.610577    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"]
May 27 00:01:16.616931 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.616910    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:16.619611 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.619591    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\""
May 27 00:01:16.619723 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.619626    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\""
May 27 00:01:16.619723 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.619626    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\""
May 27 00:01:16.619723 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.619651    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-thrh9\""
May 27 00:01:16.624716 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.624695    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"]
May 27 00:01:16.726021 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.725990    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxvnh\" (UniqueName: \"kubernetes.io/projected/c776bbfd-4ab2-4162-a3de-36537eafe5c4-kube-api-access-lxvnh\") pod \"feast-credit-scoring-7864c47475-jwbzn\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") " pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:16.726177 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.726043    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c776bbfd-4ab2-4162-a3de-36537eafe5c4-feast-data\") pod \"feast-credit-scoring-7864c47475-jwbzn\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") " pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:16.726177 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.726124    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-registry-tls\") pod \"feast-credit-scoring-7864c47475-jwbzn\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") " pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:16.726177 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.726153    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-online-tls\") pod \"feast-credit-scoring-7864c47475-jwbzn\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") " pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:16.827250 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.827215    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-registry-tls\") pod \"feast-credit-scoring-7864c47475-jwbzn\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") " pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:16.827250 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.827255    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-online-tls\") pod \"feast-credit-scoring-7864c47475-jwbzn\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") " pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:16.827457 ip-10-0-136-129 kubenswrapper[2566]: E0527 00:01:16.827385    2566 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found
May 27 00:01:16.827457 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.827399    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lxvnh\" (UniqueName: \"kubernetes.io/projected/c776bbfd-4ab2-4162-a3de-36537eafe5c4-kube-api-access-lxvnh\") pod \"feast-credit-scoring-7864c47475-jwbzn\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") " pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:16.827523 ip-10-0-136-129 kubenswrapper[2566]: E0527 00:01:16.827462    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-registry-tls podName:c776bbfd-4ab2-4162-a3de-36537eafe5c4 nodeName:}" failed. No retries permitted until 2026-05-27 00:01:17.327440458 +0000 UTC m=+1041.503471611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-registry-tls") pod "feast-credit-scoring-7864c47475-jwbzn" (UID: "c776bbfd-4ab2-4162-a3de-36537eafe5c4") : secret "feast-credit-scoring-registry-tls" not found
May 27 00:01:16.827575 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.827521    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c776bbfd-4ab2-4162-a3de-36537eafe5c4-feast-data\") pod \"feast-credit-scoring-7864c47475-jwbzn\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") " pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:16.827925 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.827873    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c776bbfd-4ab2-4162-a3de-36537eafe5c4-feast-data\") pod \"feast-credit-scoring-7864c47475-jwbzn\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") " pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:16.830001 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.829977    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-online-tls\") pod \"feast-credit-scoring-7864c47475-jwbzn\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") " pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:16.835795 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:16.835762    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxvnh\" (UniqueName: \"kubernetes.io/projected/c776bbfd-4ab2-4162-a3de-36537eafe5c4-kube-api-access-lxvnh\") pod \"feast-credit-scoring-7864c47475-jwbzn\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") " pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:17.331100 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:17.331070    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-registry-tls\") pod \"feast-credit-scoring-7864c47475-jwbzn\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") " pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:17.333633 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:17.333614    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-registry-tls\") pod \"feast-credit-scoring-7864c47475-jwbzn\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") " pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:17.527333 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:17.527290    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:17.656050 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:17.656026    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"]
May 27 00:01:17.657931 ip-10-0-136-129 kubenswrapper[2566]: W0527 00:01:17.657883    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc776bbfd_4ab2_4162_a3de_36537eafe5c4.slice/crio-e43c95a215556fd9376b614ee929e27d8a80040de1ae8109b7cad040207215f8 WatchSource:0}: Error finding container e43c95a215556fd9376b614ee929e27d8a80040de1ae8109b7cad040207215f8: Status 404 returned error can't find the container with id e43c95a215556fd9376b614ee929e27d8a80040de1ae8109b7cad040207215f8
May 27 00:01:18.654661 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:18.654629    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" event={"ID":"c776bbfd-4ab2-4162-a3de-36537eafe5c4","Type":"ContainerStarted","Data":"dc3bd6df572cd17c3005a2767e99f8fb9ab585df1dd44a37f72629c1c75dc51e"}
May 27 00:01:18.654661 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:18.654669    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" event={"ID":"c776bbfd-4ab2-4162-a3de-36537eafe5c4","Type":"ContainerStarted","Data":"e43c95a215556fd9376b614ee929e27d8a80040de1ae8109b7cad040207215f8"}
May 27 00:01:19.659333 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:19.659295    2566 generic.go:358] "Generic (PLEG): container finished" podID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerID="dc3bd6df572cd17c3005a2767e99f8fb9ab585df1dd44a37f72629c1c75dc51e" exitCode=0
May 27 00:01:19.659778 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:19.659393    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" event={"ID":"c776bbfd-4ab2-4162-a3de-36537eafe5c4","Type":"ContainerDied","Data":"dc3bd6df572cd17c3005a2767e99f8fb9ab585df1dd44a37f72629c1c75dc51e"}
May 27 00:01:20.664661 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:20.664620    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" event={"ID":"c776bbfd-4ab2-4162-a3de-36537eafe5c4","Type":"ContainerStarted","Data":"2a8a3d787f2edcef60700fc4893ff74dd88d9d6f43b6d738e6d62bb8dbe63818"}
May 27 00:01:27.687744 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:27.687706    2566 generic.go:358] "Generic (PLEG): container finished" podID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerID="2a8a3d787f2edcef60700fc4893ff74dd88d9d6f43b6d738e6d62bb8dbe63818" exitCode=0
May 27 00:01:27.688165 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:27.687778    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" event={"ID":"c776bbfd-4ab2-4162-a3de-36537eafe5c4","Type":"ContainerDied","Data":"2a8a3d787f2edcef60700fc4893ff74dd88d9d6f43b6d738e6d62bb8dbe63818"}
May 27 00:01:28.693354 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:28.693311    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" event={"ID":"c776bbfd-4ab2-4162-a3de-36537eafe5c4","Type":"ContainerStarted","Data":"c3175ee8502ab891d1bb4dc1b9c07672502dbed31973a6b92cfb9e3779dd78b4"}
May 27 00:01:28.693861 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:28.693363    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" event={"ID":"c776bbfd-4ab2-4162-a3de-36537eafe5c4","Type":"ContainerStarted","Data":"3b319cdea5adaf3ec8542ffa6048a0cce65d7b720748227210f5d1166454d471"}
May 27 00:01:28.714111 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:28.714049    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" podStartSLOduration=12.71403056 podStartE2EDuration="12.71403056s" podCreationTimestamp="2026-05-27 00:01:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 00:01:28.712645829 +0000 UTC m=+1052.888676991" watchObservedRunningTime="2026-05-27 00:01:28.71403056 +0000 UTC m=+1052.890061725"
May 27 00:01:29.528455 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:29.528418    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:29.528653 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:29.528470    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:29.529932 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:29.529878    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused"
May 27 00:01:29.530035 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:29.529952    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 27 00:01:32.527979 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:32.527937    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused"
May 27 00:01:32.527979 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:32.527971    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 27 00:01:35.528580 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:35.528532    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused"
May 27 00:01:35.534005 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:35.533976    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:35.534371 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:35.534348    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:35.537235 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:35.537216    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:37.528425 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:37.528386    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:38.529123 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:38.529087    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:38.735736 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:38.735668    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:01:42.524427 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.524384    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"]
May 27 00:01:42.529432 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.529400    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.531677 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.531656    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-ptdzl\""
May 27 00:01:42.531677 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.531661    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\""
May 27 00:01:42.531879 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.531860    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\""
May 27 00:01:42.531976 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.531885    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\""
May 27 00:01:42.536103 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.536079    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"]
May 27 00:01:42.649221 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.649184    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5c111402-c0df-43a9-aabc-4c566fd7c27f-online-tls\") pod \"feast-driver-ranking-7f769c7b44-9z4hq\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") " pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.649397 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.649241    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzglb\" (UniqueName: \"kubernetes.io/projected/5c111402-c0df-43a9-aabc-4c566fd7c27f-kube-api-access-rzglb\") pod \"feast-driver-ranking-7f769c7b44-9z4hq\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") " pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.649397 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.649269    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5c111402-c0df-43a9-aabc-4c566fd7c27f-feast-data\") pod \"feast-driver-ranking-7f769c7b44-9z4hq\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") " pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.649496 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.649403    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5c111402-c0df-43a9-aabc-4c566fd7c27f-registry-tls\") pod \"feast-driver-ranking-7f769c7b44-9z4hq\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") " pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.750666 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.750636    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5c111402-c0df-43a9-aabc-4c566fd7c27f-online-tls\") pod \"feast-driver-ranking-7f769c7b44-9z4hq\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") " pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.750842 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.750675    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rzglb\" (UniqueName: \"kubernetes.io/projected/5c111402-c0df-43a9-aabc-4c566fd7c27f-kube-api-access-rzglb\") pod \"feast-driver-ranking-7f769c7b44-9z4hq\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") " pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.750842 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.750696    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5c111402-c0df-43a9-aabc-4c566fd7c27f-feast-data\") pod \"feast-driver-ranking-7f769c7b44-9z4hq\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") " pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.750842 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.750826    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5c111402-c0df-43a9-aabc-4c566fd7c27f-registry-tls\") pod \"feast-driver-ranking-7f769c7b44-9z4hq\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") " pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.751043 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.751029    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5c111402-c0df-43a9-aabc-4c566fd7c27f-feast-data\") pod \"feast-driver-ranking-7f769c7b44-9z4hq\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") " pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.753257 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.753235    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5c111402-c0df-43a9-aabc-4c566fd7c27f-online-tls\") pod \"feast-driver-ranking-7f769c7b44-9z4hq\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") " pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.753366 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.753309    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5c111402-c0df-43a9-aabc-4c566fd7c27f-registry-tls\") pod \"feast-driver-ranking-7f769c7b44-9z4hq\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") " pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.758698 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.758672    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzglb\" (UniqueName: \"kubernetes.io/projected/5c111402-c0df-43a9-aabc-4c566fd7c27f-kube-api-access-rzglb\") pod \"feast-driver-ranking-7f769c7b44-9z4hq\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") " pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.841641 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.841603    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:42.978070 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:42.977955    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"]
May 27 00:01:42.980915 ip-10-0-136-129 kubenswrapper[2566]: W0527 00:01:42.980869    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c111402_c0df_43a9_aabc_4c566fd7c27f.slice/crio-0535dda2b01f4fc73a13772cbae9934f5fbe984fcbae37e266b5343c62700391 WatchSource:0}: Error finding container 0535dda2b01f4fc73a13772cbae9934f5fbe984fcbae37e266b5343c62700391: Status 404 returned error can't find the container with id 0535dda2b01f4fc73a13772cbae9934f5fbe984fcbae37e266b5343c62700391
May 27 00:01:43.750946 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:43.750885    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" event={"ID":"5c111402-c0df-43a9-aabc-4c566fd7c27f","Type":"ContainerStarted","Data":"f82672c1448bda127e0c0ff9e130e68978353b65c52c4c4f9d02b0e8578da9f9"}
May 27 00:01:43.750946 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:43.750948    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" event={"ID":"5c111402-c0df-43a9-aabc-4c566fd7c27f","Type":"ContainerStarted","Data":"0535dda2b01f4fc73a13772cbae9934f5fbe984fcbae37e266b5343c62700391"}
May 27 00:01:47.764846 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:47.764759    2566 generic.go:358] "Generic (PLEG): container finished" podID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerID="f82672c1448bda127e0c0ff9e130e68978353b65c52c4c4f9d02b0e8578da9f9" exitCode=0
May 27 00:01:47.764846 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:47.764814    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" event={"ID":"5c111402-c0df-43a9-aabc-4c566fd7c27f","Type":"ContainerDied","Data":"f82672c1448bda127e0c0ff9e130e68978353b65c52c4c4f9d02b0e8578da9f9"}
May 27 00:01:48.770492 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:48.770456    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" event={"ID":"5c111402-c0df-43a9-aabc-4c566fd7c27f","Type":"ContainerStarted","Data":"a12c1117f1e9eb0c4012e1d316fc2d35d6f1a1381edf84923560dacee6a4197d"}
May 27 00:01:54.790574 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:54.790539    2566 generic.go:358] "Generic (PLEG): container finished" podID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerID="a12c1117f1e9eb0c4012e1d316fc2d35d6f1a1381edf84923560dacee6a4197d" exitCode=0
May 27 00:01:54.790954 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:54.790610    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" event={"ID":"5c111402-c0df-43a9-aabc-4c566fd7c27f","Type":"ContainerDied","Data":"a12c1117f1e9eb0c4012e1d316fc2d35d6f1a1381edf84923560dacee6a4197d"}
May 27 00:01:55.796419 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:55.796383    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" event={"ID":"5c111402-c0df-43a9-aabc-4c566fd7c27f","Type":"ContainerStarted","Data":"8fd5c64513a4eab3c83101be8006eb91b9f0170c52c7754bb6fd98e41cec197d"}
May 27 00:01:55.796419 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:55.796425    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" event={"ID":"5c111402-c0df-43a9-aabc-4c566fd7c27f","Type":"ContainerStarted","Data":"e02bdd4cfe090d0805ba45e5fc9d55db2a215455efc2f3965de063f40253ca15"}
May 27 00:01:55.818126 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:55.818059    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" podStartSLOduration=13.818036721 podStartE2EDuration="13.818036721s" podCreationTimestamp="2026-05-27 00:01:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 00:01:55.816008722 +0000 UTC m=+1079.992039883" watchObservedRunningTime="2026-05-27 00:01:55.818036721 +0000 UTC m=+1079.994067885"
May 27 00:01:57.841818 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:57.841767    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:57.842314 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:57.841833    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:01:57.843450 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:57.843421    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.34:6571: connect: connection refused"
May 27 00:01:57.843579 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:01:57.843475    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="online" probeResult="failure" output="Get \"https://10.133.0.34:6567/health\": dial tcp 10.133.0.34:6567: connect: connection refused"
May 27 00:02:00.842569 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:02:00.842525    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.34:6571: connect: connection refused"
May 27 00:02:00.909313 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:02:00.909281    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:02:00.909691 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:02:00.909659    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:02:00.912440 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:02:00.912415    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:02:02.842323 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:02:02.842282    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:02:03.842331 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:02:03.842299    2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.34:6571: connect: connection refused"
May 27 00:02:06.842551 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:02:06.842522    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:02:06.843187 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:02:06.843166    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:03:21.705204 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:21.705107    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"]
May 27 00:03:21.705767 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:21.705476    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z" podUID="2071980f-5f2e-4da2-8b5e-c63d65d7ae7c" containerName="manager" containerID="cri-o://2b4322aa713f18f157bc1c50564fc026191bc4c21c87c0688a3679308bef9220" gracePeriod=10
May 27 00:03:21.903380 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:21.903326    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"]
May 27 00:03:21.903537 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:21.903426    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"]
May 27 00:03:21.904729 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:21.904659    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="registry" containerID="cri-o://e02bdd4cfe090d0805ba45e5fc9d55db2a215455efc2f3965de063f40253ca15" gracePeriod=30
May 27 00:03:21.905191 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:21.905139    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="online" containerID="cri-o://8fd5c64513a4eab3c83101be8006eb91b9f0170c52c7754bb6fd98e41cec197d" gracePeriod=30
May 27 00:03:21.905794 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:21.905622    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="registry" containerID="cri-o://3b319cdea5adaf3ec8542ffa6048a0cce65d7b720748227210f5d1166454d471" gracePeriod=30
May 27 00:03:21.905952 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:21.905768    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="online" containerID="cri-o://c3175ee8502ab891d1bb4dc1b9c07672502dbed31973a6b92cfb9e3779dd78b4" gracePeriod=30
May 27 00:03:21.988343 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:21.988321    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"
May 27 00:03:22.063519 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:22.063481    2566 generic.go:358] "Generic (PLEG): container finished" podID="2071980f-5f2e-4da2-8b5e-c63d65d7ae7c" containerID="2b4322aa713f18f157bc1c50564fc026191bc4c21c87c0688a3679308bef9220" exitCode=0
May 27 00:03:22.063691 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:22.063538    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"
May 27 00:03:22.063691 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:22.063540    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z" event={"ID":"2071980f-5f2e-4da2-8b5e-c63d65d7ae7c","Type":"ContainerDied","Data":"2b4322aa713f18f157bc1c50564fc026191bc4c21c87c0688a3679308bef9220"}
May 27 00:03:22.063691 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:22.063582    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z" event={"ID":"2071980f-5f2e-4da2-8b5e-c63d65d7ae7c","Type":"ContainerDied","Data":"6655efe2b74fe3fffdd476567bbe63d87e519e85533c5b1732b15c986ae78191"}
May 27 00:03:22.063691 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:22.063598    2566 scope.go:117] "RemoveContainer" containerID="2b4322aa713f18f157bc1c50564fc026191bc4c21c87c0688a3679308bef9220"
May 27 00:03:22.071535 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:22.071510    2566 scope.go:117] "RemoveContainer" containerID="2b4322aa713f18f157bc1c50564fc026191bc4c21c87c0688a3679308bef9220"
May 27 00:03:22.071788 ip-10-0-136-129 kubenswrapper[2566]: E0527 00:03:22.071764    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b4322aa713f18f157bc1c50564fc026191bc4c21c87c0688a3679308bef9220\": container with ID starting with 2b4322aa713f18f157bc1c50564fc026191bc4c21c87c0688a3679308bef9220 not found: ID does not exist" containerID="2b4322aa713f18f157bc1c50564fc026191bc4c21c87c0688a3679308bef9220"
May 27 00:03:22.071858 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:22.071801    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b4322aa713f18f157bc1c50564fc026191bc4c21c87c0688a3679308bef9220"} err="failed to get container status \"2b4322aa713f18f157bc1c50564fc026191bc4c21c87c0688a3679308bef9220\": rpc error: code = NotFound desc = could not find container \"2b4322aa713f18f157bc1c50564fc026191bc4c21c87c0688a3679308bef9220\": container with ID starting with 2b4322aa713f18f157bc1c50564fc026191bc4c21c87c0688a3679308bef9220 not found: ID does not exist"
May 27 00:03:22.105403 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:22.105374    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czsbs\" (UniqueName: \"kubernetes.io/projected/2071980f-5f2e-4da2-8b5e-c63d65d7ae7c-kube-api-access-czsbs\") pod \"2071980f-5f2e-4da2-8b5e-c63d65d7ae7c\" (UID: \"2071980f-5f2e-4da2-8b5e-c63d65d7ae7c\") "
May 27 00:03:22.107818 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:22.107788    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2071980f-5f2e-4da2-8b5e-c63d65d7ae7c-kube-api-access-czsbs" (OuterVolumeSpecName: "kube-api-access-czsbs") pod "2071980f-5f2e-4da2-8b5e-c63d65d7ae7c" (UID: "2071980f-5f2e-4da2-8b5e-c63d65d7ae7c"). InnerVolumeSpecName "kube-api-access-czsbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 00:03:22.205991 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:22.205951    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czsbs\" (UniqueName: \"kubernetes.io/projected/2071980f-5f2e-4da2-8b5e-c63d65d7ae7c-kube-api-access-czsbs\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:03:22.384765 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:22.384729    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"]
May 27 00:03:22.388617 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:22.388590    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f95b9f996-nf85z"]
May 27 00:03:22.431883 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:22.431855    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2071980f-5f2e-4da2-8b5e-c63d65d7ae7c" path="/var/lib/kubelet/pods/2071980f-5f2e-4da2-8b5e-c63d65d7ae7c/volumes"
May 27 00:03:23.068658 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:23.068628    2566 generic.go:358] "Generic (PLEG): container finished" podID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerID="8fd5c64513a4eab3c83101be8006eb91b9f0170c52c7754bb6fd98e41cec197d" exitCode=0
May 27 00:03:23.069082 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:23.068708    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" event={"ID":"5c111402-c0df-43a9-aabc-4c566fd7c27f","Type":"ContainerDied","Data":"8fd5c64513a4eab3c83101be8006eb91b9f0170c52c7754bb6fd98e41cec197d"}
May 27 00:03:23.071470 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:23.071447    2566 generic.go:358] "Generic (PLEG): container finished" podID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerID="c3175ee8502ab891d1bb4dc1b9c07672502dbed31973a6b92cfb9e3779dd78b4" exitCode=0
May 27 00:03:23.071588 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:23.071477    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" event={"ID":"c776bbfd-4ab2-4162-a3de-36537eafe5c4","Type":"ContainerDied","Data":"c3175ee8502ab891d1bb4dc1b9c07672502dbed31973a6b92cfb9e3779dd78b4"}
May 27 00:03:24.201434 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.201393    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"]
May 27 00:03:24.201841 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.201599    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr" podUID="b2c3f6cf-442d-4269-8fb1-41777cf894f0" containerName="postgres" containerID="cri-o://d4a5253a7313206e8b01262ae6cb62a8e574a8dcc1b9a978468271f3786bef0e" gracePeriod=30
May 27 00:03:24.206040 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.206015    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-d5wmm"]
May 27 00:03:24.206248 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.206222    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/redis-854544c579-d5wmm" podUID="91bd24e3-e31e-4bf2-931d-36d9dc671b9f" containerName="redis" containerID="cri-o://bdac3dee5a29414accc453ed6bce8bd8556745f1322a7863e5d980be0f74671c" gracePeriod=30
May 27 00:03:24.469868 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.469843    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-d5wmm"
May 27 00:03:24.473037 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.473018    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"
May 27 00:03:24.626916 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.626860    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/b2c3f6cf-442d-4269-8fb1-41777cf894f0-postgresdata\") pod \"b2c3f6cf-442d-4269-8fb1-41777cf894f0\" (UID: \"b2c3f6cf-442d-4269-8fb1-41777cf894f0\") "
May 27 00:03:24.627067 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.626955    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqnxw\" (UniqueName: \"kubernetes.io/projected/91bd24e3-e31e-4bf2-931d-36d9dc671b9f-kube-api-access-xqnxw\") pod \"91bd24e3-e31e-4bf2-931d-36d9dc671b9f\" (UID: \"91bd24e3-e31e-4bf2-931d-36d9dc671b9f\") "
May 27 00:03:24.627067 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.626992    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zj6p\" (UniqueName: \"kubernetes.io/projected/b2c3f6cf-442d-4269-8fb1-41777cf894f0-kube-api-access-6zj6p\") pod \"b2c3f6cf-442d-4269-8fb1-41777cf894f0\" (UID: \"b2c3f6cf-442d-4269-8fb1-41777cf894f0\") "
May 27 00:03:24.629301 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.629271    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91bd24e3-e31e-4bf2-931d-36d9dc671b9f-kube-api-access-xqnxw" (OuterVolumeSpecName: "kube-api-access-xqnxw") pod "91bd24e3-e31e-4bf2-931d-36d9dc671b9f" (UID: "91bd24e3-e31e-4bf2-931d-36d9dc671b9f"). InnerVolumeSpecName "kube-api-access-xqnxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 00:03:24.629405 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.629274    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2c3f6cf-442d-4269-8fb1-41777cf894f0-kube-api-access-6zj6p" (OuterVolumeSpecName: "kube-api-access-6zj6p") pod "b2c3f6cf-442d-4269-8fb1-41777cf894f0" (UID: "b2c3f6cf-442d-4269-8fb1-41777cf894f0"). InnerVolumeSpecName "kube-api-access-6zj6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 00:03:24.655845 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.655810    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2c3f6cf-442d-4269-8fb1-41777cf894f0-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "b2c3f6cf-442d-4269-8fb1-41777cf894f0" (UID: "b2c3f6cf-442d-4269-8fb1-41777cf894f0"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 27 00:03:24.728222 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.728148    2566 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/b2c3f6cf-442d-4269-8fb1-41777cf894f0-postgresdata\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:03:24.728222 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.728176    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqnxw\" (UniqueName: \"kubernetes.io/projected/91bd24e3-e31e-4bf2-931d-36d9dc671b9f-kube-api-access-xqnxw\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:03:24.728222 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:24.728186    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zj6p\" (UniqueName: \"kubernetes.io/projected/b2c3f6cf-442d-4269-8fb1-41777cf894f0-kube-api-access-6zj6p\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:03:25.079099 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.079070    2566 generic.go:358] "Generic (PLEG): container finished" podID="b2c3f6cf-442d-4269-8fb1-41777cf894f0" containerID="d4a5253a7313206e8b01262ae6cb62a8e574a8dcc1b9a978468271f3786bef0e" exitCode=0
May 27 00:03:25.079335 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.079132    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"
May 27 00:03:25.079335 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.079155    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr" event={"ID":"b2c3f6cf-442d-4269-8fb1-41777cf894f0","Type":"ContainerDied","Data":"d4a5253a7313206e8b01262ae6cb62a8e574a8dcc1b9a978468271f3786bef0e"}
May 27 00:03:25.079335 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.079196    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-7mkhr" event={"ID":"b2c3f6cf-442d-4269-8fb1-41777cf894f0","Type":"ContainerDied","Data":"0ec05d07281bb8878bcb678d0c94df92a5789df1f42744efff92d14292446ea0"}
May 27 00:03:25.079335 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.079215    2566 scope.go:117] "RemoveContainer" containerID="d4a5253a7313206e8b01262ae6cb62a8e574a8dcc1b9a978468271f3786bef0e"
May 27 00:03:25.080266 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.080143    2566 generic.go:358] "Generic (PLEG): container finished" podID="91bd24e3-e31e-4bf2-931d-36d9dc671b9f" containerID="bdac3dee5a29414accc453ed6bce8bd8556745f1322a7863e5d980be0f74671c" exitCode=0
May 27 00:03:25.080266 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.080189    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-d5wmm"
May 27 00:03:25.080266 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.080200    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-d5wmm" event={"ID":"91bd24e3-e31e-4bf2-931d-36d9dc671b9f","Type":"ContainerDied","Data":"bdac3dee5a29414accc453ed6bce8bd8556745f1322a7863e5d980be0f74671c"}
May 27 00:03:25.080266 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.080223    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-d5wmm" event={"ID":"91bd24e3-e31e-4bf2-931d-36d9dc671b9f","Type":"ContainerDied","Data":"3684f0f665511bd02b530ab7c383422af009690d991dc9355f34fb0844df0dc6"}
May 27 00:03:25.087991 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.087966    2566 scope.go:117] "RemoveContainer" containerID="d4a5253a7313206e8b01262ae6cb62a8e574a8dcc1b9a978468271f3786bef0e"
May 27 00:03:25.088254 ip-10-0-136-129 kubenswrapper[2566]: E0527 00:03:25.088235    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4a5253a7313206e8b01262ae6cb62a8e574a8dcc1b9a978468271f3786bef0e\": container with ID starting with d4a5253a7313206e8b01262ae6cb62a8e574a8dcc1b9a978468271f3786bef0e not found: ID does not exist" containerID="d4a5253a7313206e8b01262ae6cb62a8e574a8dcc1b9a978468271f3786bef0e"
May 27 00:03:25.088335 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.088260    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4a5253a7313206e8b01262ae6cb62a8e574a8dcc1b9a978468271f3786bef0e"} err="failed to get container status \"d4a5253a7313206e8b01262ae6cb62a8e574a8dcc1b9a978468271f3786bef0e\": rpc error: code = NotFound desc = could not find container \"d4a5253a7313206e8b01262ae6cb62a8e574a8dcc1b9a978468271f3786bef0e\": container with ID starting with d4a5253a7313206e8b01262ae6cb62a8e574a8dcc1b9a978468271f3786bef0e not found: ID does not exist"
May 27 00:03:25.088335 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.088277    2566 scope.go:117] "RemoveContainer" containerID="bdac3dee5a29414accc453ed6bce8bd8556745f1322a7863e5d980be0f74671c"
May 27 00:03:25.095744 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.095720    2566 scope.go:117] "RemoveContainer" containerID="bdac3dee5a29414accc453ed6bce8bd8556745f1322a7863e5d980be0f74671c"
May 27 00:03:25.096072 ip-10-0-136-129 kubenswrapper[2566]: E0527 00:03:25.096056    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdac3dee5a29414accc453ed6bce8bd8556745f1322a7863e5d980be0f74671c\": container with ID starting with bdac3dee5a29414accc453ed6bce8bd8556745f1322a7863e5d980be0f74671c not found: ID does not exist" containerID="bdac3dee5a29414accc453ed6bce8bd8556745f1322a7863e5d980be0f74671c"
May 27 00:03:25.096131 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.096080    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdac3dee5a29414accc453ed6bce8bd8556745f1322a7863e5d980be0f74671c"} err="failed to get container status \"bdac3dee5a29414accc453ed6bce8bd8556745f1322a7863e5d980be0f74671c\": rpc error: code = NotFound desc = could not find container \"bdac3dee5a29414accc453ed6bce8bd8556745f1322a7863e5d980be0f74671c\": container with ID starting with bdac3dee5a29414accc453ed6bce8bd8556745f1322a7863e5d980be0f74671c not found: ID does not exist"
May 27 00:03:25.101797 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.101775    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"]
May 27 00:03:25.105985 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.105967    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-7mkhr"]
May 27 00:03:25.114978 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.114956    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-d5wmm"]
May 27 00:03:25.116662 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.116643    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-854544c579-d5wmm"]
May 27 00:03:25.535451 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:25.535362    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 27 00:03:26.431439 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:26.431406    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91bd24e3-e31e-4bf2-931d-36d9dc671b9f" path="/var/lib/kubelet/pods/91bd24e3-e31e-4bf2-931d-36d9dc671b9f/volumes"
May 27 00:03:26.431755 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:26.431741    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2c3f6cf-442d-4269-8fb1-41777cf894f0" path="/var/lib/kubelet/pods/b2c3f6cf-442d-4269-8fb1-41777cf894f0/volumes"
May 27 00:03:30.910373 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:30.910333    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="online" probeResult="failure" output="Get \"https://10.133.0.34:6567/health\": dial tcp 10.133.0.34:6567: connect: connection refused"
May 27 00:03:35.535220 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:35.535175    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 27 00:03:40.910219 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:40.910173    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="online" probeResult="failure" output="Get \"https://10.133.0.34:6567/health\": dial tcp 10.133.0.34:6567: connect: connection refused"
May 27 00:03:45.534589 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:45.534545    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 27 00:03:45.535085 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:45.534696    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:03:50.910675 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:50.910636    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="online" probeResult="failure" output="Get \"https://10.133.0.34:6567/health\": dial tcp 10.133.0.34:6567: connect: connection refused"
May 27 00:03:50.911049 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:50.910761    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:03:52.162818 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.162715    2566 generic.go:358] "Generic (PLEG): container finished" podID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerID="3b319cdea5adaf3ec8542ffa6048a0cce65d7b720748227210f5d1166454d471" exitCode=137
May 27 00:03:52.162818 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.162796    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" event={"ID":"c776bbfd-4ab2-4162-a3de-36537eafe5c4","Type":"ContainerDied","Data":"3b319cdea5adaf3ec8542ffa6048a0cce65d7b720748227210f5d1166454d471"}
May 27 00:03:52.164767 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.164748    2566 generic.go:358] "Generic (PLEG): container finished" podID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerID="e02bdd4cfe090d0805ba45e5fc9d55db2a215455efc2f3965de063f40253ca15" exitCode=137
May 27 00:03:52.164884 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.164806    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" event={"ID":"5c111402-c0df-43a9-aabc-4c566fd7c27f","Type":"ContainerDied","Data":"e02bdd4cfe090d0805ba45e5fc9d55db2a215455efc2f3965de063f40253ca15"}
May 27 00:03:52.551373 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.551348    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:03:52.586774 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.586747    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:03:52.657210 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.657184    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-online-tls\") pod \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") "
May 27 00:03:52.657469 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.657229    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzglb\" (UniqueName: \"kubernetes.io/projected/5c111402-c0df-43a9-aabc-4c566fd7c27f-kube-api-access-rzglb\") pod \"5c111402-c0df-43a9-aabc-4c566fd7c27f\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") "
May 27 00:03:52.657469 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.657257    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5c111402-c0df-43a9-aabc-4c566fd7c27f-registry-tls\") pod \"5c111402-c0df-43a9-aabc-4c566fd7c27f\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") "
May 27 00:03:52.657469 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.657377    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-registry-tls\") pod \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") "
May 27 00:03:52.657469 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.657427    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5c111402-c0df-43a9-aabc-4c566fd7c27f-feast-data\") pod \"5c111402-c0df-43a9-aabc-4c566fd7c27f\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") "
May 27 00:03:52.657469 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.657461    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c776bbfd-4ab2-4162-a3de-36537eafe5c4-feast-data\") pod \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") "
May 27 00:03:52.657726 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.657498    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5c111402-c0df-43a9-aabc-4c566fd7c27f-online-tls\") pod \"5c111402-c0df-43a9-aabc-4c566fd7c27f\" (UID: \"5c111402-c0df-43a9-aabc-4c566fd7c27f\") "
May 27 00:03:52.657726 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.657555    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxvnh\" (UniqueName: \"kubernetes.io/projected/c776bbfd-4ab2-4162-a3de-36537eafe5c4-kube-api-access-lxvnh\") pod \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\" (UID: \"c776bbfd-4ab2-4162-a3de-36537eafe5c4\") "
May 27 00:03:52.659646 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.659615    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c111402-c0df-43a9-aabc-4c566fd7c27f-feast-data" (OuterVolumeSpecName: "feast-data") pod "5c111402-c0df-43a9-aabc-4c566fd7c27f" (UID: "5c111402-c0df-43a9-aabc-4c566fd7c27f"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 27 00:03:52.660154 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.660116    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-online-tls" (OuterVolumeSpecName: "online-tls") pod "c776bbfd-4ab2-4162-a3de-36537eafe5c4" (UID: "c776bbfd-4ab2-4162-a3de-36537eafe5c4"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 00:03:52.660154 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.660127    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c111402-c0df-43a9-aabc-4c566fd7c27f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5c111402-c0df-43a9-aabc-4c566fd7c27f" (UID: "5c111402-c0df-43a9-aabc-4c566fd7c27f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 00:03:52.660154 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.660146    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c111402-c0df-43a9-aabc-4c566fd7c27f-kube-api-access-rzglb" (OuterVolumeSpecName: "kube-api-access-rzglb") pod "5c111402-c0df-43a9-aabc-4c566fd7c27f" (UID: "5c111402-c0df-43a9-aabc-4c566fd7c27f"). InnerVolumeSpecName "kube-api-access-rzglb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 00:03:52.660411 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.660388    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c111402-c0df-43a9-aabc-4c566fd7c27f-online-tls" (OuterVolumeSpecName: "online-tls") pod "5c111402-c0df-43a9-aabc-4c566fd7c27f" (UID: "5c111402-c0df-43a9-aabc-4c566fd7c27f"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 00:03:52.660480 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.660423    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c776bbfd-4ab2-4162-a3de-36537eafe5c4-kube-api-access-lxvnh" (OuterVolumeSpecName: "kube-api-access-lxvnh") pod "c776bbfd-4ab2-4162-a3de-36537eafe5c4" (UID: "c776bbfd-4ab2-4162-a3de-36537eafe5c4"). InnerVolumeSpecName "kube-api-access-lxvnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 00:03:52.661667 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.661640    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c776bbfd-4ab2-4162-a3de-36537eafe5c4" (UID: "c776bbfd-4ab2-4162-a3de-36537eafe5c4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 27 00:03:52.668193 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.668170    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c776bbfd-4ab2-4162-a3de-36537eafe5c4-feast-data" (OuterVolumeSpecName: "feast-data") pod "c776bbfd-4ab2-4162-a3de-36537eafe5c4" (UID: "c776bbfd-4ab2-4162-a3de-36537eafe5c4"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 27 00:03:52.758987 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.758916    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5c111402-c0df-43a9-aabc-4c566fd7c27f-registry-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:03:52.758987 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.758939    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-registry-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:03:52.758987 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.758950    2566 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5c111402-c0df-43a9-aabc-4c566fd7c27f-feast-data\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:03:52.758987 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.758958    2566 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c776bbfd-4ab2-4162-a3de-36537eafe5c4-feast-data\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:03:52.758987 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.758966    2566 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5c111402-c0df-43a9-aabc-4c566fd7c27f-online-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:03:52.758987 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.758976    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxvnh\" (UniqueName: \"kubernetes.io/projected/c776bbfd-4ab2-4162-a3de-36537eafe5c4-kube-api-access-lxvnh\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:03:52.758987 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.758985    2566 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c776bbfd-4ab2-4162-a3de-36537eafe5c4-online-tls\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:03:52.758987 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:52.758994    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzglb\" (UniqueName: \"kubernetes.io/projected/5c111402-c0df-43a9-aabc-4c566fd7c27f-kube-api-access-rzglb\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:03:53.169947 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.169882    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn" event={"ID":"c776bbfd-4ab2-4162-a3de-36537eafe5c4","Type":"ContainerDied","Data":"e43c95a215556fd9376b614ee929e27d8a80040de1ae8109b7cad040207215f8"}
May 27 00:03:53.169947 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.169945    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"
May 27 00:03:53.170456 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.169962    2566 scope.go:117] "RemoveContainer" containerID="c3175ee8502ab891d1bb4dc1b9c07672502dbed31973a6b92cfb9e3779dd78b4"
May 27 00:03:53.172276 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.172251    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq" event={"ID":"5c111402-c0df-43a9-aabc-4c566fd7c27f","Type":"ContainerDied","Data":"0535dda2b01f4fc73a13772cbae9934f5fbe984fcbae37e266b5343c62700391"}
May 27 00:03:53.172380 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.172270    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"
May 27 00:03:53.179770 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.179750    2566 scope.go:117] "RemoveContainer" containerID="3b319cdea5adaf3ec8542ffa6048a0cce65d7b720748227210f5d1166454d471"
May 27 00:03:53.187411 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.187394    2566 scope.go:117] "RemoveContainer" containerID="2a8a3d787f2edcef60700fc4893ff74dd88d9d6f43b6d738e6d62bb8dbe63818"
May 27 00:03:53.194366 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.194335    2566 scope.go:117] "RemoveContainer" containerID="dc3bd6df572cd17c3005a2767e99f8fb9ab585df1dd44a37f72629c1c75dc51e"
May 27 00:03:53.195058 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.195039    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"]
May 27 00:03:53.201115 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.201093    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7f769c7b44-9z4hq"]
May 27 00:03:53.201781 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.201756    2566 scope.go:117] "RemoveContainer" containerID="8fd5c64513a4eab3c83101be8006eb91b9f0170c52c7754bb6fd98e41cec197d"
May 27 00:03:53.208479 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.208447    2566 scope.go:117] "RemoveContainer" containerID="e02bdd4cfe090d0805ba45e5fc9d55db2a215455efc2f3965de063f40253ca15"
May 27 00:03:53.210625 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.210606    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"]
May 27 00:03:53.213329 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.213309    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7864c47475-jwbzn"]
May 27 00:03:53.216306 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.216294    2566 scope.go:117] "RemoveContainer" containerID="a12c1117f1e9eb0c4012e1d316fc2d35d6f1a1381edf84923560dacee6a4197d"
May 27 00:03:53.223049 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:53.223032    2566 scope.go:117] "RemoveContainer" containerID="f82672c1448bda127e0c0ff9e130e68978353b65c52c4c4f9d02b0e8578da9f9"
May 27 00:03:54.431241 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:54.431200    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" path="/var/lib/kubelet/pods/5c111402-c0df-43a9-aabc-4c566fd7c27f/volumes"
May 27 00:03:54.431888 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:54.431867    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" path="/var/lib/kubelet/pods/c776bbfd-4ab2-4162-a3de-36537eafe5c4/volumes"
May 27 00:03:56.380687 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:56.380660    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 27 00:03:56.383640 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:03:56.383610    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 27 00:08:56.406327 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:08:56.406293    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 27 00:08:56.407977 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:08:56.407956    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 27 00:13:56.428434 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:13:56.428404    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 27 00:13:56.431434 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:13:56.431407    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 27 00:14:19.653716 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.653673    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vp9pd/must-gather-w8fzr"]
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.653998    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="registry"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654011    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="registry"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654019    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="feast-apply"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654025    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="feast-apply"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654030    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="registry"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654036    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="registry"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654046    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c3f6cf-442d-4269-8fb1-41777cf894f0" containerName="postgres"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654051    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c3f6cf-442d-4269-8fb1-41777cf894f0" containerName="postgres"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654057    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="feast-init"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654062    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="feast-init"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654070    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="online"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654075    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="online"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654088    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="feast-init"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654093    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="feast-init"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654098    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="online"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654102    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="online"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654109    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="feast-apply"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654114    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="feast-apply"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654121    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2071980f-5f2e-4da2-8b5e-c63d65d7ae7c" containerName="manager"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654127    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2071980f-5f2e-4da2-8b5e-c63d65d7ae7c" containerName="manager"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654133    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91bd24e3-e31e-4bf2-931d-36d9dc671b9f" containerName="redis"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654137    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="91bd24e3-e31e-4bf2-931d-36d9dc671b9f" containerName="redis"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654183    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="2071980f-5f2e-4da2-8b5e-c63d65d7ae7c" containerName="manager"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654192    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="online"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654198    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="online"
May 27 00:14:19.654198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654205    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="b2c3f6cf-442d-4269-8fb1-41777cf894f0" containerName="postgres"
May 27 00:14:19.654962 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654213    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="91bd24e3-e31e-4bf2-931d-36d9dc671b9f" containerName="redis"
May 27 00:14:19.654962 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654220    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="c776bbfd-4ab2-4162-a3de-36537eafe5c4" containerName="registry"
May 27 00:14:19.654962 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.654226    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="5c111402-c0df-43a9-aabc-4c566fd7c27f" containerName="registry"
May 27 00:14:19.657147 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.657132    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp9pd/must-gather-w8fzr"
May 27 00:14:19.659311 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.659284    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vp9pd\"/\"openshift-service-ca.crt\""
May 27 00:14:19.660201 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.660184    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vp9pd\"/\"default-dockercfg-8vgdx\""
May 27 00:14:19.660290 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.660189    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vp9pd\"/\"kube-root-ca.crt\""
May 27 00:14:19.665383 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.665359    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp9pd/must-gather-w8fzr"]
May 27 00:14:19.762365 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.762328    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4sx4\" (UniqueName: \"kubernetes.io/projected/26f890ae-9e6b-4642-9f8f-3fe924de64e7-kube-api-access-n4sx4\") pod \"must-gather-w8fzr\" (UID: \"26f890ae-9e6b-4642-9f8f-3fe924de64e7\") " pod="openshift-must-gather-vp9pd/must-gather-w8fzr"
May 27 00:14:19.762535 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.762410    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/26f890ae-9e6b-4642-9f8f-3fe924de64e7-must-gather-output\") pod \"must-gather-w8fzr\" (UID: \"26f890ae-9e6b-4642-9f8f-3fe924de64e7\") " pod="openshift-must-gather-vp9pd/must-gather-w8fzr"
May 27 00:14:19.863773 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.863732    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/26f890ae-9e6b-4642-9f8f-3fe924de64e7-must-gather-output\") pod \"must-gather-w8fzr\" (UID: \"26f890ae-9e6b-4642-9f8f-3fe924de64e7\") " pod="openshift-must-gather-vp9pd/must-gather-w8fzr"
May 27 00:14:19.863978 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.863787    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n4sx4\" (UniqueName: \"kubernetes.io/projected/26f890ae-9e6b-4642-9f8f-3fe924de64e7-kube-api-access-n4sx4\") pod \"must-gather-w8fzr\" (UID: \"26f890ae-9e6b-4642-9f8f-3fe924de64e7\") " pod="openshift-must-gather-vp9pd/must-gather-w8fzr"
May 27 00:14:19.864180 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.864157    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/26f890ae-9e6b-4642-9f8f-3fe924de64e7-must-gather-output\") pod \"must-gather-w8fzr\" (UID: \"26f890ae-9e6b-4642-9f8f-3fe924de64e7\") " pod="openshift-must-gather-vp9pd/must-gather-w8fzr"
May 27 00:14:19.871174 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.871145    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4sx4\" (UniqueName: \"kubernetes.io/projected/26f890ae-9e6b-4642-9f8f-3fe924de64e7-kube-api-access-n4sx4\") pod \"must-gather-w8fzr\" (UID: \"26f890ae-9e6b-4642-9f8f-3fe924de64e7\") " pod="openshift-must-gather-vp9pd/must-gather-w8fzr"
May 27 00:14:19.972648 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:19.972559    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp9pd/must-gather-w8fzr"
May 27 00:14:20.097121 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:20.097054    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp9pd/must-gather-w8fzr"]
May 27 00:14:20.099280 ip-10-0-136-129 kubenswrapper[2566]: W0527 00:14:20.099249    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26f890ae_9e6b_4642_9f8f_3fe924de64e7.slice/crio-7ab961ad9b315061957552512402eb332f094bec894ef166594b8a785aec3315 WatchSource:0}: Error finding container 7ab961ad9b315061957552512402eb332f094bec894ef166594b8a785aec3315: Status 404 returned error can't find the container with id 7ab961ad9b315061957552512402eb332f094bec894ef166594b8a785aec3315
May 27 00:14:20.100941 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:20.100921    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 27 00:14:20.973917 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:20.973860    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp9pd/must-gather-w8fzr" event={"ID":"26f890ae-9e6b-4642-9f8f-3fe924de64e7","Type":"ContainerStarted","Data":"7ab961ad9b315061957552512402eb332f094bec894ef166594b8a785aec3315"}
May 27 00:14:27.002117 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:27.002079    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp9pd/must-gather-w8fzr" event={"ID":"26f890ae-9e6b-4642-9f8f-3fe924de64e7","Type":"ContainerStarted","Data":"7a1772078de0a514a28e8a3ffcae49d177fb8c9899ea11b3cb2b64f21b03f57e"}
May 27 00:14:27.002117 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:27.002120    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp9pd/must-gather-w8fzr" event={"ID":"26f890ae-9e6b-4642-9f8f-3fe924de64e7","Type":"ContainerStarted","Data":"e967cb0e86a8dd824f27cfcbf99141adb5ca27aa73f86a0a0724efc5451d03b2"}
May 27 00:14:27.019749 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:27.019694    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-vp9pd/must-gather-w8fzr" podStartSLOduration=2.008751619 podStartE2EDuration="8.019679503s" podCreationTimestamp="2026-05-27 00:14:19 +0000 UTC" firstStartedPulling="2026-05-27 00:14:20.101050733 +0000 UTC m=+1824.277081876" lastFinishedPulling="2026-05-27 00:14:26.111978617 +0000 UTC m=+1830.288009760" observedRunningTime="2026-05-27 00:14:27.017859381 +0000 UTC m=+1831.193890537" watchObservedRunningTime="2026-05-27 00:14:27.019679503 +0000 UTC m=+1831.195710704"
May 27 00:14:36.029665 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:36.029570    2566 generic.go:358] "Generic (PLEG): container finished" podID="26f890ae-9e6b-4642-9f8f-3fe924de64e7" containerID="e967cb0e86a8dd824f27cfcbf99141adb5ca27aa73f86a0a0724efc5451d03b2" exitCode=0
May 27 00:14:36.030080 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:36.029653    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp9pd/must-gather-w8fzr" event={"ID":"26f890ae-9e6b-4642-9f8f-3fe924de64e7","Type":"ContainerDied","Data":"e967cb0e86a8dd824f27cfcbf99141adb5ca27aa73f86a0a0724efc5451d03b2"}
May 27 00:14:36.030080 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:36.030001    2566 scope.go:117] "RemoveContainer" containerID="e967cb0e86a8dd824f27cfcbf99141adb5ca27aa73f86a0a0724efc5451d03b2"
May 27 00:14:36.760371 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:36.760342    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vp9pd_must-gather-w8fzr_26f890ae-9e6b-4642-9f8f-3fe924de64e7/gather/0.log"
May 27 00:14:39.772478 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:39.772450    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2zdft_2077a3a9-df5b-4a8c-8ed8-b7572ce31153/global-pull-secret-syncer/0.log"
May 27 00:14:39.971516 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:39.971486    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ml74s_95b1c1fa-d2ee-4912-b837-7cb5db42db65/konnectivity-agent/0.log"
May 27 00:14:40.033250 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:40.033171    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-129.ec2.internal_bebf15267ab3a3a2579421856274a9a6/haproxy/0.log"
May 27 00:14:42.102249 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.102211    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-vp9pd/must-gather-w8fzr"]
May 27 00:14:42.102644 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.102431    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-must-gather-vp9pd/must-gather-w8fzr" podUID="26f890ae-9e6b-4642-9f8f-3fe924de64e7" containerName="copy" containerID="cri-o://7a1772078de0a514a28e8a3ffcae49d177fb8c9899ea11b3cb2b64f21b03f57e" gracePeriod=2
May 27 00:14:42.104670 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.104637    2566 status_manager.go:895] "Failed to get status for pod" podUID="26f890ae-9e6b-4642-9f8f-3fe924de64e7" pod="openshift-must-gather-vp9pd/must-gather-w8fzr" err="pods \"must-gather-w8fzr\" is forbidden: User \"system:node:ip-10-0-136-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vp9pd\": no relationship found between node 'ip-10-0-136-129.ec2.internal' and this object"
May 27 00:14:42.105995 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.105964    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-vp9pd/must-gather-w8fzr"]
May 27 00:14:42.328538 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.328515    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vp9pd_must-gather-w8fzr_26f890ae-9e6b-4642-9f8f-3fe924de64e7/copy/0.log"
May 27 00:14:42.328852 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.328838    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp9pd/must-gather-w8fzr"
May 27 00:14:42.330739 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.330713    2566 status_manager.go:895] "Failed to get status for pod" podUID="26f890ae-9e6b-4642-9f8f-3fe924de64e7" pod="openshift-must-gather-vp9pd/must-gather-w8fzr" err="pods \"must-gather-w8fzr\" is forbidden: User \"system:node:ip-10-0-136-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vp9pd\": no relationship found between node 'ip-10-0-136-129.ec2.internal' and this object"
May 27 00:14:42.363062 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.363005    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/26f890ae-9e6b-4642-9f8f-3fe924de64e7-must-gather-output\") pod \"26f890ae-9e6b-4642-9f8f-3fe924de64e7\" (UID: \"26f890ae-9e6b-4642-9f8f-3fe924de64e7\") "
May 27 00:14:42.363062 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.363041    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4sx4\" (UniqueName: \"kubernetes.io/projected/26f890ae-9e6b-4642-9f8f-3fe924de64e7-kube-api-access-n4sx4\") pod \"26f890ae-9e6b-4642-9f8f-3fe924de64e7\" (UID: \"26f890ae-9e6b-4642-9f8f-3fe924de64e7\") "
May 27 00:14:42.363344 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.363315    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26f890ae-9e6b-4642-9f8f-3fe924de64e7-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "26f890ae-9e6b-4642-9f8f-3fe924de64e7" (UID: "26f890ae-9e6b-4642-9f8f-3fe924de64e7"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 27 00:14:42.365302 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.365277    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26f890ae-9e6b-4642-9f8f-3fe924de64e7-kube-api-access-n4sx4" (OuterVolumeSpecName: "kube-api-access-n4sx4") pod "26f890ae-9e6b-4642-9f8f-3fe924de64e7" (UID: "26f890ae-9e6b-4642-9f8f-3fe924de64e7"). InnerVolumeSpecName "kube-api-access-n4sx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 27 00:14:42.431569 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.431533    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26f890ae-9e6b-4642-9f8f-3fe924de64e7" path="/var/lib/kubelet/pods/26f890ae-9e6b-4642-9f8f-3fe924de64e7/volumes"
May 27 00:14:42.464039 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.464016    2566 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/26f890ae-9e6b-4642-9f8f-3fe924de64e7-must-gather-output\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:14:42.464039 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.464040    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4sx4\" (UniqueName: \"kubernetes.io/projected/26f890ae-9e6b-4642-9f8f-3fe924de64e7-kube-api-access-n4sx4\") on node \"ip-10-0-136-129.ec2.internal\" DevicePath \"\""
May 27 00:14:42.836040 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.836015    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-pp9bb_c5172d29-f6c4-42e6-a5e7-845b3115ce06/kube-state-metrics/0.log"
May 27 00:14:42.856497 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.856472    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-pp9bb_c5172d29-f6c4-42e6-a5e7-845b3115ce06/kube-rbac-proxy-main/0.log"
May 27 00:14:42.880723 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.880700    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-pp9bb_c5172d29-f6c4-42e6-a5e7-845b3115ce06/kube-rbac-proxy-self/0.log"
May 27 00:14:42.919286 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.919259    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6f796f9c6-2bvsh_83683ed4-eb7a-41cc-8820-b576c4692611/metrics-server/0.log"
May 27 00:14:42.945414 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:42.945390    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-665c9c8584-x964q_d68b3b9b-2da4-4eeb-b951-d50657d5b23c/monitoring-plugin/0.log"
May 27 00:14:43.047011 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.046978    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-928r2_cd14e3a7-dd58-4b30-aa32-84deae62c9d5/node-exporter/0.log"
May 27 00:14:43.049626 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.049610    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vp9pd_must-gather-w8fzr_26f890ae-9e6b-4642-9f8f-3fe924de64e7/copy/0.log"
May 27 00:14:43.049955 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.049935    2566 generic.go:358] "Generic (PLEG): container finished" podID="26f890ae-9e6b-4642-9f8f-3fe924de64e7" containerID="7a1772078de0a514a28e8a3ffcae49d177fb8c9899ea11b3cb2b64f21b03f57e" exitCode=143
May 27 00:14:43.050013 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.049984    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp9pd/must-gather-w8fzr"
May 27 00:14:43.050056 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.050027    2566 scope.go:117] "RemoveContainer" containerID="7a1772078de0a514a28e8a3ffcae49d177fb8c9899ea11b3cb2b64f21b03f57e"
May 27 00:14:43.057331 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.057306    2566 scope.go:117] "RemoveContainer" containerID="e967cb0e86a8dd824f27cfcbf99141adb5ca27aa73f86a0a0724efc5451d03b2"
May 27 00:14:43.066814 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.066795    2566 scope.go:117] "RemoveContainer" containerID="7a1772078de0a514a28e8a3ffcae49d177fb8c9899ea11b3cb2b64f21b03f57e"
May 27 00:14:43.067117 ip-10-0-136-129 kubenswrapper[2566]: E0527 00:14:43.067101    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a1772078de0a514a28e8a3ffcae49d177fb8c9899ea11b3cb2b64f21b03f57e\": container with ID starting with 7a1772078de0a514a28e8a3ffcae49d177fb8c9899ea11b3cb2b64f21b03f57e not found: ID does not exist" containerID="7a1772078de0a514a28e8a3ffcae49d177fb8c9899ea11b3cb2b64f21b03f57e"
May 27 00:14:43.067192 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.067126    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a1772078de0a514a28e8a3ffcae49d177fb8c9899ea11b3cb2b64f21b03f57e"} err="failed to get container status \"7a1772078de0a514a28e8a3ffcae49d177fb8c9899ea11b3cb2b64f21b03f57e\": rpc error: code = NotFound desc = could not find container \"7a1772078de0a514a28e8a3ffcae49d177fb8c9899ea11b3cb2b64f21b03f57e\": container with ID starting with 7a1772078de0a514a28e8a3ffcae49d177fb8c9899ea11b3cb2b64f21b03f57e not found: ID does not exist"
May 27 00:14:43.067192 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.067141    2566 scope.go:117] "RemoveContainer" containerID="e967cb0e86a8dd824f27cfcbf99141adb5ca27aa73f86a0a0724efc5451d03b2"
May 27 00:14:43.067396 ip-10-0-136-129 kubenswrapper[2566]: E0527 00:14:43.067364    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e967cb0e86a8dd824f27cfcbf99141adb5ca27aa73f86a0a0724efc5451d03b2\": container with ID starting with e967cb0e86a8dd824f27cfcbf99141adb5ca27aa73f86a0a0724efc5451d03b2 not found: ID does not exist" containerID="e967cb0e86a8dd824f27cfcbf99141adb5ca27aa73f86a0a0724efc5451d03b2"
May 27 00:14:43.067453 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.067409    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e967cb0e86a8dd824f27cfcbf99141adb5ca27aa73f86a0a0724efc5451d03b2"} err="failed to get container status \"e967cb0e86a8dd824f27cfcbf99141adb5ca27aa73f86a0a0724efc5451d03b2\": rpc error: code = NotFound desc = could not find container \"e967cb0e86a8dd824f27cfcbf99141adb5ca27aa73f86a0a0724efc5451d03b2\": container with ID starting with e967cb0e86a8dd824f27cfcbf99141adb5ca27aa73f86a0a0724efc5451d03b2 not found: ID does not exist"
May 27 00:14:43.068734 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.068719    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-928r2_cd14e3a7-dd58-4b30-aa32-84deae62c9d5/kube-rbac-proxy/0.log"
May 27 00:14:43.088240 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.088187    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-928r2_cd14e3a7-dd58-4b30-aa32-84deae62c9d5/init-textfile/0.log"
May 27 00:14:43.434122 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.434040    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-cw9c4_cf6447a1-0f11-4d83-9518-853e223c171e/prometheus-operator/0.log"
May 27 00:14:43.455756 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.455732    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-cw9c4_cf6447a1-0f11-4d83-9518-853e223c171e/kube-rbac-proxy/0.log"
May 27 00:14:43.477997 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.477967    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-j244p_6e0c95d8-bccf-4800-8b47-0888e194da94/prometheus-operator-admission-webhook/0.log"
May 27 00:14:43.570303 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.570278    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59cb967f8-vsddq_0c800763-6b48-4319-882c-ea1637f6b41b/thanos-query/0.log"
May 27 00:14:43.589319 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.589297    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59cb967f8-vsddq_0c800763-6b48-4319-882c-ea1637f6b41b/kube-rbac-proxy-web/0.log"
May 27 00:14:43.608475 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.608457    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59cb967f8-vsddq_0c800763-6b48-4319-882c-ea1637f6b41b/kube-rbac-proxy/0.log"
May 27 00:14:43.630921 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.630883    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59cb967f8-vsddq_0c800763-6b48-4319-882c-ea1637f6b41b/prom-label-proxy/0.log"
May 27 00:14:43.652543 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.652521    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59cb967f8-vsddq_0c800763-6b48-4319-882c-ea1637f6b41b/kube-rbac-proxy-rules/0.log"
May 27 00:14:43.672731 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:43.672711    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59cb967f8-vsddq_0c800763-6b48-4319-882c-ea1637f6b41b/kube-rbac-proxy-metrics/0.log"
May 27 00:14:45.203121 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:45.203086    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/1.log"
May 27 00:14:45.208161 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:45.208132    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kwxzl_9375de25-ec32-4041-9c93-f241a8bc94ca/console-operator/2.log"
May 27 00:14:46.551009 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.550981    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"]
May 27 00:14:46.551426 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.551266    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26f890ae-9e6b-4642-9f8f-3fe924de64e7" containerName="gather"
May 27 00:14:46.551426 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.551278    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="26f890ae-9e6b-4642-9f8f-3fe924de64e7" containerName="gather"
May 27 00:14:46.551426 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.551295    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26f890ae-9e6b-4642-9f8f-3fe924de64e7" containerName="copy"
May 27 00:14:46.551426 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.551300    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="26f890ae-9e6b-4642-9f8f-3fe924de64e7" containerName="copy"
May 27 00:14:46.551426 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.551356    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="26f890ae-9e6b-4642-9f8f-3fe924de64e7" containerName="copy"
May 27 00:14:46.551426 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.551365    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="26f890ae-9e6b-4642-9f8f-3fe924de64e7" containerName="gather"
May 27 00:14:46.554661 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.554646    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.557048 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.557028    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-twt48\"/\"openshift-service-ca.crt\""
May 27 00:14:46.557760 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.557747    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-twt48\"/\"kube-root-ca.crt\""
May 27 00:14:46.557760 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.557754    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-twt48\"/\"default-dockercfg-lcfnn\""
May 27 00:14:46.560345 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.560322    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"]
May 27 00:14:46.595864 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.595833    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35c4718f-443d-4299-90d3-572cb46b1739-sys\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.595864 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.595863    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/35c4718f-443d-4299-90d3-572cb46b1739-proc\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.596099 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.595885    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnl6s\" (UniqueName: \"kubernetes.io/projected/35c4718f-443d-4299-90d3-572cb46b1739-kube-api-access-rnl6s\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.596099 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.596013    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/35c4718f-443d-4299-90d3-572cb46b1739-lib-modules\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.596099 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.596059    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/35c4718f-443d-4299-90d3-572cb46b1739-podres\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.618159 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.618136    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lzfn9_7f61ca38-a898-4dea-9951-8550a6fb2629/dns/0.log"
May 27 00:14:46.637050 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.637028    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lzfn9_7f61ca38-a898-4dea-9951-8550a6fb2629/kube-rbac-proxy/0.log"
May 27 00:14:46.658314 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.658294    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2bg9t_f31218b0-fbd7-43cf-b279-47cc71a12652/dns-node-resolver/0.log"
May 27 00:14:46.696658 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.696632    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/35c4718f-443d-4299-90d3-572cb46b1739-lib-modules\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.696748 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.696666    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/35c4718f-443d-4299-90d3-572cb46b1739-podres\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.696748 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.696717    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35c4718f-443d-4299-90d3-572cb46b1739-sys\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.696748 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.696733    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/35c4718f-443d-4299-90d3-572cb46b1739-proc\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.696953 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.696753    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rnl6s\" (UniqueName: \"kubernetes.io/projected/35c4718f-443d-4299-90d3-572cb46b1739-kube-api-access-rnl6s\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.696953 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.696804    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/35c4718f-443d-4299-90d3-572cb46b1739-lib-modules\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.696953 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.696844    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35c4718f-443d-4299-90d3-572cb46b1739-sys\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.696953 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.696891    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/35c4718f-443d-4299-90d3-572cb46b1739-podres\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.696953 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.696921    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/35c4718f-443d-4299-90d3-572cb46b1739-proc\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.705297 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.705276    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnl6s\" (UniqueName: \"kubernetes.io/projected/35c4718f-443d-4299-90d3-572cb46b1739-kube-api-access-rnl6s\") pod \"perf-node-gather-daemonset-nvhps\" (UID: \"35c4718f-443d-4299-90d3-572cb46b1739\") " pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.865964 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.865855    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:46.989660 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:46.989628    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"]
May 27 00:14:46.992848 ip-10-0-136-129 kubenswrapper[2566]: W0527 00:14:46.992819    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod35c4718f_443d_4299_90d3_572cb46b1739.slice/crio-76afea9ba742721391b1a009bf5fd6fbd61609290848ff049acb6ec96c679c86 WatchSource:0}: Error finding container 76afea9ba742721391b1a009bf5fd6fbd61609290848ff049acb6ec96c679c86: Status 404 returned error can't find the container with id 76afea9ba742721391b1a009bf5fd6fbd61609290848ff049acb6ec96c679c86
May 27 00:14:47.063222 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:47.063189    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps" event={"ID":"35c4718f-443d-4299-90d3-572cb46b1739","Type":"ContainerStarted","Data":"76afea9ba742721391b1a009bf5fd6fbd61609290848ff049acb6ec96c679c86"}
May 27 00:14:47.085123 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:47.085096    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29664000-6fm6d_f3a1ac08-cc8a-4440-90f4-ac17b7148aa9/image-pruner/0.log"
May 27 00:14:47.114571 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:47.114542    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-99c8764b-wrs4q_46c63e21-7e21-4033-96c3-697183ff9a0e/registry/0.log"
May 27 00:14:47.176881 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:47.176818    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vjw25_52bec009-08b7-4931-8c9f-50435f32eb50/node-ca/0.log"
May 27 00:14:48.067122 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:48.067086    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps" event={"ID":"35c4718f-443d-4299-90d3-572cb46b1739","Type":"ContainerStarted","Data":"e4ce09db98ea9378aca33687af494cc59dc42039f3486a17480ed35b05629bd1"}
May 27 00:14:48.067511 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:48.067207    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:48.082328 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:48.082282    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps" podStartSLOduration=2.082268745 podStartE2EDuration="2.082268745s" podCreationTimestamp="2026-05-27 00:14:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 00:14:48.081830668 +0000 UTC m=+1852.257861842" watchObservedRunningTime="2026-05-27 00:14:48.082268745 +0000 UTC m=+1852.258299904"
May 27 00:14:48.125264 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:48.125239    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9jd8w_c09ef6c1-6d34-446d-ba8b-f0bcc511af50/serve-healthcheck-canary/0.log"
May 27 00:14:48.527611 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:48.527539    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b8svw_7f3892b4-11de-40ef-8b92-a974a0082ecd/kube-rbac-proxy/0.log"
May 27 00:14:48.546026 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:48.546003    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b8svw_7f3892b4-11de-40ef-8b92-a974a0082ecd/exporter/0.log"
May 27 00:14:48.565832 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:48.565810    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b8svw_7f3892b4-11de-40ef-8b92-a974a0082ecd/extractor/0.log"
May 27 00:14:52.386198 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:52.386169    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-54hw7_eea2ccfc-3469-4a45-b2dd-e099acb0bbd1/migrator/0.log"
May 27 00:14:52.404173 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:52.404149    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-54hw7_eea2ccfc-3469-4a45-b2dd-e099acb0bbd1/graceful-termination/0.log"
May 27 00:14:52.662356 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:52.662244    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-d4prq_6000982f-147b-41ee-ba1f-00ccc23543a2/kube-storage-version-migrator-operator/1.log"
May 27 00:14:52.663136 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:52.663115    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-d4prq_6000982f-147b-41ee-ba1f-00ccc23543a2/kube-storage-version-migrator-operator/0.log"
May 27 00:14:53.677254 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:53.677226    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vjq8g_8e2588a1-9d18-4ea2-8ec4-e14b672058b4/kube-multus-additional-cni-plugins/0.log"
May 27 00:14:53.695539 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:53.695515    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vjq8g_8e2588a1-9d18-4ea2-8ec4-e14b672058b4/egress-router-binary-copy/0.log"
May 27 00:14:53.713877 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:53.713853    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vjq8g_8e2588a1-9d18-4ea2-8ec4-e14b672058b4/cni-plugins/0.log"
May 27 00:14:53.737402 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:53.737377    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vjq8g_8e2588a1-9d18-4ea2-8ec4-e14b672058b4/bond-cni-plugin/0.log"
May 27 00:14:53.757324 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:53.757303    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vjq8g_8e2588a1-9d18-4ea2-8ec4-e14b672058b4/routeoverride-cni/0.log"
May 27 00:14:53.775967 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:53.775946    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vjq8g_8e2588a1-9d18-4ea2-8ec4-e14b672058b4/whereabouts-cni-bincopy/0.log"
May 27 00:14:53.794248 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:53.794226    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vjq8g_8e2588a1-9d18-4ea2-8ec4-e14b672058b4/whereabouts-cni/0.log"
May 27 00:14:53.866067 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:53.866047    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pmsgm_a26f4c89-994b-46d2-aae3-67c5c13e7f0a/kube-multus/0.log"
May 27 00:14:53.922467 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:53.922443    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6mdld_6d68dfa4-c945-4139-b299-2664709a5651/network-metrics-daemon/0.log"
May 27 00:14:53.939869 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:53.939811    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6mdld_6d68dfa4-c945-4139-b299-2664709a5651/kube-rbac-proxy/0.log"
May 27 00:14:54.080241 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:54.080211    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-twt48/perf-node-gather-daemonset-nvhps"
May 27 00:14:54.881134 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:54.881105    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh9zh_fc54c161-1832-427d-b1b4-5e89b5f06223/ovn-controller/0.log"
May 27 00:14:54.904467 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:54.904440    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh9zh_fc54c161-1832-427d-b1b4-5e89b5f06223/ovn-acl-logging/0.log"
May 27 00:14:54.921600 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:54.921565    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh9zh_fc54c161-1832-427d-b1b4-5e89b5f06223/kube-rbac-proxy-node/0.log"
May 27 00:14:54.941987 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:54.941951    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh9zh_fc54c161-1832-427d-b1b4-5e89b5f06223/kube-rbac-proxy-ovn-metrics/0.log"
May 27 00:14:54.957186 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:54.957155    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh9zh_fc54c161-1832-427d-b1b4-5e89b5f06223/northd/0.log"
May 27 00:14:54.977306 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:54.977281    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh9zh_fc54c161-1832-427d-b1b4-5e89b5f06223/nbdb/0.log"
May 27 00:14:54.997139 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:54.997115    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh9zh_fc54c161-1832-427d-b1b4-5e89b5f06223/sbdb/0.log"
May 27 00:14:55.084548 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:55.084480    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nh9zh_fc54c161-1832-427d-b1b4-5e89b5f06223/ovnkube-controller/0.log"
May 27 00:14:56.416548 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:56.416521    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-fplbc_969b31ff-aeec-4ea2-aaae-9675876698b0/network-check-target-container/0.log"
May 27 00:14:57.344161 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:57.344140    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-z68mm_c98e9301-f2aa-41c5-a412-6cc60125e170/iptables-alerter/0.log"
May 27 00:14:57.961038 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:14:57.960946    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-4r7ws_5534bef0-4f4f-42e7-84d1-dd88853880b8/tuned/0.log"
May 27 00:15:00.666846 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:15:00.666818    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca_service-ca-67d667c7d-9fh4q_20b6e215-dc0a-4c47-a22f-61e685bc3b55/service-ca-controller/0.log"
May 27 00:15:01.117742 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:15:01.117713    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-zjxxb_c684f499-3fe8-4ae1-9c4b-8969786d1079/csi-driver/0.log"
May 27 00:15:01.137843 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:15:01.137814    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-zjxxb_c684f499-3fe8-4ae1-9c4b-8969786d1079/csi-node-driver-registrar/0.log"
May 27 00:15:01.159256 ip-10-0-136-129 kubenswrapper[2566]: I0527 00:15:01.159228    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-zjxxb_c684f499-3fe8-4ae1-9c4b-8969786d1079/csi-liveness-probe/0.log"