May 28 18:46:55.605022 ip-10-0-137-226 systemd[1]: Starting Kubernetes Kubelet...
May 28 18:46:56.104804 ip-10-0-137-226 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 18:46:56.104804 ip-10-0-137-226 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 18:46:56.104804 ip-10-0-137-226 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 18:46:56.104804 ip-10-0-137-226 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 18:46:56.104804 ip-10-0-137-226 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 18:46:56.107890 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.107794    2577 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 18:46:56.110639 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110623    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 18:46:56.110639 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110639    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110643    2577 feature_gate.go:328] unrecognized feature gate: Example2
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110646    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110649    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110652    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110654    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110657    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110661    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110665    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110669    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110673    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110676    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110679    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110686    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110690    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110692    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110695    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110698    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110701    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 18:46:56.110706 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110703    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110706    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110709    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110712    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110715    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110717    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110720    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110723    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110725    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110729    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110732    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110735    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110737    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110740    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110743    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110745    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110747    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110750    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110752    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 18:46:56.111158 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110755    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110757    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110760    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110762    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110764    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110767    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110769    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110772    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110775    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110777    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110780    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110782    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110785    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110787    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110789    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110793    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110796    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110798    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110801    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110803    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 18:46:56.111651 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110806    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110809    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110811    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110814    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110817    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110820    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110822    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110825    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110827    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110830    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110832    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110834    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110837    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110839    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110842    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110844    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110847    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110849    2577 feature_gate.go:328] unrecognized feature gate: Example
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110851    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110854    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 18:46:56.112180 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110856    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110859    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110861    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110864    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110866    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110869    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.110872    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111279    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111284    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111288    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111291    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111295    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111299    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111302    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111305    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111308    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111311    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111315    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111318    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 18:46:56.112752 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111321    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111323    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111326    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111329    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111331    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111334    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111336    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111340    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111343    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111345    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111348    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111350    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111353    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111355    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111359    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111362    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111365    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111367    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111370    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111372    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 18:46:56.113248 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111377    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111379    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111382    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111385    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111387    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111390    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111392    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111394    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111397    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111399    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111402    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111404    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111407    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111410    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111412    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111414    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111417    2577 feature_gate.go:328] unrecognized feature gate: Example
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111419    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111422    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111424    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 18:46:56.113764 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111427    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111429    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111432    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111434    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111437    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111439    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111443    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111446    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111449    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111452    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111454    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111457    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111461    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111463    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111466    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111468    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111471    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111473    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111476    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 18:46:56.114287 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111479    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111481    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111484    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111486    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111488    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111491    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111493    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111496    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111498    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111501    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111503    2577 feature_gate.go:328] unrecognized feature gate: Example2
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111506    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111508    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111510    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.111513    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112210    2577 flags.go:64] FLAG: --address="0.0.0.0"
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112221    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112228    2577 flags.go:64] FLAG: --anonymous-auth="true"
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112233    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112237    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112240    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 18:46:56.114801 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112245    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112251    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112254    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112257    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112261    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112264    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112267    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112270    2577 flags.go:64] FLAG: --cgroup-root=""
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112274    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112277    2577 flags.go:64] FLAG: --client-ca-file=""
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112280    2577 flags.go:64] FLAG: --cloud-config=""
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112283    2577 flags.go:64] FLAG: --cloud-provider="external"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112286    2577 flags.go:64] FLAG: --cluster-dns="[]"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112289    2577 flags.go:64] FLAG: --cluster-domain=""
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112292    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112295    2577 flags.go:64] FLAG: --config-dir=""
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112298    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112302    2577 flags.go:64] FLAG: --container-log-max-files="5"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112306    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112309    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112312    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112315    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112318    2577 flags.go:64] FLAG: --contention-profiling="false"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112321    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 18:46:56.115397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112324    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112327    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112331    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112335    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112338    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112341    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112344    2577 flags.go:64] FLAG: --enable-load-reader="false"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112348    2577 flags.go:64] FLAG: --enable-server="true"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112351    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112361    2577 flags.go:64] FLAG: --event-burst="100"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112365    2577 flags.go:64] FLAG: --event-qps="50"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112368    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112371    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112374    2577 flags.go:64] FLAG: --eviction-hard=""
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112378    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112381    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112384    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112387    2577 flags.go:64] FLAG: --eviction-soft=""
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112390    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112393    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112396    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112399    2577 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112402    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112405    2577 flags.go:64] FLAG: --fail-swap-on="true"
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112409    2577 flags.go:64] FLAG: --feature-gates=""
May 28 18:46:56.116062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112413    2577 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112416    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112419    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112422    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112425    2577 flags.go:64] FLAG: --healthz-port="10248"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112428    2577 flags.go:64] FLAG: --help="false"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112431    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-137-226.ec2.internal"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112435    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112439    2577 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112442    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112445    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112449    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112452    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112455    2577 flags.go:64] FLAG: --image-service-endpoint=""
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112458    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112461    2577 flags.go:64] FLAG: --kube-api-burst="100"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112464    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112467    2577 flags.go:64] FLAG: --kube-api-qps="50"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112470    2577 flags.go:64] FLAG: --kube-reserved=""
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112473    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112476    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112479    2577 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112482    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112485    2577 flags.go:64] FLAG: --lock-file=""
May 28 18:46:56.116760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112488    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112491    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112494    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112499    2577 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112502    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112504    2577 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112507    2577 flags.go:64] FLAG: --logging-format="text"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112511    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112514    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112517    2577 flags.go:64] FLAG: --manifest-url=""
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112520    2577 flags.go:64] FLAG: --manifest-url-header=""
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112524    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112527    2577 flags.go:64] FLAG: --max-open-files="1000000"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112532    2577 flags.go:64] FLAG: --max-pods="110"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112535    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112538    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112543    2577 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112546    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112549    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112552    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112555    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112563    2577 flags.go:64] FLAG: --node-status-max-images="50"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112566    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112569    2577 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 18:46:56.117398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112572    2577 flags.go:64] FLAG: --pod-cidr=""
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112575    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112582    2577 flags.go:64] FLAG: --pod-manifest-path=""
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112585    2577 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112589    2577 flags.go:64] FLAG: --pods-per-core="0"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112592    2577 flags.go:64] FLAG: --port="10250"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112595    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112597    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0070981e1afd2c9e8"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112601    2577 flags.go:64] FLAG: --qos-reserved=""
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112618    2577 flags.go:64] FLAG: --read-only-port="10255"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112622    2577 flags.go:64] FLAG: --register-node="true"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112625    2577 flags.go:64] FLAG: --register-schedulable="true"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112628    2577 flags.go:64] FLAG: --register-with-taints=""
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112631    2577 flags.go:64] FLAG: --registry-burst="10"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112636    2577 flags.go:64] FLAG: --registry-qps="5"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112639    2577 flags.go:64] FLAG: --reserved-cpus=""
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112642    2577 flags.go:64] FLAG: --reserved-memory=""
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112646    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112649    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112652    2577 flags.go:64] FLAG: --rotate-certificates="false"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112655    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112658    2577 flags.go:64] FLAG: --runonce="false"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112661    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112664    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112667    2577 flags.go:64] FLAG: --seccomp-default="false"
May 28 18:46:56.118082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112671    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112674    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112677    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112681    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112684    2577 flags.go:64] FLAG: --storage-driver-password="root"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112687    2577 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112690    2577 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112693    2577 flags.go:64] FLAG: --storage-driver-user="root"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112696    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112699    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112703    2577 flags.go:64] FLAG: --system-cgroups=""
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112706    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112711    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112714    2577 flags.go:64] FLAG: --tls-cert-file=""
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112716    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112721    2577 flags.go:64] FLAG: --tls-min-version=""
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112724    2577 flags.go:64] FLAG: --tls-private-key-file=""
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112726    2577 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112729    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112732    2577 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112735    2577 flags.go:64] FLAG: --v="2"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112741    2577 flags.go:64] FLAG: --version="false"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112745    2577 flags.go:64] FLAG: --vmodule=""
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112750    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.112753    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 18:46:56.118763 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112850    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112854    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112856    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112860    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112862    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112865    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112868    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112872    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112875    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112877    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112880    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112884    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112888    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112891    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112894    2577 feature_gate.go:328] unrecognized feature gate: Example
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112897    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112900    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112902    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112905    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 18:46:56.119400 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112908    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112911    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112913    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112916    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112918    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112920    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112923    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112926    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112928    2577 feature_gate.go:328] unrecognized feature gate: Example2
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112932    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112935    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112937    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112940    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112942    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112945    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112948    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112952    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112955    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112957    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112960    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 18:46:56.119874 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112964    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112966    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112969    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112971    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112974    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112976    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112979    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112981    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112984    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112986    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112989    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112992    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112995    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.112997    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113000    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113003    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113005    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113008    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113011    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113013    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 18:46:56.120700 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113016    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113019    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113022    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113025    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113028    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113030    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113032    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113035    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113037    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113040    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113043    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113045    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113049    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113052    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113054    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113057    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113059    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113062    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113064    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113067    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 18:46:56.121474 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113069    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 18:46:56.122004 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113072    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 18:46:56.122004 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113074    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 18:46:56.122004 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113077    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 18:46:56.122004 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113079    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 18:46:56.122004 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113082    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 18:46:56.122004 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.113085    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 18:46:56.122004 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.113865    2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 18:46:56.122004 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.121864    2577 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 18:46:56.122004 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.121986    2577 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122037    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122042    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122046    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122049    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122052    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122055    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122058    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122060    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122063    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122066    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122069    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122071    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122074    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122077    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122079    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122082    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122084    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122088    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 18:46:56.122232 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122092    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122095    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122098    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122101    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122103    2577 feature_gate.go:328] unrecognized feature gate: Example2
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122105    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122109    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122114    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122117    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122119    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122122    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122124    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122127    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122130    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122132    2577 feature_gate.go:328] unrecognized feature gate: Example
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122136    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122139    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122141    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122144    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122147    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 18:46:56.122747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122149    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122152    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122154    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122157    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122159    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122164    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122166    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122169    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122172    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122174    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122178    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122180    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122183    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122186    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122188    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122191    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122193    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122196    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122198    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122201    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 18:46:56.123255 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122203    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122206    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122209    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122212    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122214    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122217    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122220    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122223    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122226    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122228    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122231    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122233    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122236    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122239    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122241    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122244    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122246    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122249    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122253    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122256    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 18:46:56.123771 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122259    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.122261    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123120    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123125    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123128    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123131    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123134    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123136    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.123141    2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123246    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123253    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123256    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123259    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123263    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123267    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 18:46:56.124261 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123270    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123274    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123277    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123280    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123283    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123286    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123289    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123291    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123294    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123296    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123299    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123301    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123304    2577 feature_gate.go:328] unrecognized feature gate: Example
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123307    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123309    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123312    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123315    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123318    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123320    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123323    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 18:46:56.124652 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123326    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123328    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123331    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123333    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123336    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123338    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123341    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123343    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123346    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123348    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123351    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123353    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123356    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123359    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123361    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123364    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123366    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123369    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123372    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123374    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 18:46:56.125155 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123377    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123380    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123382    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123385    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123387    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123390    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123393    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123395    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123398    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123400    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123403    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123406    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123408    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123411    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123413    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123416    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123418    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123421    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123423    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123426    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 18:46:56.125666 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123428    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123431    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123433    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123436    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123438    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123440    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123443    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123446    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123449    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123452    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123455    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123457    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123460    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123463    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123465    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123468    2577 feature_gate.go:328] unrecognized feature gate: Example2
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123470    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123473    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123475    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 18:46:56.126154 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:56.123478    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 18:46:56.126620 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.123483    2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 18:46:56.126620 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.124293    2577 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 18:46:56.127461 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.127447    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 18:46:56.128905 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.128893    2577 server.go:1019] "Starting client certificate rotation"
May 28 18:46:56.129012 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.128995    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 18:46:56.129045 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.129035    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 18:46:56.157243 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.157222    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 18:46:56.175451 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.161488    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 18:46:56.178977 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.178960    2577 log.go:25] "Validated CRI v1 runtime API"
May 28 18:46:56.184805 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.184788    2577 log.go:25] "Validated CRI v1 image API"
May 28 18:46:56.188161 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.188138    2577 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 18:46:56.190513 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.190495    2577 fs.go:135] Filesystem UUIDs: map[62b1d98b-8562-4415-8537-3644b3cfc071:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7dca97bb-bbbf-4aaa-b646-3a46f1afef2d:/dev/nvme0n1p3]
May 28 18:46:56.190579 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.190514    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 18:46:56.196896 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.196780    2577 manager.go:217] Machine: {Timestamp:2026-05-28 18:46:56.196263207 +0000 UTC m=+0.455485364 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098288 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec257943d0011f87e0e51579bb1daee3 SystemUUID:ec257943-d001-1f87-e0e5-1579bb1daee3 BootID:bb2c7d4a-2078-4eae-84c8-e0200a610822 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6e:92:a1:25:bf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6e:92:a1:25:bf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:e1:50:a0:83:4f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 18:46:56.196896 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.196881    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 18:46:56.197035 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.197006    2577 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 18:46:56.200074 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.200048    2577 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 18:46:56.200221 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.200077    2577 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-226.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 18:46:56.200275 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.200231    2577 topology_manager.go:138] "Creating topology manager with none policy"
May 28 18:46:56.200275 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.200240    2577 container_manager_linux.go:306] "Creating device plugin manager"
May 28 18:46:56.200275 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.200248    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 18:46:56.200275 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.200267    2577 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 18:46:56.202361 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.202348    2577 state_mem.go:36] "Initialized new in-memory state store"
May 28 18:46:56.202467 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.202458    2577 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 18:46:56.203311 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.203290    2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 18:46:56.205137 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.205126    2577 kubelet.go:491] "Attempting to sync node with API server"
May 28 18:46:56.205191 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.205141    2577 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 18:46:56.205883 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.205873    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 18:46:56.205924 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.205887    2577 kubelet.go:397] "Adding apiserver pod source"
May 28 18:46:56.205924 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.205897    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 18:46:56.207134 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.207117    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 18:46:56.207199 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.207146    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 18:46:56.211509 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.211490    2577 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 18:46:56.213580 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.213564    2577 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 18:46:56.214498 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.214487    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 18:46:56.214545 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.214505    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 18:46:56.214545 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.214511    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 18:46:56.214545 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.214516    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 18:46:56.214545 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.214524    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 18:46:56.214545 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.214530    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 18:46:56.214545 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.214536    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 18:46:56.214545 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.214542    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 18:46:56.214746 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.214549    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 18:46:56.214746 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.214554    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 18:46:56.214746 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.214570    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 18:46:56.214819 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.214769    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 18:46:56.215646 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.215636    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 18:46:56.215682 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.215646    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 18:46:56.217881 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.217859    2577 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 28 18:46:56.217881 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.217880    2577 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-226.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 28 18:46:56.219045 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.219031    2577 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 18:46:56.219093 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.219070    2577 server.go:1295] "Started kubelet"
May 28 18:46:56.219229 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.219178    2577 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 18:46:56.219304 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.219254    2577 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 18:46:56.219342 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.219242    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 18:46:56.219863 ip-10-0-137-226 systemd[1]: Started Kubernetes Kubelet.
May 28 18:46:56.220812 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.220565    2577 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 18:46:56.222417 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.222403    2577 server.go:317] "Adding debug handlers to kubelet server"
May 28 18:46:56.222497 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.222415    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f9m9t"
May 28 18:46:56.226658 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.226638    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 18:46:56.227279 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.227263    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 18:46:56.227473 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.227458    2577 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 18:46:56.227535 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.227474    2577 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 18:46:56.227665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.227648    2577 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 18:46:56.228199 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.228181    2577 reconstruct.go:97] "Volume reconstruction finished"
May 28 18:46:56.228199 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.228197    2577 reconciler.go:26] "Reconciler: start to sync state"
May 28 18:46:56.228420 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.228385    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:56.228560 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.228459    2577 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 28 18:46:56.228806 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.228792    2577 factory.go:55] Registering systemd factory
May 28 18:46:56.228895 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.228845    2577 factory.go:223] Registration of the systemd container factory successfully
May 28 18:46:56.229148 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.229133    2577 factory.go:153] Registering CRI-O factory
May 28 18:46:56.229187 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.229153    2577 factory.go:223] Registration of the crio container factory successfully
May 28 18:46:56.229215 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.229204    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 18:46:56.229266 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.229221    2577 factory.go:103] Registering Raw factory
May 28 18:46:56.229266 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.229234    2577 manager.go:1196] Started watching for new ooms in manager
May 28 18:46:56.230529 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.230502    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f9m9t"
May 28 18:46:56.230674 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.230658    2577 manager.go:319] Starting recovery of all containers
May 28 18:46:56.231203 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.231182    2577 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-226.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 28 18:46:56.231274 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.231200    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-226.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 28 18:46:56.231330 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.231300    2577 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 28 18:46:56.232490 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.231392    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18b3cf207f0e9df4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-05-28 18:46:56.219045364 +0000 UTC m=+0.478267520,LastTimestamp:2026-05-28 18:46:56.219045364 +0000 UTC m=+0.478267520,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
May 28 18:46:56.242641 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.242624    2577 manager.go:324] Recovery completed
May 28 18:46:56.246767 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.246750    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 18:46:56.248806 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.248788    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
May 28 18:46:56.248888 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.248824    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
May 28 18:46:56.248888 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.248837    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
May 28 18:46:56.249403 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.249388    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 18:46:56.249403 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.249400    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 18:46:56.249518 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.249446    2577 state_mem.go:36] "Initialized new in-memory state store"
May 28 18:46:56.251870 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.251857    2577 policy_none.go:49] "None policy: Start"
May 28 18:46:56.251924 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.251873    2577 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 18:46:56.251924 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.251884    2577 state_mem.go:35] "Initializing new in-memory state store"
May 28 18:46:56.297241 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.283111    2577 manager.go:341] "Starting Device Plugin manager"
May 28 18:46:56.297241 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.283152    2577 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 18:46:56.297241 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.283164    2577 server.go:85] "Starting device plugin registration server"
May 28 18:46:56.297241 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.283414    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 18:46:56.297241 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.283428    2577 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 18:46:56.297241 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.283562    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 18:46:56.297241 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.283669    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 18:46:56.297241 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.283681    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 18:46:56.297241 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.284155    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 18:46:56.297241 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.284189    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:56.352392 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.352347    2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 18:46:56.353567 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.353546    2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 18:46:56.353704 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.353577    2577 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 18:46:56.353704 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.353596    2577 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 18:46:56.353704 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.353617    2577 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 18:46:56.353926 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.353707    2577 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 28 18:46:56.357138 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.357089    2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 18:46:56.384226 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.384204    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 18:46:56.385091 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.385076    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
May 28 18:46:56.385180 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.385105    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
May 28 18:46:56.385180 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.385115    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
May 28 18:46:56.385180 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.385142    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-226.ec2.internal"
May 28 18:46:56.390706 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.390684    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-226.ec2.internal"
May 28 18:46:56.390820 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.390709    2577 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-226.ec2.internal\": node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:56.408083 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.408057    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:56.454160 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.454130    2577 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"]
May 28 18:46:56.454327 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.454201    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 18:46:56.455181 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.455165    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
May 28 18:46:56.455290 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.455194    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
May 28 18:46:56.455290 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.455208    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
May 28 18:46:56.456493 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.456480    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 18:46:56.456675 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.456662    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
May 28 18:46:56.456717 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.456690    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 18:46:56.457595 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.457579    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
May 28 18:46:56.457710 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.457579    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
May 28 18:46:56.457710 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.457624    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
May 28 18:46:56.457710 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.457639    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
May 28 18:46:56.457710 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.457653    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
May 28 18:46:56.457710 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.457671    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
May 28 18:46:56.458805 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.458790    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"
May 28 18:46:56.458895 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.458818    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 18:46:56.459836 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.459818    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
May 28 18:46:56.459925 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.459845    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
May 28 18:46:56.459925 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.459856    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
May 28 18:46:56.475188 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.475168    2577 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-226.ec2.internal\" not found" node="ip-10-0-137-226.ec2.internal"
May 28 18:46:56.479038 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.479021    2577 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-226.ec2.internal\" not found" node="ip-10-0-137-226.ec2.internal"
May 28 18:46:56.509127 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.509093    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:56.530089 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.530058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/96a6b651f0a2501b8229c49f3f7b2695-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal\" (UID: \"96a6b651f0a2501b8229c49f3f7b2695\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
May 28 18:46:56.530089 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.530090    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96a6b651f0a2501b8229c49f3f7b2695-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal\" (UID: \"96a6b651f0a2501b8229c49f3f7b2695\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
May 28 18:46:56.530254 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.530109    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ec0af40b48c17550ff6150b136d1afe4-config\") pod \"kube-apiserver-proxy-ip-10-0-137-226.ec2.internal\" (UID: \"ec0af40b48c17550ff6150b136d1afe4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"
May 28 18:46:56.610138 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.610070    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:56.630412 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.630384    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/96a6b651f0a2501b8229c49f3f7b2695-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal\" (UID: \"96a6b651f0a2501b8229c49f3f7b2695\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
May 28 18:46:56.630489 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.630418    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96a6b651f0a2501b8229c49f3f7b2695-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal\" (UID: \"96a6b651f0a2501b8229c49f3f7b2695\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
May 28 18:46:56.630489 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.630435    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ec0af40b48c17550ff6150b136d1afe4-config\") pod \"kube-apiserver-proxy-ip-10-0-137-226.ec2.internal\" (UID: \"ec0af40b48c17550ff6150b136d1afe4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"
May 28 18:46:56.630489 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.630486    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96a6b651f0a2501b8229c49f3f7b2695-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal\" (UID: \"96a6b651f0a2501b8229c49f3f7b2695\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
May 28 18:46:56.630628 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.630504    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/96a6b651f0a2501b8229c49f3f7b2695-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal\" (UID: \"96a6b651f0a2501b8229c49f3f7b2695\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
May 28 18:46:56.630628 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.630547    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ec0af40b48c17550ff6150b136d1afe4-config\") pod \"kube-apiserver-proxy-ip-10-0-137-226.ec2.internal\" (UID: \"ec0af40b48c17550ff6150b136d1afe4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"
May 28 18:46:56.710754 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.710714    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:56.778165 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.778128    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
May 28 18:46:56.782034 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:56.782007    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"
May 28 18:46:56.811697 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.811663    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:56.912209 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:56.912130    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:57.012699 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:57.012667    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:57.058439 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.058413    2577 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 18:46:57.113532 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:57.113496    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:57.128701 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.128674    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 18:46:57.128833 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.128814    2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 18:46:57.128876 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.128816    2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 18:46:57.214625 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:57.214407    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:57.227590 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.227563    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 18:46:57.233050 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.233018    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 18:41:56 +0000 UTC" deadline="2028-03-11 15:45:15.826228534 +0000 UTC"
May 28 18:46:57.233050 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.233044    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15668h58m18.593187576s"
May 28 18:46:57.249572 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.249550    2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 18:46:57.255137 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:57.255105    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec0af40b48c17550ff6150b136d1afe4.slice/crio-b21f6c9be5f7084c6c3c2756ffd74ce120ae773e3a598616132d6b75ed4f606a WatchSource:0}: Error finding container b21f6c9be5f7084c6c3c2756ffd74ce120ae773e3a598616132d6b75ed4f606a: Status 404 returned error can't find the container with id b21f6c9be5f7084c6c3c2756ffd74ce120ae773e3a598616132d6b75ed4f606a
May 28 18:46:57.255385 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:57.255357    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a6b651f0a2501b8229c49f3f7b2695.slice/crio-bb4fe635a2e7d5b65c465dd9d1354680beb4716e7c54128fefa3a876eb6431be WatchSource:0}: Error finding container bb4fe635a2e7d5b65c465dd9d1354680beb4716e7c54128fefa3a876eb6431be: Status 404 returned error can't find the container with id bb4fe635a2e7d5b65c465dd9d1354680beb4716e7c54128fefa3a876eb6431be
May 28 18:46:57.259397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.259379    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 18:46:57.274054 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.274031    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cpfgc"
May 28 18:46:57.281387 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.281359    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cpfgc"
May 28 18:46:57.315011 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:57.314976    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:57.357342 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.357297    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal" event={"ID":"ec0af40b48c17550ff6150b136d1afe4","Type":"ContainerStarted","Data":"b21f6c9be5f7084c6c3c2756ffd74ce120ae773e3a598616132d6b75ed4f606a"}
May 28 18:46:57.358267 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.358241    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal" event={"ID":"96a6b651f0a2501b8229c49f3f7b2695","Type":"ContainerStarted","Data":"bb4fe635a2e7d5b65c465dd9d1354680beb4716e7c54128fefa3a876eb6431be"}
May 28 18:46:57.415521 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:57.415489    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:57.430828 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.430799    2577 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 18:46:57.516245 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:57.516160    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:57.616730 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:57.616686    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
May 28 18:46:57.653874 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.653849    2577 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 18:46:57.727685 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.727653    2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"
May 28 18:46:57.737584 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.737553    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 18:46:57.738601 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.738559    2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
May 28 18:46:57.749096 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:57.749044    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 18:46:58.206233 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.206197    2577 apiserver.go:52] "Watching apiserver"
May 28 18:46:58.214033 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.214002    2577 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 18:46:58.214431 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.214409    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-qbwcr","openshift-multus/multus-additional-cni-plugins-cg96j","openshift-network-diagnostics/network-check-target-p24tw","kube-system/konnectivity-agent-pm7cd","kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal","openshift-cluster-node-tuning-operator/tuned-d7lqp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal","openshift-multus/multus-98z5n","openshift-multus/network-metrics-daemon-9xhpt","openshift-network-operator/iptables-alerter-9mmz8","openshift-ovn-kubernetes/ovnkube-node-9h2tl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq","openshift-dns/node-resolver-c4fhx"]
May 28 18:46:58.215939 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.215921    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9mmz8"
May 28 18:46:58.217048 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.217024    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.218450 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.218425    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:46:58.218548 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.218516    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:46:58.218717 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.218677    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 18:46:58.218812 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.218720    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 18:46:58.218971 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.218946    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jt5w4\""
May 28 18:46:58.219891 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.219237    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 18:46:58.219891 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.219698    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 18:46:58.219891 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.219833    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 18:46:58.220091 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.219941    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 18:46:58.220144 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.220110    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 18:46:58.220195 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.220151    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 18:46:58.221417 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.221377    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pm7cd"
May 28 18:46:58.221514 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.221492    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k2ck6\""
May 28 18:46:58.223324 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.223303    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.224490 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.224435    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-65b8j\""
May 28 18:46:58.224589 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.224539    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 18:46:58.224589 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.224441    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 18:46:58.224719 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.224590    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-98z5n"
May 28 18:46:58.225381 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.225361    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-llrp7\""
May 28 18:46:58.225473 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.225433    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 18:46:58.225861 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.225699    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 18:46:58.225928 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.225869    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:46:58.225928 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.225920    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:46:58.227339 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.227197    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9pw69\""
May 28 18:46:58.227339 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.227203    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qbwcr"
May 28 18:46:58.227339 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.227261    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 18:46:58.228678 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.228654    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.229499 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.229482    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kh4vn\""
May 28 18:46:58.229579 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.229486    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 18:46:58.229894 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.229746    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 18:46:58.229894 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.229822    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 18:46:58.230099 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.230084    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.230899 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.230865    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 18:46:58.231058 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.231039    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 18:46:58.231338 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.231318    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6c8tn\""
May 28 18:46:58.231424 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.231358    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c4fhx"
May 28 18:46:58.232238 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.232210    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 18:46:58.232332 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.232238    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 18:46:58.232387 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.232354    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 18:46:58.232459 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.232442    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 18:46:58.232704 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.232675    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2pj82\""
May 28 18:46:58.232787 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.232688    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 18:46:58.233067 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.233049    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 18:46:58.233243 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.233223    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 18:46:58.233415 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.233377    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xs5fm\""
May 28 18:46:58.233415 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.233380    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 18:46:58.233573 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.233559    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 18:46:58.239015 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.238996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-cnibin\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.239113 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-run-multus-certs\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.239113 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239046    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.239113 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfwgr\" (UniqueName: \"kubernetes.io/projected/b56331e8-f658-4d66-9369-6c0828f65d6d-kube-api-access-mfwgr\") pod \"iptables-alerter-9mmz8\" (UID: \"b56331e8-f658-4d66-9369-6c0828f65d6d\") " pod="openshift-network-operator/iptables-alerter-9mmz8"
May 28 18:46:58.239264 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/01a5120d-33a6-42dd-b48c-7490c470f11f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.239264 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-system-cni-dir\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.239264 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-systemd-units\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.239264 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239183    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-systemd\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.239264 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgvm4\" (UniqueName: \"kubernetes.io/projected/96afef30-52b0-4e8e-b918-4ff7f35dc403-kube-api-access-rgvm4\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.239264 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdcph\" (UniqueName: \"kubernetes.io/projected/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-kube-api-access-gdcph\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.239264 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239246    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-slash\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.239550 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239290    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-env-overrides\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.239550 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239329    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-ovnkube-script-lib\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.239550 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239381    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b1230db-d35f-489e-a592-e9d91bbb6d74-host\") pod \"node-ca-qbwcr\" (UID: \"4b1230db-d35f-489e-a592-e9d91bbb6d74\") " pod="openshift-image-registry/node-ca-qbwcr"
May 28 18:46:58.239550 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239422    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-device-dir\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.239550 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239467    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/01a5120d-33a6-42dd-b48c-7490c470f11f-os-release\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.239550 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239512    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01a5120d-33a6-42dd-b48c-7490c470f11f-cni-binary-copy\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.239550 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239536    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzm44\" (UniqueName: \"kubernetes.io/projected/01a5120d-33a6-42dd-b48c-7490c470f11f-kube-api-access-wzm44\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.239843 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239558    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-multus-socket-dir-parent\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.239843 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-run-netns\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.239843 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-ovnkube-config\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.239843 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239674    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01a5120d-33a6-42dd-b48c-7490c470f11f-cnibin\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.239843 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239697    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-etc-openvswitch\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.239843 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239740    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-run-ovn\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.239843 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.239843 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239792    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b56331e8-f658-4d66-9369-6c0828f65d6d-host-slash\") pod \"iptables-alerter-9mmz8\" (UID: \"b56331e8-f658-4d66-9369-6c0828f65d6d\") " pod="openshift-network-operator/iptables-alerter-9mmz8"
May 28 18:46:58.239843 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239813    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/01a5120d-33a6-42dd-b48c-7490c470f11f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.239843 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-modprobe-d\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239854    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-782sj\" (UniqueName: \"kubernetes.io/projected/4b1230db-d35f-489e-a592-e9d91bbb6d74-kube-api-access-782sj\") pod \"node-ca-qbwcr\" (UID: \"4b1230db-d35f-489e-a592-e9d91bbb6d74\") " pod="openshift-image-registry/node-ca-qbwcr"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2bkc\" (UniqueName: \"kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc\") pod \"network-check-target-p24tw\" (UID: \"f70415e6-52d5-4096-9d21-82b7ca994f61\") " pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/532d9740-8a10-452a-88d7-64ecd63e0f66-agent-certs\") pod \"konnectivity-agent-pm7cd\" (UID: \"532d9740-8a10-452a-88d7-64ecd63e0f66\") " pod="kube-system/konnectivity-agent-pm7cd"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239958    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-var-lib-kubelet\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.239986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-kubelet\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240011    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-run-netns\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240050    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-ovn-node-metrics-cert\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240090    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-sysctl-conf\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01a5120d-33a6-42dd-b48c-7490c470f11f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-os-release\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240170    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-var-lib-cni-multus\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-multus-conf-dir\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240216    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-sys\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-var-lib-kubelet\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240261    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1f18d702-cac3-41b8-b691-872943d4862a-etc-tuned\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.240293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240298    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-var-lib-openvswitch\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-node-log\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240347    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-cni-bin\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240382    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-host\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-cni-netd\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6nfs\" (UniqueName: \"kubernetes.io/projected/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-kube-api-access-r6nfs\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240468    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-socket-dir\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-kubernetes\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240536    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf5k2\" (UniqueName: \"kubernetes.io/projected/1625d5d8-1437-4fb6-b18e-320acb890173-kube-api-access-wf5k2\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240576    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-etc-selinux\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240589    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-hostroot\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240626    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b56331e8-f658-4d66-9369-6c0828f65d6d-iptables-alerter-script\") pod \"iptables-alerter-9mmz8\" (UID: \"b56331e8-f658-4d66-9369-6c0828f65d6d\") " pod="openshift-network-operator/iptables-alerter-9mmz8"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-lib-modules\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240683    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-multus-cni-dir\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240725    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b1230db-d35f-489e-a592-e9d91bbb6d74-serviceca\") pod \"node-ca-qbwcr\" (UID: \"4b1230db-d35f-489e-a592-e9d91bbb6d74\") " pod="openshift-image-registry/node-ca-qbwcr"
May 28 18:46:58.240995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01a5120d-33a6-42dd-b48c-7490c470f11f-system-cni-dir\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-run-k8s-cni-cncf-io\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240863    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-var-lib-cni-bin\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-multus-daemon-config\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240917    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240941    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-cni-binary-copy\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240963    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-run-systemd\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.240998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-log-socket\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.241030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-sysctl-d\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.241057    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9htt\" (UniqueName: \"kubernetes.io/projected/1f18d702-cac3-41b8-b691-872943d4862a-kube-api-access-p9htt\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.241094    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/532d9740-8a10-452a-88d7-64ecd63e0f66-konnectivity-ca\") pod \"konnectivity-agent-pm7cd\" (UID: \"532d9740-8a10-452a-88d7-64ecd63e0f66\") " pod="kube-system/konnectivity-agent-pm7cd"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.241112    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-run\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.241126    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f18d702-cac3-41b8-b691-872943d4862a-tmp\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.241169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-registration-dir\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.241189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-etc-kubernetes\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.241210    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-run-openvswitch\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.241233    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-sysconfig\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.241650 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.241277    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-sys-fs\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.282205 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.282178    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 18:41:57 +0000 UTC" deadline="2028-03-01 17:41:29.912385502 +0000 UTC"
May 28 18:46:58.282205 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.282202    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15430h54m31.630185416s"
May 28 18:46:58.328398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.328372    2577 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 18:46:58.342503 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342472    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-device-dir\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.342692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342513    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/01a5120d-33a6-42dd-b48c-7490c470f11f-os-release\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.342692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342539    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01a5120d-33a6-42dd-b48c-7490c470f11f-cni-binary-copy\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.342692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342586    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wzm44\" (UniqueName: \"kubernetes.io/projected/01a5120d-33a6-42dd-b48c-7490c470f11f-kube-api-access-wzm44\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.342692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342604    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-device-dir\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.342692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342641    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/01a5120d-33a6-42dd-b48c-7490c470f11f-os-release\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.342692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342628    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-multus-socket-dir-parent\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.342692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342678    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-multus-socket-dir-parent\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.342692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342687    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-run-netns\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342714    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-ovnkube-config\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342741    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01a5120d-33a6-42dd-b48c-7490c470f11f-cnibin\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342764    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-etc-openvswitch\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342770    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-run-netns\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342788    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-run-ovn\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342818    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-etc-openvswitch\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342831    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01a5120d-33a6-42dd-b48c-7490c470f11f-cnibin\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342835    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-run-ovn\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342854    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342881    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b56331e8-f658-4d66-9369-6c0828f65d6d-host-slash\") pod \"iptables-alerter-9mmz8\" (UID: \"b56331e8-f658-4d66-9369-6c0828f65d6d\") " pod="openshift-network-operator/iptables-alerter-9mmz8"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342906    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a83d99d8-7c19-4e57-aca6-26f8a033d681-tmp-dir\") pod \"node-resolver-c4fhx\" (UID: \"a83d99d8-7c19-4e57-aca6-26f8a033d681\") " pod="openshift-dns/node-resolver-c4fhx"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342934    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342948    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/01a5120d-33a6-42dd-b48c-7490c470f11f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342977    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-modprobe-d\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.342992    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b56331e8-f658-4d66-9369-6c0828f65d6d-host-slash\") pod \"iptables-alerter-9mmz8\" (UID: \"b56331e8-f658-4d66-9369-6c0828f65d6d\") " pod="openshift-network-operator/iptables-alerter-9mmz8"
May 28 18:46:58.343103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343055    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a83d99d8-7c19-4e57-aca6-26f8a033d681-hosts-file\") pod \"node-resolver-c4fhx\" (UID: \"a83d99d8-7c19-4e57-aca6-26f8a033d681\") " pod="openshift-dns/node-resolver-c4fhx"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343142    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-modprobe-d\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343187    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-782sj\" (UniqueName: \"kubernetes.io/projected/4b1230db-d35f-489e-a592-e9d91bbb6d74-kube-api-access-782sj\") pod \"node-ca-qbwcr\" (UID: \"4b1230db-d35f-489e-a592-e9d91bbb6d74\") " pod="openshift-image-registry/node-ca-qbwcr"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343292    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bkc\" (UniqueName: \"kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc\") pod \"network-check-target-p24tw\" (UID: \"f70415e6-52d5-4096-9d21-82b7ca994f61\") " pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343321    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/532d9740-8a10-452a-88d7-64ecd63e0f66-agent-certs\") pod \"konnectivity-agent-pm7cd\" (UID: \"532d9740-8a10-452a-88d7-64ecd63e0f66\") " pod="kube-system/konnectivity-agent-pm7cd"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343349    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-var-lib-kubelet\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343369    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-ovnkube-config\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343378    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-kubelet\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343405    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-run-netns\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343431    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-ovn-node-metrics-cert\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343457    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-sysctl-conf\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343453    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-kubelet\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343484    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01a5120d-33a6-42dd-b48c-7490c470f11f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343503    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-var-lib-kubelet\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343507    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/01a5120d-33a6-42dd-b48c-7490c470f11f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343511    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-os-release\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343536    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-var-lib-cni-multus\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.343918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343562    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-multus-conf-dir\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343587    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-sys\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343629    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-var-lib-kubelet\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343653    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1f18d702-cac3-41b8-b691-872943d4862a-etc-tuned\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343678    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-var-lib-openvswitch\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343682    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-os-release\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343703    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-node-log\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343721    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-run-netns\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343729    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-cni-bin\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343756    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-host\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343745    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343777    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-cni-netd\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343785    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-var-lib-kubelet\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343801    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r6nfs\" (UniqueName: \"kubernetes.io/projected/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-kube-api-access-r6nfs\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343825    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-socket-dir\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343826    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-sysctl-conf\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343835    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-multus-conf-dir\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343840    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-kubernetes\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.344750 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343864    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-cni-bin\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343876    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343900    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-host\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343903    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wf5k2\" (UniqueName: \"kubernetes.io/projected/1625d5d8-1437-4fb6-b18e-320acb890173-kube-api-access-wf5k2\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343922    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-cni-netd\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343928    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-etc-selinux\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343802    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-var-lib-cni-multus\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343964    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-hostroot\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343970    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01a5120d-33a6-42dd-b48c-7490c470f11f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343989    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b56331e8-f658-4d66-9369-6c0828f65d6d-iptables-alerter-script\") pod \"iptables-alerter-9mmz8\" (UID: \"b56331e8-f658-4d66-9369-6c0828f65d6d\") " pod="openshift-network-operator/iptables-alerter-9mmz8"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344016    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-lib-modules\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344021    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-kubernetes\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344033    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-socket-dir\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.344046    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjn8d\" (UniqueName: \"kubernetes.io/projected/a83d99d8-7c19-4e57-aca6-26f8a033d681-kube-api-access-mjn8d\") pod \"node-resolver-c4fhx\" (UID: \"a83d99d8-7c19-4e57-aca6-26f8a033d681\") " pod="openshift-dns/node-resolver-c4fhx"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344071    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-etc-selinux\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344088    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-multus-cni-dir\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344101    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-var-lib-openvswitch\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.345723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344140    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-multus-cni-dir\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.344158    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs podName:1625d5d8-1437-4fb6-b18e-320acb890173 nodeName:}" failed. No retries permitted until 2026-05-28 18:46:58.844099672 +0000 UTC m=+3.103321837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs") pod "network-metrics-daemon-9xhpt" (UID: "1625d5d8-1437-4fb6-b18e-320acb890173") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344160    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-hostroot\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344172    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-lib-modules\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344198    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-node-log\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.343768    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-sys\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344251    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b1230db-d35f-489e-a592-e9d91bbb6d74-serviceca\") pod \"node-ca-qbwcr\" (UID: \"4b1230db-d35f-489e-a592-e9d91bbb6d74\") " pod="openshift-image-registry/node-ca-qbwcr"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344277    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01a5120d-33a6-42dd-b48c-7490c470f11f-system-cni-dir\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344301    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-run-k8s-cni-cncf-io\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344322    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-var-lib-cni-bin\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344325    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01a5120d-33a6-42dd-b48c-7490c470f11f-system-cni-dir\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344364    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-run-k8s-cni-cncf-io\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344365    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-var-lib-cni-bin\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344386    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-multus-daemon-config\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344420    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344446    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-cni-binary-copy\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344470    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-run-systemd\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.346718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344494    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-log-socket\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344519    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-sysctl-d\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344546    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p9htt\" (UniqueName: \"kubernetes.io/projected/1f18d702-cac3-41b8-b691-872943d4862a-kube-api-access-p9htt\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344571    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/532d9740-8a10-452a-88d7-64ecd63e0f66-konnectivity-ca\") pod \"konnectivity-agent-pm7cd\" (UID: \"532d9740-8a10-452a-88d7-64ecd63e0f66\") " pod="kube-system/konnectivity-agent-pm7cd"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344595    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-run\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344639    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f18d702-cac3-41b8-b691-872943d4862a-tmp\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344663    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-registration-dir\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344675    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b1230db-d35f-489e-a592-e9d91bbb6d74-serviceca\") pod \"node-ca-qbwcr\" (UID: \"4b1230db-d35f-489e-a592-e9d91bbb6d74\") " pod="openshift-image-registry/node-ca-qbwcr"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344690    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b56331e8-f658-4d66-9369-6c0828f65d6d-iptables-alerter-script\") pod \"iptables-alerter-9mmz8\" (UID: \"b56331e8-f658-4d66-9369-6c0828f65d6d\") " pod="openshift-network-operator/iptables-alerter-9mmz8"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344717    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-etc-kubernetes\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344742    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-run-openvswitch\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344764    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-sysconfig\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344789    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-sys-fs\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344815    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-cnibin\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344821    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-run-systemd\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344840    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-run-multus-certs\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344859    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-log-socket\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.347561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344872    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344899    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mfwgr\" (UniqueName: \"kubernetes.io/projected/b56331e8-f658-4d66-9369-6c0828f65d6d-kube-api-access-mfwgr\") pod \"iptables-alerter-9mmz8\" (UID: \"b56331e8-f658-4d66-9369-6c0828f65d6d\") " pod="openshift-network-operator/iptables-alerter-9mmz8"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344926    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/01a5120d-33a6-42dd-b48c-7490c470f11f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344918    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-run\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344959    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-system-cni-dir\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344977    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-sysconfig\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344984    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-systemd-units\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344993    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-host-run-multus-certs\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.344790    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-sysctl-d\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345021    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-systemd\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345027    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-systemd-units\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345051    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-cnibin\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345051    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rgvm4\" (UniqueName: \"kubernetes.io/projected/96afef30-52b0-4e8e-b918-4ff7f35dc403-kube-api-access-rgvm4\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345071    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345091    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gdcph\" (UniqueName: \"kubernetes.io/projected/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-kube-api-access-gdcph\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345108    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01a5120d-33a6-42dd-b48c-7490c470f11f-cni-binary-copy\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345118    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-slash\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.348271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345153    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-env-overrides\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345158    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-run-openvswitch\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345187    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-ovnkube-script-lib\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345214    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b1230db-d35f-489e-a592-e9d91bbb6d74-host\") pod \"node-ca-qbwcr\" (UID: \"4b1230db-d35f-489e-a592-e9d91bbb6d74\") " pod="openshift-image-registry/node-ca-qbwcr"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345268    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-multus-daemon-config\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345279    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b1230db-d35f-489e-a592-e9d91bbb6d74-host\") pod \"node-ca-qbwcr\" (UID: \"4b1230db-d35f-489e-a592-e9d91bbb6d74\") " pod="openshift-image-registry/node-ca-qbwcr"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345335    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345346    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/532d9740-8a10-452a-88d7-64ecd63e0f66-konnectivity-ca\") pod \"konnectivity-agent-pm7cd\" (UID: \"532d9740-8a10-452a-88d7-64ecd63e0f66\") " pod="kube-system/konnectivity-agent-pm7cd"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345379    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-host-slash\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345413    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-sys-fs\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345503    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96afef30-52b0-4e8e-b918-4ff7f35dc403-registration-dir\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345545    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-etc-kubernetes\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345682    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/01a5120d-33a6-42dd-b48c-7490c470f11f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345698    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1f18d702-cac3-41b8-b691-872943d4862a-etc-systemd\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345687    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-system-cni-dir\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345850    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-cni-binary-copy\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.345921    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-env-overrides\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.346206    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-ovnkube-script-lib\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.348942 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.348406    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f18d702-cac3-41b8-b691-872943d4862a-tmp\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.349789 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.348455    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1f18d702-cac3-41b8-b691-872943d4862a-etc-tuned\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.349789 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.348482    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-ovn-node-metrics-cert\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.349789 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.348630    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/532d9740-8a10-452a-88d7-64ecd63e0f66-agent-certs\") pod \"konnectivity-agent-pm7cd\" (UID: \"532d9740-8a10-452a-88d7-64ecd63e0f66\") " pod="kube-system/konnectivity-agent-pm7cd"
May 28 18:46:58.349789 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.348705    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 18:46:58.349789 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.348719    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 18:46:58.349789 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.348730    2577 projected.go:194] Error preparing data for projected volume kube-api-access-b2bkc for pod openshift-network-diagnostics/network-check-target-p24tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:46:58.349789 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.348785    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc podName:f70415e6-52d5-4096-9d21-82b7ca994f61 nodeName:}" failed. No retries permitted until 2026-05-28 18:46:58.848772947 +0000 UTC m=+3.107995091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bkc" (UniqueName: "kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc") pod "network-check-target-p24tw" (UID: "f70415e6-52d5-4096-9d21-82b7ca994f61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:46:58.350713 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.350683    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-782sj\" (UniqueName: \"kubernetes.io/projected/4b1230db-d35f-489e-a592-e9d91bbb6d74-kube-api-access-782sj\") pod \"node-ca-qbwcr\" (UID: \"4b1230db-d35f-489e-a592-e9d91bbb6d74\") " pod="openshift-image-registry/node-ca-qbwcr"
May 28 18:46:58.350980 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.350964    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzm44\" (UniqueName: \"kubernetes.io/projected/01a5120d-33a6-42dd-b48c-7490c470f11f-kube-api-access-wzm44\") pod \"multus-additional-cni-plugins-cg96j\" (UID: \"01a5120d-33a6-42dd-b48c-7490c470f11f\") " pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.354379 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.354353    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6nfs\" (UniqueName: \"kubernetes.io/projected/1573bbfa-cdc9-48b4-a667-ab2be639d3a8-kube-api-access-r6nfs\") pod \"ovnkube-node-9h2tl\" (UID: \"1573bbfa-cdc9-48b4-a667-ab2be639d3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.354737 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.354690    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf5k2\" (UniqueName: \"kubernetes.io/projected/1625d5d8-1437-4fb6-b18e-320acb890173-kube-api-access-wf5k2\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:46:58.355041 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.355024    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdcph\" (UniqueName: \"kubernetes.io/projected/90a5b717-a5bb-4b1e-b516-b3dec5e1601e-kube-api-access-gdcph\") pod \"multus-98z5n\" (UID: \"90a5b717-a5bb-4b1e-b516-b3dec5e1601e\") " pod="openshift-multus/multus-98z5n"
May 28 18:46:58.355210 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.355194    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfwgr\" (UniqueName: \"kubernetes.io/projected/b56331e8-f658-4d66-9369-6c0828f65d6d-kube-api-access-mfwgr\") pod \"iptables-alerter-9mmz8\" (UID: \"b56331e8-f658-4d66-9369-6c0828f65d6d\") " pod="openshift-network-operator/iptables-alerter-9mmz8"
May 28 18:46:58.355582 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.355564    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgvm4\" (UniqueName: \"kubernetes.io/projected/96afef30-52b0-4e8e-b918-4ff7f35dc403-kube-api-access-rgvm4\") pod \"aws-ebs-csi-driver-node-pvbkq\" (UID: \"96afef30-52b0-4e8e-b918-4ff7f35dc403\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.356291 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.356270    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9htt\" (UniqueName: \"kubernetes.io/projected/1f18d702-cac3-41b8-b691-872943d4862a-kube-api-access-p9htt\") pod \"tuned-d7lqp\" (UID: \"1f18d702-cac3-41b8-b691-872943d4862a\") " pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.443268 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.443235    2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 18:46:58.445513 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.445480    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a83d99d8-7c19-4e57-aca6-26f8a033d681-tmp-dir\") pod \"node-resolver-c4fhx\" (UID: \"a83d99d8-7c19-4e57-aca6-26f8a033d681\") " pod="openshift-dns/node-resolver-c4fhx"
May 28 18:46:58.445665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.445524    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a83d99d8-7c19-4e57-aca6-26f8a033d681-hosts-file\") pod \"node-resolver-c4fhx\" (UID: \"a83d99d8-7c19-4e57-aca6-26f8a033d681\") " pod="openshift-dns/node-resolver-c4fhx"
May 28 18:46:58.445665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.445582    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mjn8d\" (UniqueName: \"kubernetes.io/projected/a83d99d8-7c19-4e57-aca6-26f8a033d681-kube-api-access-mjn8d\") pod \"node-resolver-c4fhx\" (UID: \"a83d99d8-7c19-4e57-aca6-26f8a033d681\") " pod="openshift-dns/node-resolver-c4fhx"
May 28 18:46:58.446094 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.446068    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a83d99d8-7c19-4e57-aca6-26f8a033d681-hosts-file\") pod \"node-resolver-c4fhx\" (UID: \"a83d99d8-7c19-4e57-aca6-26f8a033d681\") " pod="openshift-dns/node-resolver-c4fhx"
May 28 18:46:58.446214 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.446105    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a83d99d8-7c19-4e57-aca6-26f8a033d681-tmp-dir\") pod \"node-resolver-c4fhx\" (UID: \"a83d99d8-7c19-4e57-aca6-26f8a033d681\") " pod="openshift-dns/node-resolver-c4fhx"
May 28 18:46:58.453572 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.453546    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjn8d\" (UniqueName: \"kubernetes.io/projected/a83d99d8-7c19-4e57-aca6-26f8a033d681-kube-api-access-mjn8d\") pod \"node-resolver-c4fhx\" (UID: \"a83d99d8-7c19-4e57-aca6-26f8a033d681\") " pod="openshift-dns/node-resolver-c4fhx"
May 28 18:46:58.530170 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.530067    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9mmz8"
May 28 18:46:58.537832 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.537801    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cg96j"
May 28 18:46:58.546554 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.546525    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pm7cd"
May 28 18:46:58.553297 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.553278    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d7lqp"
May 28 18:46:58.559924 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.559903    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-98z5n"
May 28 18:46:58.566499 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.566480    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qbwcr"
May 28 18:46:58.573054 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.573024    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:46:58.580594 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.580577    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq"
May 28 18:46:58.586161 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.586144    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c4fhx"
May 28 18:46:58.768337 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.768304    2577 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 18:46:58.848847 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.848763    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:46:58.848847 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:58.848834    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bkc\" (UniqueName: \"kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc\") pod \"network-check-target-p24tw\" (UID: \"f70415e6-52d5-4096-9d21-82b7ca994f61\") " pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:46:58.849080 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.848908    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:46:58.849080 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.848954    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 18:46:58.849080 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.848971    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 18:46:58.849080 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.848976    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs podName:1625d5d8-1437-4fb6-b18e-320acb890173 nodeName:}" failed. No retries permitted until 2026-05-28 18:46:59.848960837 +0000 UTC m=+4.108182986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs") pod "network-metrics-daemon-9xhpt" (UID: "1625d5d8-1437-4fb6-b18e-320acb890173") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:46:58.849080 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.848983    2577 projected.go:194] Error preparing data for projected volume kube-api-access-b2bkc for pod openshift-network-diagnostics/network-check-target-p24tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:46:58.849080 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:58.849029    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc podName:f70415e6-52d5-4096-9d21-82b7ca994f61 nodeName:}" failed. No retries permitted until 2026-05-28 18:46:59.849014949 +0000 UTC m=+4.108237098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bkc" (UniqueName: "kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc") pod "network-check-target-p24tw" (UID: "f70415e6-52d5-4096-9d21-82b7ca994f61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:46:58.867717 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:58.867671    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b1230db_d35f_489e_a592_e9d91bbb6d74.slice/crio-242427aed4fbbaf67a52bff528ff2a736d9e930e1afdeaa74472b441fd19dcc2 WatchSource:0}: Error finding container 242427aed4fbbaf67a52bff528ff2a736d9e930e1afdeaa74472b441fd19dcc2: Status 404 returned error can't find the container with id 242427aed4fbbaf67a52bff528ff2a736d9e930e1afdeaa74472b441fd19dcc2
May 28 18:46:58.868894 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:58.868868    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96afef30_52b0_4e8e_b918_4ff7f35dc403.slice/crio-74cbb3533a0b33177647653d3a1418ad1ae482750e0c6ad94aa2dc27862f9c03 WatchSource:0}: Error finding container 74cbb3533a0b33177647653d3a1418ad1ae482750e0c6ad94aa2dc27862f9c03: Status 404 returned error can't find the container with id 74cbb3533a0b33177647653d3a1418ad1ae482750e0c6ad94aa2dc27862f9c03
May 28 18:46:58.874917 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:58.874894    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb56331e8_f658_4d66_9369_6c0828f65d6d.slice/crio-70f52fd6f4794143ccea4dac8a456e8765b2996d4165f532e3d0508eb5320641 WatchSource:0}: Error finding container 70f52fd6f4794143ccea4dac8a456e8765b2996d4165f532e3d0508eb5320641: Status 404 returned error can't find the container with id 70f52fd6f4794143ccea4dac8a456e8765b2996d4165f532e3d0508eb5320641
May 28 18:46:58.875811 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:58.875781    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod532d9740_8a10_452a_88d7_64ecd63e0f66.slice/crio-efe5ec008f8af18646843e9fd5c62e2125696142298fac41bcb965726e4b335b WatchSource:0}: Error finding container efe5ec008f8af18646843e9fd5c62e2125696142298fac41bcb965726e4b335b: Status 404 returned error can't find the container with id efe5ec008f8af18646843e9fd5c62e2125696142298fac41bcb965726e4b335b
May 28 18:46:58.877656 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:58.877633    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1573bbfa_cdc9_48b4_a667_ab2be639d3a8.slice/crio-2b4d687082d7bb000db3c8b5c7427e611f7f8665b4b3f84a30710fe40218709c WatchSource:0}: Error finding container 2b4d687082d7bb000db3c8b5c7427e611f7f8665b4b3f84a30710fe40218709c: Status 404 returned error can't find the container with id 2b4d687082d7bb000db3c8b5c7427e611f7f8665b4b3f84a30710fe40218709c
May 28 18:46:58.878722 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:58.878674    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90a5b717_a5bb_4b1e_b516_b3dec5e1601e.slice/crio-a8ca0942108b154c6393ce640bf7bf95d66d9bf01aa310dbe5e98c1a09e11a4a WatchSource:0}: Error finding container a8ca0942108b154c6393ce640bf7bf95d66d9bf01aa310dbe5e98c1a09e11a4a: Status 404 returned error can't find the container with id a8ca0942108b154c6393ce640bf7bf95d66d9bf01aa310dbe5e98c1a09e11a4a
May 28 18:46:58.879581 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:58.879553    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a5120d_33a6_42dd_b48c_7490c470f11f.slice/crio-41de38f4adf4257f11a808b6cd4be8907725947d56a8b1a963967238848ab450 WatchSource:0}: Error finding container 41de38f4adf4257f11a808b6cd4be8907725947d56a8b1a963967238848ab450: Status 404 returned error can't find the container with id 41de38f4adf4257f11a808b6cd4be8907725947d56a8b1a963967238848ab450
May 28 18:46:58.880169 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:46:58.880133    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda83d99d8_7c19_4e57_aca6_26f8a033d681.slice/crio-a09e4e448c12b0d42c6b18ecdca8ea03291f1920bb38c5fe464c651154fcb218 WatchSource:0}: Error finding container a09e4e448c12b0d42c6b18ecdca8ea03291f1920bb38c5fe464c651154fcb218: Status 404 returned error can't find the container with id a09e4e448c12b0d42c6b18ecdca8ea03291f1920bb38c5fe464c651154fcb218
May 28 18:46:59.282686 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.282640    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 18:41:57 +0000 UTC" deadline="2028-02-08 21:49:44.235303832 +0000 UTC"
May 28 18:46:59.282686 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.282683    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14907h2m44.952624376s"
May 28 18:46:59.367219 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.367098    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c4fhx" event={"ID":"a83d99d8-7c19-4e57-aca6-26f8a033d681","Type":"ContainerStarted","Data":"a09e4e448c12b0d42c6b18ecdca8ea03291f1920bb38c5fe464c651154fcb218"}
May 28 18:46:59.368812 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.368750    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-98z5n" event={"ID":"90a5b717-a5bb-4b1e-b516-b3dec5e1601e","Type":"ContainerStarted","Data":"a8ca0942108b154c6393ce640bf7bf95d66d9bf01aa310dbe5e98c1a09e11a4a"}
May 28 18:46:59.370070 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.370017    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cg96j" event={"ID":"01a5120d-33a6-42dd-b48c-7490c470f11f","Type":"ContainerStarted","Data":"41de38f4adf4257f11a808b6cd4be8907725947d56a8b1a963967238848ab450"}
May 28 18:46:59.373885 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.373839    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl" event={"ID":"1573bbfa-cdc9-48b4-a667-ab2be639d3a8","Type":"ContainerStarted","Data":"2b4d687082d7bb000db3c8b5c7427e611f7f8665b4b3f84a30710fe40218709c"}
May 28 18:46:59.378552 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.378497    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pm7cd" event={"ID":"532d9740-8a10-452a-88d7-64ecd63e0f66","Type":"ContainerStarted","Data":"efe5ec008f8af18646843e9fd5c62e2125696142298fac41bcb965726e4b335b"}
May 28 18:46:59.381438 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.381393    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq" event={"ID":"96afef30-52b0-4e8e-b918-4ff7f35dc403","Type":"ContainerStarted","Data":"74cbb3533a0b33177647653d3a1418ad1ae482750e0c6ad94aa2dc27862f9c03"}
May 28 18:46:59.382547 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.382528    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9mmz8" event={"ID":"b56331e8-f658-4d66-9369-6c0828f65d6d","Type":"ContainerStarted","Data":"70f52fd6f4794143ccea4dac8a456e8765b2996d4165f532e3d0508eb5320641"}
May 28 18:46:59.385893 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.385868    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d7lqp" event={"ID":"1f18d702-cac3-41b8-b691-872943d4862a","Type":"ContainerStarted","Data":"36f1bd64a1d6c7588faee820d087f77a83cdb43eecddb71520f140bf15970e12"}
May 28 18:46:59.388748 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.388709    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qbwcr" event={"ID":"4b1230db-d35f-489e-a592-e9d91bbb6d74","Type":"ContainerStarted","Data":"242427aed4fbbaf67a52bff528ff2a736d9e930e1afdeaa74472b441fd19dcc2"}
May 28 18:46:59.393234 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.393185    2577 generic.go:358] "Generic (PLEG): container finished" podID="96a6b651f0a2501b8229c49f3f7b2695" containerID="23aca948ec6e2ac44cab1842149c9e804f37469a7d0824b83e7acdfbb8d8e4c2" exitCode=0
May 28 18:46:59.393329 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.393251    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal" event={"ID":"96a6b651f0a2501b8229c49f3f7b2695","Type":"ContainerDied","Data":"23aca948ec6e2ac44cab1842149c9e804f37469a7d0824b83e7acdfbb8d8e4c2"}
May 28 18:46:59.403020 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.402510    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal" event={"ID":"ec0af40b48c17550ff6150b136d1afe4","Type":"ContainerStarted","Data":"84772aeb3280b01c2affb5c2b053f431a99f67a89e48d1a03fb0be8dac2b4f5f"}
May 28 18:46:59.856929 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.856122    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bkc\" (UniqueName: \"kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc\") pod \"network-check-target-p24tw\" (UID: \"f70415e6-52d5-4096-9d21-82b7ca994f61\") " pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:46:59.856929 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:46:59.856189    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:46:59.856929 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:59.856320    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:46:59.856929 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:59.856385    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs podName:1625d5d8-1437-4fb6-b18e-320acb890173 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:01.856365458 +0000 UTC m=+6.115587621 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs") pod "network-metrics-daemon-9xhpt" (UID: "1625d5d8-1437-4fb6-b18e-320acb890173") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:46:59.856929 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:59.856812    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 18:46:59.856929 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:59.856833    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 18:46:59.856929 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:59.856846    2577 projected.go:194] Error preparing data for projected volume kube-api-access-b2bkc for pod openshift-network-diagnostics/network-check-target-p24tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:46:59.856929 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:46:59.856892    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc podName:f70415e6-52d5-4096-9d21-82b7ca994f61 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:01.856877448 +0000 UTC m=+6.116099607 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bkc" (UniqueName: "kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc") pod "network-check-target-p24tw" (UID: "f70415e6-52d5-4096-9d21-82b7ca994f61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:47:00.354871 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:00.354801    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:00.355331 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:00.354976    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:00.355882 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:00.355471    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:00.355882 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:00.355563    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:00.425406 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:00.425160    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal" event={"ID":"96a6b651f0a2501b8229c49f3f7b2695","Type":"ContainerStarted","Data":"cf2061f659084c2c09b40eb7720e6e9402ee6634a8fe9b2bf4a6835a39110a62"}
May 28 18:47:00.440135 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:00.440074    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal" podStartSLOduration=3.440056303 podStartE2EDuration="3.440056303s" podCreationTimestamp="2026-05-28 18:46:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 18:46:59.421826062 +0000 UTC m=+3.681048231" watchObservedRunningTime="2026-05-28 18:47:00.440056303 +0000 UTC m=+4.699278481"
May 28 18:47:01.872237 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:01.872199    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bkc\" (UniqueName: \"kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc\") pod \"network-check-target-p24tw\" (UID: \"f70415e6-52d5-4096-9d21-82b7ca994f61\") " pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:01.872718 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:01.872267    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:01.872718 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:01.872392    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:47:01.872718 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:01.872453    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs podName:1625d5d8-1437-4fb6-b18e-320acb890173 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:05.872433415 +0000 UTC m=+10.131655564 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs") pod "network-metrics-daemon-9xhpt" (UID: "1625d5d8-1437-4fb6-b18e-320acb890173") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:47:01.872969 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:01.872897    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 18:47:01.872969 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:01.872920    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 18:47:01.872969 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:01.872934    2577 projected.go:194] Error preparing data for projected volume kube-api-access-b2bkc for pod openshift-network-diagnostics/network-check-target-p24tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:47:01.873119 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:01.872993    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc podName:f70415e6-52d5-4096-9d21-82b7ca994f61 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:05.872978118 +0000 UTC m=+10.132200275 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bkc" (UniqueName: "kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc") pod "network-check-target-p24tw" (UID: "f70415e6-52d5-4096-9d21-82b7ca994f61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:47:02.354698 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:02.354654    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:02.354698 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:02.354695    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:02.354926 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:02.354880    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:02.355138 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:02.355114    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:04.354339 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:04.354186    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:04.354339 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:04.354221    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:04.354843 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:04.354338    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:04.354843 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:04.354790    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:05.909769 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:05.909720    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bkc\" (UniqueName: \"kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc\") pod \"network-check-target-p24tw\" (UID: \"f70415e6-52d5-4096-9d21-82b7ca994f61\") " pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:05.910242 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:05.909786    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:05.910242 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:05.909913    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:47:05.910242 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:05.909938    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 18:47:05.910242 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:05.909962    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 18:47:05.910242 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:05.909975    2577 projected.go:194] Error preparing data for projected volume kube-api-access-b2bkc for pod openshift-network-diagnostics/network-check-target-p24tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:47:05.910242 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:05.909992    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs podName:1625d5d8-1437-4fb6-b18e-320acb890173 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:13.909971002 +0000 UTC m=+18.169193161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs") pod "network-metrics-daemon-9xhpt" (UID: "1625d5d8-1437-4fb6-b18e-320acb890173") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:47:05.910242 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:05.910028    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc podName:f70415e6-52d5-4096-9d21-82b7ca994f61 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:13.910012863 +0000 UTC m=+18.169235011 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bkc" (UniqueName: "kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc") pod "network-check-target-p24tw" (UID: "f70415e6-52d5-4096-9d21-82b7ca994f61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:47:06.355878 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:06.355242    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:06.355878 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:06.355357    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:06.355878 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:06.355440    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:06.355878 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:06.355837    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:08.354325 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:08.354283    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:08.354717 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:08.354283    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:08.354717 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:08.354411    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:08.354717 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:08.354438    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:10.354219 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:10.353994    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:10.354760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:10.353995    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:10.354760 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:10.354314    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:10.354760 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:10.354397    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:12.354207 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:12.354172    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:12.354207 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:12.354208    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:12.354743 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:12.354314    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:12.354743 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:12.354419    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:13.971699 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:13.971666    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:13.972169 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:13.971734    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bkc\" (UniqueName: \"kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc\") pod \"network-check-target-p24tw\" (UID: \"f70415e6-52d5-4096-9d21-82b7ca994f61\") " pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:13.972169 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:13.971844    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:47:13.972169 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:13.971913    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs podName:1625d5d8-1437-4fb6-b18e-320acb890173 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:29.971893925 +0000 UTC m=+34.231116084 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs") pod "network-metrics-daemon-9xhpt" (UID: "1625d5d8-1437-4fb6-b18e-320acb890173") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:47:13.972169 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:13.971947    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 18:47:13.972169 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:13.971962    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 18:47:13.972169 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:13.971975    2577 projected.go:194] Error preparing data for projected volume kube-api-access-b2bkc for pod openshift-network-diagnostics/network-check-target-p24tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:47:13.972169 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:13.972022    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc podName:f70415e6-52d5-4096-9d21-82b7ca994f61 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:29.972009958 +0000 UTC m=+34.231232118 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bkc" (UniqueName: "kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc") pod "network-check-target-p24tw" (UID: "f70415e6-52d5-4096-9d21-82b7ca994f61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:47:14.353863 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:14.353783    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:14.354015 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:14.353780    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:14.354015 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:14.353942    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:14.354015 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:14.354004    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:16.354943 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:16.354903    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:16.355361 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:16.354992    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:16.355361 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:16.355072    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:16.355361 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:16.355159    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:17.459737 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.459463    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d7lqp" event={"ID":"1f18d702-cac3-41b8-b691-872943d4862a","Type":"ContainerStarted","Data":"64b02aa131bc1bb6be94e966054d27cb03a70781f9a365a826e06a4fa966f0a6"}
May 28 18:47:17.460878 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.460851    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qbwcr" event={"ID":"4b1230db-d35f-489e-a592-e9d91bbb6d74","Type":"ContainerStarted","Data":"9e31a0cd68580bb37deec94c20b66bff06159eca0d14ae8332be418afd78281e"}
May 28 18:47:17.462151 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.462123    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c4fhx" event={"ID":"a83d99d8-7c19-4e57-aca6-26f8a033d681","Type":"ContainerStarted","Data":"0964fc6d872b9f81538e8704266ab89f467750648291c4d1c805cc4bd52cea48"}
May 28 18:47:17.463542 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.463507    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-98z5n" event={"ID":"90a5b717-a5bb-4b1e-b516-b3dec5e1601e","Type":"ContainerStarted","Data":"331f5cf06aabf919a5eaf8c38f85938daeb69bea4beae40b2d68af99ff6bce4f"}
May 28 18:47:17.464883 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.464863    2577 generic.go:358] "Generic (PLEG): container finished" podID="01a5120d-33a6-42dd-b48c-7490c470f11f" containerID="2db6090d423bb2746bbd48114749b04bdd24961e3f4dfc29000c8423ee4e8fa8" exitCode=0
May 28 18:47:17.464952 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.464939    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cg96j" event={"ID":"01a5120d-33a6-42dd-b48c-7490c470f11f","Type":"ContainerDied","Data":"2db6090d423bb2746bbd48114749b04bdd24961e3f4dfc29000c8423ee4e8fa8"}
May 28 18:47:17.468084 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.468060    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl" event={"ID":"1573bbfa-cdc9-48b4-a667-ab2be639d3a8","Type":"ContainerStarted","Data":"0ec65e877e37c2bc5666e189a9f1b1b61e1f2c15f4b0a9b26e1b612cddaa7cfb"}
May 28 18:47:17.468156 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.468086    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl" event={"ID":"1573bbfa-cdc9-48b4-a667-ab2be639d3a8","Type":"ContainerStarted","Data":"7d143efb0da90b4693d7255ed19bc343ba6673a0ae238434a035b9db1d5e713d"}
May 28 18:47:17.468156 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.468100    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl" event={"ID":"1573bbfa-cdc9-48b4-a667-ab2be639d3a8","Type":"ContainerStarted","Data":"de6e6c5013fb24c113e40e0a17d121e0ce33fae09d960953880c47a8790a4fad"}
May 28 18:47:17.468156 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.468108    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl" event={"ID":"1573bbfa-cdc9-48b4-a667-ab2be639d3a8","Type":"ContainerStarted","Data":"9f2cbc7af91769c40bb9ea59e295427833cb947f9c2684f5d6c314003f9a4f9f"}
May 28 18:47:17.468156 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.468125    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl" event={"ID":"1573bbfa-cdc9-48b4-a667-ab2be639d3a8","Type":"ContainerStarted","Data":"64d2d4726a71ca0bca80501f7e54080a53d4c9e72ba47c65c8475a026611c06e"}
May 28 18:47:17.469265 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.469240    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pm7cd" event={"ID":"532d9740-8a10-452a-88d7-64ecd63e0f66","Type":"ContainerStarted","Data":"8c69c7fbeb26e907df615c3fe04cf90579222134ec4501360fffae82fb5b76bb"}
May 28 18:47:17.470363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.470343    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq" event={"ID":"96afef30-52b0-4e8e-b918-4ff7f35dc403","Type":"ContainerStarted","Data":"ecbc9181ca5f61ed7319dfbfff0b943aed1b5a0f5e002170a477d0dbfd7a8594"}
May 28 18:47:17.475811 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.475751    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-d7lqp" podStartSLOduration=3.437986923 podStartE2EDuration="21.47574106s" podCreationTimestamp="2026-05-28 18:46:56 +0000 UTC" firstStartedPulling="2026-05-28 18:46:58.872571922 +0000 UTC m=+3.131794079" lastFinishedPulling="2026-05-28 18:47:16.91032606 +0000 UTC m=+21.169548216" observedRunningTime="2026-05-28 18:47:17.475178919 +0000 UTC m=+21.734401086" watchObservedRunningTime="2026-05-28 18:47:17.47574106 +0000 UTC m=+21.734963225"
May 28 18:47:17.475892 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.475837    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal" podStartSLOduration=20.475832072 podStartE2EDuration="20.475832072s" podCreationTimestamp="2026-05-28 18:46:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 18:47:00.440833441 +0000 UTC m=+4.700055608" watchObservedRunningTime="2026-05-28 18:47:17.475832072 +0000 UTC m=+21.735054239"
May 28 18:47:17.490685 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.490633    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qbwcr" podStartSLOduration=7.422267232 podStartE2EDuration="21.490598327s" podCreationTimestamp="2026-05-28 18:46:56 +0000 UTC" firstStartedPulling="2026-05-28 18:46:58.870829968 +0000 UTC m=+3.130052122" lastFinishedPulling="2026-05-28 18:47:12.939161069 +0000 UTC m=+17.198383217" observedRunningTime="2026-05-28 18:47:17.490561199 +0000 UTC m=+21.749783365" watchObservedRunningTime="2026-05-28 18:47:17.490598327 +0000 UTC m=+21.749820475"
May 28 18:47:17.504434 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.504382    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-c4fhx" podStartSLOduration=3.478031097 podStartE2EDuration="21.504368489s" podCreationTimestamp="2026-05-28 18:46:56 +0000 UTC" firstStartedPulling="2026-05-28 18:46:58.88224851 +0000 UTC m=+3.141470661" lastFinishedPulling="2026-05-28 18:47:16.908585909 +0000 UTC m=+21.167808053" observedRunningTime="2026-05-28 18:47:17.503697859 +0000 UTC m=+21.762920025" watchObservedRunningTime="2026-05-28 18:47:17.504368489 +0000 UTC m=+21.763590654"
May 28 18:47:17.540797 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.540744    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pm7cd" podStartSLOduration=3.509699837 podStartE2EDuration="21.540726887s" podCreationTimestamp="2026-05-28 18:46:56 +0000 UTC" firstStartedPulling="2026-05-28 18:46:58.877629434 +0000 UTC m=+3.136851589" lastFinishedPulling="2026-05-28 18:47:16.908656488 +0000 UTC m=+21.167878639" observedRunningTime="2026-05-28 18:47:17.540130792 +0000 UTC m=+21.799352962" watchObservedRunningTime="2026-05-28 18:47:17.540726887 +0000 UTC m=+21.799949053"
May 28 18:47:17.558249 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:17.558192    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-98z5n" podStartSLOduration=3.474568245 podStartE2EDuration="21.558175913s" podCreationTimestamp="2026-05-28 18:46:56 +0000 UTC" firstStartedPulling="2026-05-28 18:46:58.880580984 +0000 UTC m=+3.139803139" lastFinishedPulling="2026-05-28 18:47:16.964188648 +0000 UTC m=+21.223410807" observedRunningTime="2026-05-28 18:47:17.557943905 +0000 UTC m=+21.817166063" watchObservedRunningTime="2026-05-28 18:47:17.558175913 +0000 UTC m=+21.817398081"
May 28 18:47:18.062668 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:18.062424    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 18:47:18.297147 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:18.297043    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T18:47:18.062664265Z","UUID":"16cba8a6-e32a-4ecb-aa29-4c45974c2ae8","Handler":null,"Name":"","Endpoint":""}
May 28 18:47:18.298836 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:18.298809    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 18:47:18.298959 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:18.298842    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 18:47:18.353825 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:18.353792    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:18.354012 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:18.353792    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:18.354012 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:18.353929    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:18.354012 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:18.354004    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:18.475509 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:18.475422    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl" event={"ID":"1573bbfa-cdc9-48b4-a667-ab2be639d3a8","Type":"ContainerStarted","Data":"f85a07ed9561a24aca158e5875f7a395350fabba16986af37647dd19de5ae854"}
May 28 18:47:18.477104 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:18.477079    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq" event={"ID":"96afef30-52b0-4e8e-b918-4ff7f35dc403","Type":"ContainerStarted","Data":"66733e1b3470d4ee6ee875c3a7bee746c7d8968ddb1bb5f6888d726b5c4f9065"}
May 28 18:47:18.478486 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:18.478448    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9mmz8" event={"ID":"b56331e8-f658-4d66-9369-6c0828f65d6d","Type":"ContainerStarted","Data":"8d94974813067f98f658e9cdb1b894c55d8d46e8b505a139889ace7b74f41d59"}
May 28 18:47:18.493293 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:18.493245    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9mmz8" podStartSLOduration=8.430434098 podStartE2EDuration="22.493223013s" podCreationTimestamp="2026-05-28 18:46:56 +0000 UTC" firstStartedPulling="2026-05-28 18:46:58.876369831 +0000 UTC m=+3.135591990" lastFinishedPulling="2026-05-28 18:47:12.939158749 +0000 UTC m=+17.198380905" observedRunningTime="2026-05-28 18:47:18.49303248 +0000 UTC m=+22.752254662" watchObservedRunningTime="2026-05-28 18:47:18.493223013 +0000 UTC m=+22.752445179"
May 28 18:47:19.013460 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:19.013432    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pm7cd"
May 28 18:47:19.014104 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:19.014083    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pm7cd"
May 28 18:47:19.482710 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:19.482674    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq" event={"ID":"96afef30-52b0-4e8e-b918-4ff7f35dc403","Type":"ContainerStarted","Data":"afddb991fe935a59f8d65f51f154e8c8de59581ab7a1e19267f7574e20fca637"}
May 28 18:47:19.483219 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:19.483118    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pm7cd"
May 28 18:47:19.483350 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:19.483333    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pm7cd"
May 28 18:47:19.501660 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:19.500286    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pvbkq" podStartSLOduration=3.45229421 podStartE2EDuration="23.500265329s" podCreationTimestamp="2026-05-28 18:46:56 +0000 UTC" firstStartedPulling="2026-05-28 18:46:58.873007445 +0000 UTC m=+3.132229603" lastFinishedPulling="2026-05-28 18:47:18.920978565 +0000 UTC m=+23.180200722" observedRunningTime="2026-05-28 18:47:19.500019833 +0000 UTC m=+23.759242000" watchObservedRunningTime="2026-05-28 18:47:19.500265329 +0000 UTC m=+23.759487496"
May 28 18:47:20.354734 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:20.354698    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:20.354936 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:20.354698    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:20.354936 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:20.354831    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:20.355051 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:20.354941    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:20.487420 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:20.487379    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl" event={"ID":"1573bbfa-cdc9-48b4-a667-ab2be639d3a8","Type":"ContainerStarted","Data":"487395af5ea1c8d6a19031e14886a3f180ad5d9c9ad96c319a369072c539260a"}
May 28 18:47:22.356941 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:22.356754    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:22.357384 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:22.356755    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:22.357384 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:22.357022    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:22.357384 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:22.357075    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:22.492940 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:22.492850    2577 generic.go:358] "Generic (PLEG): container finished" podID="01a5120d-33a6-42dd-b48c-7490c470f11f" containerID="681e183873fdd9940305e7b173d964b7cadb638e9f24a1195e1ad32803cf748e" exitCode=0
May 28 18:47:22.493107 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:22.492935    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cg96j" event={"ID":"01a5120d-33a6-42dd-b48c-7490c470f11f","Type":"ContainerDied","Data":"681e183873fdd9940305e7b173d964b7cadb638e9f24a1195e1ad32803cf748e"}
May 28 18:47:22.496145 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:22.496120    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl" event={"ID":"1573bbfa-cdc9-48b4-a667-ab2be639d3a8","Type":"ContainerStarted","Data":"83caece0a0cd6907c1231ae6e17b59946fcd12261b38024177a40e9e6e22f0d7"}
May 28 18:47:22.496486 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:22.496462    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:47:22.496547 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:22.496488    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:47:22.512311 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:22.512284    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:47:22.538486 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:22.538432    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl" podStartSLOduration=8.484078603 podStartE2EDuration="26.538417678s" podCreationTimestamp="2026-05-28 18:46:56 +0000 UTC" firstStartedPulling="2026-05-28 18:46:58.87947854 +0000 UTC m=+3.138700698" lastFinishedPulling="2026-05-28 18:47:16.933817609 +0000 UTC m=+21.193039773" observedRunningTime="2026-05-28 18:47:22.536604134 +0000 UTC m=+26.795826323" watchObservedRunningTime="2026-05-28 18:47:22.538417678 +0000 UTC m=+26.797639861"
May 28 18:47:23.498236 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:23.498208    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:47:23.512701 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:23.512663    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:47:23.989023 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:23.988839    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9xhpt"]
May 28 18:47:23.989212 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:23.989119    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:23.989253 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:23.989206    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:23.991160 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:23.991126    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p24tw"]
May 28 18:47:23.991262 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:23.991208    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:23.991312 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:23.991280    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:24.501809 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:24.501770    2577 generic.go:358] "Generic (PLEG): container finished" podID="01a5120d-33a6-42dd-b48c-7490c470f11f" containerID="fdd94d964e49f98feef3434d57994984326a721d02cdad2f9099b14272acf373" exitCode=0
May 28 18:47:24.502227 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:24.501853    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cg96j" event={"ID":"01a5120d-33a6-42dd-b48c-7490c470f11f","Type":"ContainerDied","Data":"fdd94d964e49f98feef3434d57994984326a721d02cdad2f9099b14272acf373"}
May 28 18:47:25.354809 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:25.354773    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:25.354957 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:25.354885    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:25.354957 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:25.354940    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:25.355041 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:25.355024    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:26.507337 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:26.507305    2577 generic.go:358] "Generic (PLEG): container finished" podID="01a5120d-33a6-42dd-b48c-7490c470f11f" containerID="9f7ae02f0a4e78f0e0180d19e4786c54241e6ed7e977e6cd895d88cf4dd30f20" exitCode=0
May 28 18:47:26.508111 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:26.507342    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cg96j" event={"ID":"01a5120d-33a6-42dd-b48c-7490c470f11f","Type":"ContainerDied","Data":"9f7ae02f0a4e78f0e0180d19e4786c54241e6ed7e977e6cd895d88cf4dd30f20"}
May 28 18:47:27.354013 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:27.353974    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:27.354187 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:27.353974    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:27.354187 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:27.354095    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:27.354187 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:27.354158    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:29.354719 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:29.354684    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:29.354719 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:29.354704    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:29.355458 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:29.354815    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p24tw" podUID="f70415e6-52d5-4096-9d21-82b7ca994f61"
May 28 18:47:29.355458 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:29.354941    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xhpt" podUID="1625d5d8-1437-4fb6-b18e-320acb890173"
May 28 18:47:29.993440 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:29.993397    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bkc\" (UniqueName: \"kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc\") pod \"network-check-target-p24tw\" (UID: \"f70415e6-52d5-4096-9d21-82b7ca994f61\") " pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:29.993676 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:29.993452    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:29.993676 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:29.993555    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:47:29.993676 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:29.993556    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 18:47:29.993676 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:29.993662    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 18:47:29.993676 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:29.993673    2577 projected.go:194] Error preparing data for projected volume kube-api-access-b2bkc for pod openshift-network-diagnostics/network-check-target-p24tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:47:29.993867 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:29.993646    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs podName:1625d5d8-1437-4fb6-b18e-320acb890173 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:01.993603714 +0000 UTC m=+66.252825874 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs") pod "network-metrics-daemon-9xhpt" (UID: "1625d5d8-1437-4fb6-b18e-320acb890173") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 18:47:29.993867 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:29.993723    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc podName:f70415e6-52d5-4096-9d21-82b7ca994f61 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:01.993711273 +0000 UTC m=+66.252933417 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bkc" (UniqueName: "kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc") pod "network-check-target-p24tw" (UID: "f70415e6-52d5-4096-9d21-82b7ca994f61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 18:47:30.026503 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.026475    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeReady"
May 28 18:47:30.026708 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.026634    2577 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 18:47:30.068927 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.068892    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4n69j"]
May 28 18:47:30.089754 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.089726    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hmmqn"]
May 28 18:47:30.089960 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.089942    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4n69j"
May 28 18:47:30.092749 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.092626    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 18:47:30.092749 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.092639    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l78mj\""
May 28 18:47:30.092954 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.092931    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 18:47:30.107485 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.107460    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4n69j"]
May 28 18:47:30.107603 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.107492    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hmmqn"]
May 28 18:47:30.107673 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.107633    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:47:30.110029 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.110005    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zpgsq\""
May 28 18:47:30.110245 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.110228    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 18:47:30.110481 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.110299    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 18:47:30.110481 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.110344    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 18:47:30.195179 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.195138    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:30.195179 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.195176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert\") pod \"ingress-canary-hmmqn\" (UID: \"74ae6b85-71a4-4b53-a23e-4975b6464338\") " pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:47:30.195426 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.195210    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgvww\" (UniqueName: \"kubernetes.io/projected/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-kube-api-access-tgvww\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:30.195426 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.195349    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx66c\" (UniqueName: \"kubernetes.io/projected/74ae6b85-71a4-4b53-a23e-4975b6464338-kube-api-access-zx66c\") pod \"ingress-canary-hmmqn\" (UID: \"74ae6b85-71a4-4b53-a23e-4975b6464338\") " pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:47:30.195426 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.195392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-config-volume\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:30.195562 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.195486    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-tmp-dir\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:30.296794 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.296714    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zx66c\" (UniqueName: \"kubernetes.io/projected/74ae6b85-71a4-4b53-a23e-4975b6464338-kube-api-access-zx66c\") pod \"ingress-canary-hmmqn\" (UID: \"74ae6b85-71a4-4b53-a23e-4975b6464338\") " pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:47:30.296794 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.296756    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-config-volume\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:30.296794 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.296778    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-tmp-dir\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:30.297063 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.296810    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:30.297063 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.296825    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert\") pod \"ingress-canary-hmmqn\" (UID: \"74ae6b85-71a4-4b53-a23e-4975b6464338\") " pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:47:30.297063 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.296848    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tgvww\" (UniqueName: \"kubernetes.io/projected/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-kube-api-access-tgvww\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:30.297063 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:30.296943    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 18:47:30.297063 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:30.297009    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls podName:0c79a026-b52c-4e1e-9720-1bbe00a9ed38 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:30.796993918 +0000 UTC m=+35.056216066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls") pod "dns-default-4n69j" (UID: "0c79a026-b52c-4e1e-9720-1bbe00a9ed38") : secret "dns-default-metrics-tls" not found
May 28 18:47:30.297063 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:30.296943    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 18:47:30.297356 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:30.297116    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert podName:74ae6b85-71a4-4b53-a23e-4975b6464338 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:30.797100854 +0000 UTC m=+35.056322998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert") pod "ingress-canary-hmmqn" (UID: "74ae6b85-71a4-4b53-a23e-4975b6464338") : secret "canary-serving-cert" not found
May 28 18:47:30.297356 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.297165    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-tmp-dir\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:30.297448 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.297370    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-config-volume\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:30.308808 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.308690    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgvww\" (UniqueName: \"kubernetes.io/projected/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-kube-api-access-tgvww\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:30.308894 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.308750    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx66c\" (UniqueName: \"kubernetes.io/projected/74ae6b85-71a4-4b53-a23e-4975b6464338-kube-api-access-zx66c\") pod \"ingress-canary-hmmqn\" (UID: \"74ae6b85-71a4-4b53-a23e-4975b6464338\") " pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:47:30.801086 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.801048    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:30.801086 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:30.801088    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert\") pod \"ingress-canary-hmmqn\" (UID: \"74ae6b85-71a4-4b53-a23e-4975b6464338\") " pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:47:30.801600 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:30.801236    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 18:47:30.801600 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:30.801311    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert podName:74ae6b85-71a4-4b53-a23e-4975b6464338 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:31.801296815 +0000 UTC m=+36.060518965 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert") pod "ingress-canary-hmmqn" (UID: "74ae6b85-71a4-4b53-a23e-4975b6464338") : secret "canary-serving-cert" not found
May 28 18:47:30.801600 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:30.801234    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 18:47:30.801600 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:30.801367    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls podName:0c79a026-b52c-4e1e-9720-1bbe00a9ed38 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:31.801355691 +0000 UTC m=+36.060577839 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls") pod "dns-default-4n69j" (UID: "0c79a026-b52c-4e1e-9720-1bbe00a9ed38") : secret "dns-default-metrics-tls" not found
May 28 18:47:31.354280 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:31.354250    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:47:31.354460 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:31.354260    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:47:31.357479 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:31.357455    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2kstc\""
May 28 18:47:31.357605 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:31.357563    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 18:47:31.358549 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:31.358528    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 18:47:31.358549 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:31.358544    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-25f4v\""
May 28 18:47:31.358739 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:31.358595    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 18:47:31.808560 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:31.808521    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:31.808970 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:31.808569    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert\") pod \"ingress-canary-hmmqn\" (UID: \"74ae6b85-71a4-4b53-a23e-4975b6464338\") " pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:47:31.808970 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:31.808702    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 18:47:31.808970 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:31.808781    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls podName:0c79a026-b52c-4e1e-9720-1bbe00a9ed38 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:33.80876152 +0000 UTC m=+38.067983667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls") pod "dns-default-4n69j" (UID: "0c79a026-b52c-4e1e-9720-1bbe00a9ed38") : secret "dns-default-metrics-tls" not found
May 28 18:47:31.808970 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:31.808702    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 18:47:31.808970 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:31.808884    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert podName:74ae6b85-71a4-4b53-a23e-4975b6464338 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:33.808862113 +0000 UTC m=+38.068084295 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert") pod "ingress-canary-hmmqn" (UID: "74ae6b85-71a4-4b53-a23e-4975b6464338") : secret "canary-serving-cert" not found
May 28 18:47:33.522363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:33.522331    2577 generic.go:358] "Generic (PLEG): container finished" podID="01a5120d-33a6-42dd-b48c-7490c470f11f" containerID="c16c32568aa40545da689550193ee0ca83b6e1bbbf9ec5a3c9eec638454e2f16" exitCode=0
May 28 18:47:33.522832 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:33.522394    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cg96j" event={"ID":"01a5120d-33a6-42dd-b48c-7490c470f11f","Type":"ContainerDied","Data":"c16c32568aa40545da689550193ee0ca83b6e1bbbf9ec5a3c9eec638454e2f16"}
May 28 18:47:33.827306 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:33.827201    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:33.827306 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:33.827251    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert\") pod \"ingress-canary-hmmqn\" (UID: \"74ae6b85-71a4-4b53-a23e-4975b6464338\") " pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:47:33.827517 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:33.827347    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 18:47:33.827517 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:33.827383    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 18:47:33.827517 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:33.827414    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls podName:0c79a026-b52c-4e1e-9720-1bbe00a9ed38 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:37.827399352 +0000 UTC m=+42.086621496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls") pod "dns-default-4n69j" (UID: "0c79a026-b52c-4e1e-9720-1bbe00a9ed38") : secret "dns-default-metrics-tls" not found
May 28 18:47:33.827517 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:33.827442    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert podName:74ae6b85-71a4-4b53-a23e-4975b6464338 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:37.827428028 +0000 UTC m=+42.086650172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert") pod "ingress-canary-hmmqn" (UID: "74ae6b85-71a4-4b53-a23e-4975b6464338") : secret "canary-serving-cert" not found
May 28 18:47:34.527242 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:34.527206    2577 generic.go:358] "Generic (PLEG): container finished" podID="01a5120d-33a6-42dd-b48c-7490c470f11f" containerID="021f135e1eaa12f7c72953b5dc955c7823ff3805645496428d6074acd0a87ad1" exitCode=0
May 28 18:47:34.527719 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:34.527264    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cg96j" event={"ID":"01a5120d-33a6-42dd-b48c-7490c470f11f","Type":"ContainerDied","Data":"021f135e1eaa12f7c72953b5dc955c7823ff3805645496428d6074acd0a87ad1"}
May 28 18:47:35.531911 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:35.531869    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cg96j" event={"ID":"01a5120d-33a6-42dd-b48c-7490c470f11f","Type":"ContainerStarted","Data":"4b07caa802bf019d3474123b02ab8c1e8f5c05b04a771d4e4307a862d2b31fff"}
May 28 18:47:35.554921 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:35.554877    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cg96j" podStartSLOduration=5.575483471 podStartE2EDuration="39.554862067s" podCreationTimestamp="2026-05-28 18:46:56 +0000 UTC" firstStartedPulling="2026-05-28 18:46:58.881912287 +0000 UTC m=+3.141134434" lastFinishedPulling="2026-05-28 18:47:32.861290883 +0000 UTC m=+37.120513030" observedRunningTime="2026-05-28 18:47:35.553573305 +0000 UTC m=+39.812795471" watchObservedRunningTime="2026-05-28 18:47:35.554862067 +0000 UTC m=+39.814084233"
May 28 18:47:37.853403 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:37.853344    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:37.853403 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:37.853405    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert\") pod \"ingress-canary-hmmqn\" (UID: \"74ae6b85-71a4-4b53-a23e-4975b6464338\") " pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:47:37.853874 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:37.853511    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 18:47:37.853874 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:37.853546    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 18:47:37.853874 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:37.853597    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls podName:0c79a026-b52c-4e1e-9720-1bbe00a9ed38 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:45.853571687 +0000 UTC m=+50.112793892 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls") pod "dns-default-4n69j" (UID: "0c79a026-b52c-4e1e-9720-1bbe00a9ed38") : secret "dns-default-metrics-tls" not found
May 28 18:47:37.853874 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:37.853654    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert podName:74ae6b85-71a4-4b53-a23e-4975b6464338 nodeName:}" failed. No retries permitted until 2026-05-28 18:47:45.853643525 +0000 UTC m=+50.112865674 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert") pod "ingress-canary-hmmqn" (UID: "74ae6b85-71a4-4b53-a23e-4975b6464338") : secret "canary-serving-cert" not found
May 28 18:47:45.909857 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:45.909813    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:47:45.909857 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:45.909853    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert\") pod \"ingress-canary-hmmqn\" (UID: \"74ae6b85-71a4-4b53-a23e-4975b6464338\") " pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:47:45.910402 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:45.909963    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 18:47:45.910402 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:45.909964    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 18:47:45.910402 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:45.910019    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert podName:74ae6b85-71a4-4b53-a23e-4975b6464338 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:01.910004643 +0000 UTC m=+66.169226787 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert") pod "ingress-canary-hmmqn" (UID: "74ae6b85-71a4-4b53-a23e-4975b6464338") : secret "canary-serving-cert" not found
May 28 18:47:45.910402 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:45.910031    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls podName:0c79a026-b52c-4e1e-9720-1bbe00a9ed38 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:01.910025542 +0000 UTC m=+66.169247685 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls") pod "dns-default-4n69j" (UID: "0c79a026-b52c-4e1e-9720-1bbe00a9ed38") : secret "dns-default-metrics-tls" not found
May 28 18:47:55.011461 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.011427    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"]
May 28 18:47:55.049077 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.049037    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"]
May 28 18:47:55.049253 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.049184    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:47:55.053066 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.053037    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
May 28 18:47:55.054623 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.054585    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
May 28 18:47:55.055064 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.054748    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vqdtm\""
May 28 18:47:55.055064 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.054755    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 18:47:55.055064 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.054920    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 18:47:55.074343 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.074307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:47:55.074497 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.074358    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/932039c7-8302-443b-9af2-30c993764ffa-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:47:55.074497 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.074427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lvnk\" (UniqueName: \"kubernetes.io/projected/932039c7-8302-443b-9af2-30c993764ffa-kube-api-access-2lvnk\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:47:55.111157 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.111125    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-jx2bq"]
May 28 18:47:55.123590 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.123562    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-jx2bq"]
May 28 18:47:55.123733 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.123719    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.126385 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.126346    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
May 28 18:47:55.126385 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.126360    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
May 28 18:47:55.126593 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.126390    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 18:47:55.126593 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.126478    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 18:47:55.126763 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.126593    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mlr4z\""
May 28 18:47:55.131913 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.131896    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
May 28 18:47:55.175102 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.175061    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/932039c7-8302-443b-9af2-30c993764ffa-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:47:55.175102 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.175109    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a0962b8-484f-46c3-8f88-4eaf3130145e-service-ca-bundle\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.175348 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.175219    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4tbj\" (UniqueName: \"kubernetes.io/projected/3a0962b8-484f-46c3-8f88-4eaf3130145e-kube-api-access-t4tbj\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.175348 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.175256    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2lvnk\" (UniqueName: \"kubernetes.io/projected/932039c7-8302-443b-9af2-30c993764ffa-kube-api-access-2lvnk\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:47:55.175348 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.175278    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3a0962b8-484f-46c3-8f88-4eaf3130145e-serving-cert\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.175348 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.175323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a0962b8-484f-46c3-8f88-4eaf3130145e-tmp\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.175527 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.175394    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a0962b8-484f-46c3-8f88-4eaf3130145e-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.175527 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.175428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3a0962b8-484f-46c3-8f88-4eaf3130145e-snapshots\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.175527 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.175461    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:47:55.175690 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:55.175565    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 18:47:55.175690 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:55.175647    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls podName:932039c7-8302-443b-9af2-30c993764ffa nodeName:}" failed. No retries permitted until 2026-05-28 18:47:55.67562876 +0000 UTC m=+59.934850921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-bk4zh" (UID: "932039c7-8302-443b-9af2-30c993764ffa") : secret "cluster-monitoring-operator-tls" not found
May 28 18:47:55.175837 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.175820    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/932039c7-8302-443b-9af2-30c993764ffa-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:47:55.186543 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.186517    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lvnk\" (UniqueName: \"kubernetes.io/projected/932039c7-8302-443b-9af2-30c993764ffa-kube-api-access-2lvnk\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:47:55.276783 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.276701    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a0962b8-484f-46c3-8f88-4eaf3130145e-tmp\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.276783 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.276753    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a0962b8-484f-46c3-8f88-4eaf3130145e-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.277031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.277008    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3a0962b8-484f-46c3-8f88-4eaf3130145e-snapshots\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.277117 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.277098    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a0962b8-484f-46c3-8f88-4eaf3130145e-tmp\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.277175 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.277100    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a0962b8-484f-46c3-8f88-4eaf3130145e-service-ca-bundle\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.277175 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.277150    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t4tbj\" (UniqueName: \"kubernetes.io/projected/3a0962b8-484f-46c3-8f88-4eaf3130145e-kube-api-access-t4tbj\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.277263 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.277179    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3a0962b8-484f-46c3-8f88-4eaf3130145e-serving-cert\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.277583 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.277562    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a0962b8-484f-46c3-8f88-4eaf3130145e-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.277688 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.277589    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a0962b8-484f-46c3-8f88-4eaf3130145e-service-ca-bundle\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.277688 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.277563    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3a0962b8-484f-46c3-8f88-4eaf3130145e-snapshots\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.279284 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.279268    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3a0962b8-484f-46c3-8f88-4eaf3130145e-serving-cert\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.285112 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.285083    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4tbj\" (UniqueName: \"kubernetes.io/projected/3a0962b8-484f-46c3-8f88-4eaf3130145e-kube-api-access-t4tbj\") pod \"insights-operator-78bdfb888c-jx2bq\" (UID: \"3a0962b8-484f-46c3-8f88-4eaf3130145e\") " pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.433183 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.433147    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-jx2bq"
May 28 18:47:55.526066 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.526030    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9h2tl"
May 28 18:47:55.647129 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.647097    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-jx2bq"]
May 28 18:47:55.650481 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:47:55.650459    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a0962b8_484f_46c3_8f88_4eaf3130145e.slice/crio-210f4f0ff94802128f2b533f8737078b7f6b034896f8563723030acf0767a72f WatchSource:0}: Error finding container 210f4f0ff94802128f2b533f8737078b7f6b034896f8563723030acf0767a72f: Status 404 returned error can't find the container with id 210f4f0ff94802128f2b533f8737078b7f6b034896f8563723030acf0767a72f
May 28 18:47:55.681308 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:55.681269    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:47:55.681465 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:55.681429    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 18:47:55.681527 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:55.681494    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls podName:932039c7-8302-443b-9af2-30c993764ffa nodeName:}" failed. No retries permitted until 2026-05-28 18:47:56.681477361 +0000 UTC m=+60.940699508 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-bk4zh" (UID: "932039c7-8302-443b-9af2-30c993764ffa") : secret "cluster-monitoring-operator-tls" not found
May 28 18:47:56.572032 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:56.571997    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-jx2bq" event={"ID":"3a0962b8-484f-46c3-8f88-4eaf3130145e","Type":"ContainerStarted","Data":"210f4f0ff94802128f2b533f8737078b7f6b034896f8563723030acf0767a72f"}
May 28 18:47:56.688143 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:56.688095    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:47:56.688344 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:56.688262    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 18:47:56.688344 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:56.688343    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls podName:932039c7-8302-443b-9af2-30c993764ffa nodeName:}" failed. No retries permitted until 2026-05-28 18:47:58.688324673 +0000 UTC m=+62.947546830 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-bk4zh" (UID: "932039c7-8302-443b-9af2-30c993764ffa") : secret "cluster-monitoring-operator-tls" not found
May 28 18:47:58.580443 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:58.580401    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-jx2bq" event={"ID":"3a0962b8-484f-46c3-8f88-4eaf3130145e","Type":"ContainerStarted","Data":"2442a03d3e007f2fc4364280143d11322cfe66241746b59d911e00b4400265a5"}
May 28 18:47:58.598416 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:58.598368    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-78bdfb888c-jx2bq" podStartSLOduration=1.6193845489999998 podStartE2EDuration="3.5983538s" podCreationTimestamp="2026-05-28 18:47:55 +0000 UTC" firstStartedPulling="2026-05-28 18:47:55.652154592 +0000 UTC m=+59.911376738" lastFinishedPulling="2026-05-28 18:47:57.631123841 +0000 UTC m=+61.890345989" observedRunningTime="2026-05-28 18:47:58.597044054 +0000 UTC m=+62.856266221" watchObservedRunningTime="2026-05-28 18:47:58.5983538 +0000 UTC m=+62.857575967"
May 28 18:47:58.704091 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:47:58.704055    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:47:58.704250 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:58.704200    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 18:47:58.704294 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:47:58.704283    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls podName:932039c7-8302-443b-9af2-30c993764ffa nodeName:}" failed. No retries permitted until 2026-05-28 18:48:02.704266768 +0000 UTC m=+66.963488915 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-bk4zh" (UID: "932039c7-8302-443b-9af2-30c993764ffa") : secret "cluster-monitoring-operator-tls" not found
May 28 18:48:00.906761 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:00.906734    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c4fhx_a83d99d8-7c19-4e57-aca6-26f8a033d681/dns-node-resolver/0.log"
May 28 18:48:01.706800 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:01.706773    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qbwcr_4b1230db-d35f-489e-a592-e9d91bbb6d74/node-ca/0.log"
May 28 18:48:01.928835 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:01.928797    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:48:01.928835 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:01.928836    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert\") pod \"ingress-canary-hmmqn\" (UID: \"74ae6b85-71a4-4b53-a23e-4975b6464338\") " pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:48:01.929248 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:01.928950    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 18:48:01.929248 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:01.929015    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls podName:0c79a026-b52c-4e1e-9720-1bbe00a9ed38 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:33.929001344 +0000 UTC m=+98.188223488 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls") pod "dns-default-4n69j" (UID: "0c79a026-b52c-4e1e-9720-1bbe00a9ed38") : secret "dns-default-metrics-tls" not found
May 28 18:48:01.929248 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:01.928958    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 18:48:01.929248 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:01.929084    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert podName:74ae6b85-71a4-4b53-a23e-4975b6464338 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:33.92907271 +0000 UTC m=+98.188294854 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert") pod "ingress-canary-hmmqn" (UID: "74ae6b85-71a4-4b53-a23e-4975b6464338") : secret "canary-serving-cert" not found
May 28 18:48:02.029843 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:02.029771    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bkc\" (UniqueName: \"kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc\") pod \"network-check-target-p24tw\" (UID: \"f70415e6-52d5-4096-9d21-82b7ca994f61\") " pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:48:02.029843 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:02.029833    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:48:02.032343 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:02.032325    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 18:48:02.032523 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:02.032496    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 18:48:02.040477 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:02.040453    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 18:48:02.040579 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:02.040531    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs podName:1625d5d8-1437-4fb6-b18e-320acb890173 nodeName:}" failed. No retries permitted until 2026-05-28 18:49:06.040512544 +0000 UTC m=+130.299734688 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs") pod "network-metrics-daemon-9xhpt" (UID: "1625d5d8-1437-4fb6-b18e-320acb890173") : secret "metrics-daemon-secret" not found
May 28 18:48:02.042744 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:02.042727    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 18:48:02.053876 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:02.053850    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2bkc\" (UniqueName: \"kubernetes.io/projected/f70415e6-52d5-4096-9d21-82b7ca994f61-kube-api-access-b2bkc\") pod \"network-check-target-p24tw\" (UID: \"f70415e6-52d5-4096-9d21-82b7ca994f61\") " pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:48:02.268671 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:02.268636    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2kstc\""
May 28 18:48:02.276854 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:02.276814    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:48:02.397524 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:02.397489    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p24tw"]
May 28 18:48:02.401922 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:02.401886    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf70415e6_52d5_4096_9d21_82b7ca994f61.slice/crio-0cb6b45300636e1535528a3a1056eba8dd2285239ce135a5724989c278a8788e WatchSource:0}: Error finding container 0cb6b45300636e1535528a3a1056eba8dd2285239ce135a5724989c278a8788e: Status 404 returned error can't find the container with id 0cb6b45300636e1535528a3a1056eba8dd2285239ce135a5724989c278a8788e
May 28 18:48:02.588914 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:02.588835    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p24tw" event={"ID":"f70415e6-52d5-4096-9d21-82b7ca994f61","Type":"ContainerStarted","Data":"0cb6b45300636e1535528a3a1056eba8dd2285239ce135a5724989c278a8788e"}
May 28 18:48:02.736739 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:02.736702    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:48:02.736919 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:02.736851    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 18:48:02.736919 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:02.736915    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls podName:932039c7-8302-443b-9af2-30c993764ffa nodeName:}" failed. No retries permitted until 2026-05-28 18:48:10.736899999 +0000 UTC m=+74.996122142 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-bk4zh" (UID: "932039c7-8302-443b-9af2-30c993764ffa") : secret "cluster-monitoring-operator-tls" not found
May 28 18:48:03.137007 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:03.136973    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"]
May 28 18:48:03.139156 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:03.139130    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"
May 28 18:48:03.143147 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:03.142110    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
May 28 18:48:03.143147 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:03.142367    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
May 28 18:48:03.143147 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:03.142630    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
May 28 18:48:03.143385 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:03.143224    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tbbhl\""
May 28 18:48:03.150452 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:03.150425    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"]
May 28 18:48:03.240667 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:03.240630    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls\") pod \"cluster-samples-operator-697587599-l24d8\" (UID: \"45953cd7-31ed-4062-9c3a-f45873502c85\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"
May 28 18:48:03.240667 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:03.240670    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr69z\" (UniqueName: \"kubernetes.io/projected/45953cd7-31ed-4062-9c3a-f45873502c85-kube-api-access-qr69z\") pod \"cluster-samples-operator-697587599-l24d8\" (UID: \"45953cd7-31ed-4062-9c3a-f45873502c85\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"
May 28 18:48:03.341151 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:03.341106    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls\") pod \"cluster-samples-operator-697587599-l24d8\" (UID: \"45953cd7-31ed-4062-9c3a-f45873502c85\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"
May 28 18:48:03.341331 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:03.341163    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qr69z\" (UniqueName: \"kubernetes.io/projected/45953cd7-31ed-4062-9c3a-f45873502c85-kube-api-access-qr69z\") pod \"cluster-samples-operator-697587599-l24d8\" (UID: \"45953cd7-31ed-4062-9c3a-f45873502c85\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"
May 28 18:48:03.341331 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:03.341271    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 18:48:03.341433 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:03.341354    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls podName:45953cd7-31ed-4062-9c3a-f45873502c85 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:03.841331622 +0000 UTC m=+68.100553782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls") pod "cluster-samples-operator-697587599-l24d8" (UID: "45953cd7-31ed-4062-9c3a-f45873502c85") : secret "samples-operator-tls" not found
May 28 18:48:03.350154 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:03.350120    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr69z\" (UniqueName: \"kubernetes.io/projected/45953cd7-31ed-4062-9c3a-f45873502c85-kube-api-access-qr69z\") pod \"cluster-samples-operator-697587599-l24d8\" (UID: \"45953cd7-31ed-4062-9c3a-f45873502c85\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"
May 28 18:48:03.846761 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:03.846719    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls\") pod \"cluster-samples-operator-697587599-l24d8\" (UID: \"45953cd7-31ed-4062-9c3a-f45873502c85\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"
May 28 18:48:03.846962 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:03.846866    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 18:48:03.846962 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:03.846938    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls podName:45953cd7-31ed-4062-9c3a-f45873502c85 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:04.846919802 +0000 UTC m=+69.106141961 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls") pod "cluster-samples-operator-697587599-l24d8" (UID: "45953cd7-31ed-4062-9c3a-f45873502c85") : secret "samples-operator-tls" not found
May 28 18:48:04.043900 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.043862    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-kshzv"]
May 28 18:48:04.046038 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.046011    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.048720 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.048466    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
May 28 18:48:04.048720 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.048487    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-kngf2\""
May 28 18:48:04.048720 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.048531    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
May 28 18:48:04.048720 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.048535    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
May 28 18:48:04.049711 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.049692    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
May 28 18:48:04.054269 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.054249    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
May 28 18:48:04.055717 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.055671    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-kshzv"]
May 28 18:48:04.150058 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.150019    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3f6a673c-afbc-441f-a108-f66950a3558b-serving-cert\") pod \"console-operator-76dff48d59-kshzv\" (UID: \"3f6a673c-afbc-441f-a108-f66950a3558b\") " pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.150506 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.150072    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnrxf\" (UniqueName: \"kubernetes.io/projected/3f6a673c-afbc-441f-a108-f66950a3558b-kube-api-access-pnrxf\") pod \"console-operator-76dff48d59-kshzv\" (UID: \"3f6a673c-afbc-441f-a108-f66950a3558b\") " pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.150506 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.150196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3f6a673c-afbc-441f-a108-f66950a3558b-config\") pod \"console-operator-76dff48d59-kshzv\" (UID: \"3f6a673c-afbc-441f-a108-f66950a3558b\") " pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.150506 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.150273    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3f6a673c-afbc-441f-a108-f66950a3558b-trusted-ca\") pod \"console-operator-76dff48d59-kshzv\" (UID: \"3f6a673c-afbc-441f-a108-f66950a3558b\") " pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.250599 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.250564    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3f6a673c-afbc-441f-a108-f66950a3558b-config\") pod \"console-operator-76dff48d59-kshzv\" (UID: \"3f6a673c-afbc-441f-a108-f66950a3558b\") " pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.250810 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.250661    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3f6a673c-afbc-441f-a108-f66950a3558b-trusted-ca\") pod \"console-operator-76dff48d59-kshzv\" (UID: \"3f6a673c-afbc-441f-a108-f66950a3558b\") " pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.250810 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.250696    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3f6a673c-afbc-441f-a108-f66950a3558b-serving-cert\") pod \"console-operator-76dff48d59-kshzv\" (UID: \"3f6a673c-afbc-441f-a108-f66950a3558b\") " pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.250810 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.250722    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pnrxf\" (UniqueName: \"kubernetes.io/projected/3f6a673c-afbc-441f-a108-f66950a3558b-kube-api-access-pnrxf\") pod \"console-operator-76dff48d59-kshzv\" (UID: \"3f6a673c-afbc-441f-a108-f66950a3558b\") " pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.251245 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.251221    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3f6a673c-afbc-441f-a108-f66950a3558b-config\") pod \"console-operator-76dff48d59-kshzv\" (UID: \"3f6a673c-afbc-441f-a108-f66950a3558b\") " pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.251548 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.251527    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3f6a673c-afbc-441f-a108-f66950a3558b-trusted-ca\") pod \"console-operator-76dff48d59-kshzv\" (UID: \"3f6a673c-afbc-441f-a108-f66950a3558b\") " pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.252975 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.252956    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3f6a673c-afbc-441f-a108-f66950a3558b-serving-cert\") pod \"console-operator-76dff48d59-kshzv\" (UID: \"3f6a673c-afbc-441f-a108-f66950a3558b\") " pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.258978 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.258949    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnrxf\" (UniqueName: \"kubernetes.io/projected/3f6a673c-afbc-441f-a108-f66950a3558b-kube-api-access-pnrxf\") pod \"console-operator-76dff48d59-kshzv\" (UID: \"3f6a673c-afbc-441f-a108-f66950a3558b\") " pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.356317 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.356288    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:04.857109 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:04.857065    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls\") pod \"cluster-samples-operator-697587599-l24d8\" (UID: \"45953cd7-31ed-4062-9c3a-f45873502c85\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"
May 28 18:48:04.857297 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:04.857233    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 18:48:04.857370 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:04.857321    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls podName:45953cd7-31ed-4062-9c3a-f45873502c85 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:06.857298857 +0000 UTC m=+71.116521006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls") pod "cluster-samples-operator-697587599-l24d8" (UID: "45953cd7-31ed-4062-9c3a-f45873502c85") : secret "samples-operator-tls" not found
May 28 18:48:05.042035 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.042002    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"]
May 28 18:48:05.045500 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.045470    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"
May 28 18:48:05.048070 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.048045    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
May 28 18:48:05.048193 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.048053    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
May 28 18:48:05.049157 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.049136    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
May 28 18:48:05.049270 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.049136    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fvxqq\""
May 28 18:48:05.049270 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.049180    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
May 28 18:48:05.053840 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.053558    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"]
May 28 18:48:05.147806 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.147770    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"]
May 28 18:48:05.149849 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.149822    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"
May 28 18:48:05.154113 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.154079    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
May 28 18:48:05.155575 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.155554    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
May 28 18:48:05.155851 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.155823    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
May 28 18:48:05.156319 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.156277    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-d29lk\""
May 28 18:48:05.156319 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.156295    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
May 28 18:48:05.160174 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.160140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt772\" (UniqueName: \"kubernetes.io/projected/4f285834-d577-4e96-b077-8d242366862e-kube-api-access-xt772\") pod \"kube-storage-version-migrator-operator-565644f49-h85g9\" (UID: \"4f285834-d577-4e96-b077-8d242366862e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"
May 28 18:48:05.160270 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.160228    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f285834-d577-4e96-b077-8d242366862e-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-h85g9\" (UID: \"4f285834-d577-4e96-b077-8d242366862e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"
May 28 18:48:05.160270 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.160257    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f285834-d577-4e96-b077-8d242366862e-config\") pod \"kube-storage-version-migrator-operator-565644f49-h85g9\" (UID: \"4f285834-d577-4e96-b077-8d242366862e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"
May 28 18:48:05.173277 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.173248    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"]
May 28 18:48:05.260708 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.260677    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eedbec9e-ba9d-464c-91be-9f9d8e3688ae-config\") pod \"service-ca-operator-57b5b68b8-lngkn\" (UID: \"eedbec9e-ba9d-464c-91be-9f9d8e3688ae\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"
May 28 18:48:05.260883 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.260764    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eedbec9e-ba9d-464c-91be-9f9d8e3688ae-serving-cert\") pod \"service-ca-operator-57b5b68b8-lngkn\" (UID: \"eedbec9e-ba9d-464c-91be-9f9d8e3688ae\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"
May 28 18:48:05.260883 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.260820    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xt772\" (UniqueName: \"kubernetes.io/projected/4f285834-d577-4e96-b077-8d242366862e-kube-api-access-xt772\") pod \"kube-storage-version-migrator-operator-565644f49-h85g9\" (UID: \"4f285834-d577-4e96-b077-8d242366862e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"
May 28 18:48:05.260883 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.260878    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f285834-d577-4e96-b077-8d242366862e-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-h85g9\" (UID: \"4f285834-d577-4e96-b077-8d242366862e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"
May 28 18:48:05.261042 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.260901    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f285834-d577-4e96-b077-8d242366862e-config\") pod \"kube-storage-version-migrator-operator-565644f49-h85g9\" (UID: \"4f285834-d577-4e96-b077-8d242366862e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"
May 28 18:48:05.261042 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.260954    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5765\" (UniqueName: \"kubernetes.io/projected/eedbec9e-ba9d-464c-91be-9f9d8e3688ae-kube-api-access-t5765\") pod \"service-ca-operator-57b5b68b8-lngkn\" (UID: \"eedbec9e-ba9d-464c-91be-9f9d8e3688ae\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"
May 28 18:48:05.261468 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.261445    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f285834-d577-4e96-b077-8d242366862e-config\") pod \"kube-storage-version-migrator-operator-565644f49-h85g9\" (UID: \"4f285834-d577-4e96-b077-8d242366862e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"
May 28 18:48:05.264924 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.264901    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f285834-d577-4e96-b077-8d242366862e-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-h85g9\" (UID: \"4f285834-d577-4e96-b077-8d242366862e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"
May 28 18:48:05.270657 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.270574    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt772\" (UniqueName: \"kubernetes.io/projected/4f285834-d577-4e96-b077-8d242366862e-kube-api-access-xt772\") pod \"kube-storage-version-migrator-operator-565644f49-h85g9\" (UID: \"4f285834-d577-4e96-b077-8d242366862e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"
May 28 18:48:05.356221 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.356198    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"
May 28 18:48:05.362179 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.362084    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t5765\" (UniqueName: \"kubernetes.io/projected/eedbec9e-ba9d-464c-91be-9f9d8e3688ae-kube-api-access-t5765\") pod \"service-ca-operator-57b5b68b8-lngkn\" (UID: \"eedbec9e-ba9d-464c-91be-9f9d8e3688ae\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"
May 28 18:48:05.362179 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.362163    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eedbec9e-ba9d-464c-91be-9f9d8e3688ae-config\") pod \"service-ca-operator-57b5b68b8-lngkn\" (UID: \"eedbec9e-ba9d-464c-91be-9f9d8e3688ae\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"
May 28 18:48:05.362406 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.362223    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eedbec9e-ba9d-464c-91be-9f9d8e3688ae-serving-cert\") pod \"service-ca-operator-57b5b68b8-lngkn\" (UID: \"eedbec9e-ba9d-464c-91be-9f9d8e3688ae\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"
May 28 18:48:05.362895 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.362838    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eedbec9e-ba9d-464c-91be-9f9d8e3688ae-config\") pod \"service-ca-operator-57b5b68b8-lngkn\" (UID: \"eedbec9e-ba9d-464c-91be-9f9d8e3688ae\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"
May 28 18:48:05.364446 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.364425    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eedbec9e-ba9d-464c-91be-9f9d8e3688ae-serving-cert\") pod \"service-ca-operator-57b5b68b8-lngkn\" (UID: \"eedbec9e-ba9d-464c-91be-9f9d8e3688ae\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"
May 28 18:48:05.371299 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.371276    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5765\" (UniqueName: \"kubernetes.io/projected/eedbec9e-ba9d-464c-91be-9f9d8e3688ae-kube-api-access-t5765\") pod \"service-ca-operator-57b5b68b8-lngkn\" (UID: \"eedbec9e-ba9d-464c-91be-9f9d8e3688ae\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"
May 28 18:48:05.400210 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.400185    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-kshzv"]
May 28 18:48:05.402481 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:05.402452    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6a673c_afbc_441f_a108_f66950a3558b.slice/crio-2286759e969e507e5afd71547b1ef434c1a0d112bb9b1750d886e62a6a2cffbd WatchSource:0}: Error finding container 2286759e969e507e5afd71547b1ef434c1a0d112bb9b1750d886e62a6a2cffbd: Status 404 returned error can't find the container with id 2286759e969e507e5afd71547b1ef434c1a0d112bb9b1750d886e62a6a2cffbd
May 28 18:48:05.460997 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.460966    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"
May 28 18:48:05.477547 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.477519    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9"]
May 28 18:48:05.480398 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:05.480370    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f285834_d577_4e96_b077_8d242366862e.slice/crio-1ab089e899173bd2920040ded6b1a72fdc95ca3f0c6c6d637ff3cf2e0f56f861 WatchSource:0}: Error finding container 1ab089e899173bd2920040ded6b1a72fdc95ca3f0c6c6d637ff3cf2e0f56f861: Status 404 returned error can't find the container with id 1ab089e899173bd2920040ded6b1a72fdc95ca3f0c6c6d637ff3cf2e0f56f861
May 28 18:48:05.579856 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.579819    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn"]
May 28 18:48:05.582543 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:05.582516    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeedbec9e_ba9d_464c_91be_9f9d8e3688ae.slice/crio-17aad59d321c9b2674b2d8153c08136403c22920574f15223faed87c6ff054e1 WatchSource:0}: Error finding container 17aad59d321c9b2674b2d8153c08136403c22920574f15223faed87c6ff054e1: Status 404 returned error can't find the container with id 17aad59d321c9b2674b2d8153c08136403c22920574f15223faed87c6ff054e1
May 28 18:48:05.597139 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.597108    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p24tw" event={"ID":"f70415e6-52d5-4096-9d21-82b7ca994f61","Type":"ContainerStarted","Data":"b645d60f01f6938606191158b45e231ab0c0228a4c29faf60e1b4f1912088e6e"}
May 28 18:48:05.597269 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.597210    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:48:05.598285 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.598261    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn" event={"ID":"eedbec9e-ba9d-464c-91be-9f9d8e3688ae","Type":"ContainerStarted","Data":"17aad59d321c9b2674b2d8153c08136403c22920574f15223faed87c6ff054e1"}
May 28 18:48:05.599181 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.599164    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9" event={"ID":"4f285834-d577-4e96-b077-8d242366862e","Type":"ContainerStarted","Data":"1ab089e899173bd2920040ded6b1a72fdc95ca3f0c6c6d637ff3cf2e0f56f861"}
May 28 18:48:05.600059 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:05.600040    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-kshzv" event={"ID":"3f6a673c-afbc-441f-a108-f66950a3558b","Type":"ContainerStarted","Data":"2286759e969e507e5afd71547b1ef434c1a0d112bb9b1750d886e62a6a2cffbd"}
May 28 18:48:06.372907 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:06.372844    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-p24tw" podStartSLOduration=67.465457682 podStartE2EDuration="1m10.372824869s" podCreationTimestamp="2026-05-28 18:46:56 +0000 UTC" firstStartedPulling="2026-05-28 18:48:02.403784363 +0000 UTC m=+66.663006510" lastFinishedPulling="2026-05-28 18:48:05.311151552 +0000 UTC m=+69.570373697" observedRunningTime="2026-05-28 18:48:05.612357936 +0000 UTC m=+69.871580099" watchObservedRunningTime="2026-05-28 18:48:06.372824869 +0000 UTC m=+70.632047037"
May 28 18:48:06.873975 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:06.873929    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls\") pod \"cluster-samples-operator-697587599-l24d8\" (UID: \"45953cd7-31ed-4062-9c3a-f45873502c85\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"
May 28 18:48:06.874229 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:06.874070    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 18:48:06.874229 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:06.874147    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls podName:45953cd7-31ed-4062-9c3a-f45873502c85 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:10.874131918 +0000 UTC m=+75.133354066 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls") pod "cluster-samples-operator-697587599-l24d8" (UID: "45953cd7-31ed-4062-9c3a-f45873502c85") : secret "samples-operator-tls" not found
May 28 18:48:08.608567 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:08.608446    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn" event={"ID":"eedbec9e-ba9d-464c-91be-9f9d8e3688ae","Type":"ContainerStarted","Data":"9470b0542e0d0bef2d6282a8d4b126f48889cbc66bc5f29226545d4c1648086e"}
May 28 18:48:08.609804 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:08.609765    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9" event={"ID":"4f285834-d577-4e96-b077-8d242366862e","Type":"ContainerStarted","Data":"a8b153894155bfb33b4052acffb6e9d22ebc082aa8a35b574770d14c0b917e04"}
May 28 18:48:08.611186 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:08.611168    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/0.log"
May 28 18:48:08.611325 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:08.611200    2577 generic.go:358] "Generic (PLEG): container finished" podID="3f6a673c-afbc-441f-a108-f66950a3558b" containerID="c5649122a9dfb692ae80791c16e8c144e0130543155f99b0e776c5bf84a270ed" exitCode=255
May 28 18:48:08.611325 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:08.611226    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-kshzv" event={"ID":"3f6a673c-afbc-441f-a108-f66950a3558b","Type":"ContainerDied","Data":"c5649122a9dfb692ae80791c16e8c144e0130543155f99b0e776c5bf84a270ed"}
May 28 18:48:08.611472 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:08.611456    2577 scope.go:117] "RemoveContainer" containerID="c5649122a9dfb692ae80791c16e8c144e0130543155f99b0e776c5bf84a270ed"
May 28 18:48:08.627213 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:08.626762    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn" podStartSLOduration=0.906168168 podStartE2EDuration="3.62674639s" podCreationTimestamp="2026-05-28 18:48:05 +0000 UTC" firstStartedPulling="2026-05-28 18:48:05.584366606 +0000 UTC m=+69.843588752" lastFinishedPulling="2026-05-28 18:48:08.304944827 +0000 UTC m=+72.564166974" observedRunningTime="2026-05-28 18:48:08.625989332 +0000 UTC m=+72.885211499" watchObservedRunningTime="2026-05-28 18:48:08.62674639 +0000 UTC m=+72.885968557"
May 28 18:48:08.642747 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:08.642705    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9" podStartSLOduration=0.820168805 podStartE2EDuration="3.642690711s" podCreationTimestamp="2026-05-28 18:48:05 +0000 UTC" firstStartedPulling="2026-05-28 18:48:05.482217388 +0000 UTC m=+69.741439533" lastFinishedPulling="2026-05-28 18:48:08.304739296 +0000 UTC m=+72.563961439" observedRunningTime="2026-05-28 18:48:08.641905652 +0000 UTC m=+72.901127819" watchObservedRunningTime="2026-05-28 18:48:08.642690711 +0000 UTC m=+72.901912876"
May 28 18:48:09.615788 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:09.615713    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 18:48:09.616212 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:09.616068    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/0.log"
May 28 18:48:09.616212 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:09.616099    2577 generic.go:358] "Generic (PLEG): container finished" podID="3f6a673c-afbc-441f-a108-f66950a3558b" containerID="c0ea853193949135f7beb93004c7964f8b81c6ec277e671743365856d8f26b35" exitCode=255
May 28 18:48:09.616212 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:09.616190    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-kshzv" event={"ID":"3f6a673c-afbc-441f-a108-f66950a3558b","Type":"ContainerDied","Data":"c0ea853193949135f7beb93004c7964f8b81c6ec277e671743365856d8f26b35"}
May 28 18:48:09.616369 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:09.616227    2577 scope.go:117] "RemoveContainer" containerID="c5649122a9dfb692ae80791c16e8c144e0130543155f99b0e776c5bf84a270ed"
May 28 18:48:09.616458 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:09.616440    2577 scope.go:117] "RemoveContainer" containerID="c0ea853193949135f7beb93004c7964f8b81c6ec277e671743365856d8f26b35"
May 28 18:48:09.616683 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:09.616666    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-kshzv_openshift-console-operator(3f6a673c-afbc-441f-a108-f66950a3558b)\"" pod="openshift-console-operator/console-operator-76dff48d59-kshzv" podUID="3f6a673c-afbc-441f-a108-f66950a3558b"
May 28 18:48:10.605010 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.604980    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-qqsdr"]
May 28 18:48:10.607798 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.607774    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qqsdr"
May 28 18:48:10.610427 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.610402    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
May 28 18:48:10.611405 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.611380    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
May 28 18:48:10.611405 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.611389    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-t5wd8\""
May 28 18:48:10.618626 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.615963    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-qqsdr"]
May 28 18:48:10.622909 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.622879    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 18:48:10.623219 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.623204    2577 scope.go:117] "RemoveContainer" containerID="c0ea853193949135f7beb93004c7964f8b81c6ec277e671743365856d8f26b35"
May 28 18:48:10.623378 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:10.623361    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-kshzv_openshift-console-operator(3f6a673c-afbc-441f-a108-f66950a3558b)\"" pod="openshift-console-operator/console-operator-76dff48d59-kshzv" podUID="3f6a673c-afbc-441f-a108-f66950a3558b"
May 28 18:48:10.707449 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.707416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtpdv\" (UniqueName: \"kubernetes.io/projected/df63be57-56fd-4e9a-b2a2-f16efc2ccf4d-kube-api-access-xtpdv\") pod \"migrator-b985f5d89-qqsdr\" (UID: \"df63be57-56fd-4e9a-b2a2-f16efc2ccf4d\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qqsdr"
May 28 18:48:10.808867 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.808824    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xtpdv\" (UniqueName: \"kubernetes.io/projected/df63be57-56fd-4e9a-b2a2-f16efc2ccf4d-kube-api-access-xtpdv\") pod \"migrator-b985f5d89-qqsdr\" (UID: \"df63be57-56fd-4e9a-b2a2-f16efc2ccf4d\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qqsdr"
May 28 18:48:10.809026 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.808875    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:48:10.809026 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:10.808969    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 18:48:10.809026 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:10.809021    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls podName:932039c7-8302-443b-9af2-30c993764ffa nodeName:}" failed. No retries permitted until 2026-05-28 18:48:26.809008685 +0000 UTC m=+91.068230829 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-bk4zh" (UID: "932039c7-8302-443b-9af2-30c993764ffa") : secret "cluster-monitoring-operator-tls" not found
May 28 18:48:10.824654 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.824604    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtpdv\" (UniqueName: \"kubernetes.io/projected/df63be57-56fd-4e9a-b2a2-f16efc2ccf4d-kube-api-access-xtpdv\") pod \"migrator-b985f5d89-qqsdr\" (UID: \"df63be57-56fd-4e9a-b2a2-f16efc2ccf4d\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qqsdr"
May 28 18:48:10.910044 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.910008    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls\") pod \"cluster-samples-operator-697587599-l24d8\" (UID: \"45953cd7-31ed-4062-9c3a-f45873502c85\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"
May 28 18:48:10.910211 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:10.910124    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 18:48:10.910211 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:10.910177    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls podName:45953cd7-31ed-4062-9c3a-f45873502c85 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:18.910164198 +0000 UTC m=+83.169386341 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls") pod "cluster-samples-operator-697587599-l24d8" (UID: "45953cd7-31ed-4062-9c3a-f45873502c85") : secret "samples-operator-tls" not found
May 28 18:48:10.919856 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:10.919834    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qqsdr"
May 28 18:48:11.035946 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:11.035843    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-qqsdr"]
May 28 18:48:11.038339 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:11.038298    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf63be57_56fd_4e9a_b2a2_f16efc2ccf4d.slice/crio-b4f2f83dd26a4a3209065529db4258ae06d8e9e67fe1889e458f22f97555ea1a WatchSource:0}: Error finding container b4f2f83dd26a4a3209065529db4258ae06d8e9e67fe1889e458f22f97555ea1a: Status 404 returned error can't find the container with id b4f2f83dd26a4a3209065529db4258ae06d8e9e67fe1889e458f22f97555ea1a
May 28 18:48:11.626824 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:11.626782    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qqsdr" event={"ID":"df63be57-56fd-4e9a-b2a2-f16efc2ccf4d","Type":"ContainerStarted","Data":"b4f2f83dd26a4a3209065529db4258ae06d8e9e67fe1889e458f22f97555ea1a"}
May 28 18:48:12.350338 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.350301    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6f9td"]
May 28 18:48:12.352688 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.352658    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.355009 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.354987    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 18:48:12.355149 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.355118    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9kchc\""
May 28 18:48:12.355296 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.355276    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 18:48:12.363439 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.363415    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6f9td"]
May 28 18:48:12.422864 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.422828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eff66dae-0031-4fe2-907d-85f57640b231-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.423040 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.422958    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.423040 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.423013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2lrf\" (UniqueName: \"kubernetes.io/projected/eff66dae-0031-4fe2-907d-85f57640b231-kube-api-access-q2lrf\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.423213 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.423081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eff66dae-0031-4fe2-907d-85f57640b231-crio-socket\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.423213 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.423135    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eff66dae-0031-4fe2-907d-85f57640b231-data-volume\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.524123 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.524088    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eff66dae-0031-4fe2-907d-85f57640b231-data-volume\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.524123 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.524127    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eff66dae-0031-4fe2-907d-85f57640b231-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.524370 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.524351    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.524429 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.524409    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q2lrf\" (UniqueName: \"kubernetes.io/projected/eff66dae-0031-4fe2-907d-85f57640b231-kube-api-access-q2lrf\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.524482 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.524463    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eff66dae-0031-4fe2-907d-85f57640b231-crio-socket\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.524482 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.524467    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eff66dae-0031-4fe2-907d-85f57640b231-data-volume\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.524555 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:12.524488    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
May 28 18:48:12.524642 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.524622    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eff66dae-0031-4fe2-907d-85f57640b231-crio-socket\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.524692 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:12.524646    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls podName:eff66dae-0031-4fe2-907d-85f57640b231 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:13.024627705 +0000 UTC m=+77.283849863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6f9td" (UID: "eff66dae-0031-4fe2-907d-85f57640b231") : secret "insights-runtime-extractor-tls" not found
May 28 18:48:12.524733 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.524710    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eff66dae-0031-4fe2-907d-85f57640b231-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.535888 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.535864    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2lrf\" (UniqueName: \"kubernetes.io/projected/eff66dae-0031-4fe2-907d-85f57640b231-kube-api-access-q2lrf\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:12.631296 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:12.631267    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qqsdr" event={"ID":"df63be57-56fd-4e9a-b2a2-f16efc2ccf4d","Type":"ContainerStarted","Data":"2e951eff18f46961988d2459cfd8326de542a3a2893871438763d1467e84aac1"}
May 28 18:48:13.029241 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:13.029194    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:13.029413 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:13.029348    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
May 28 18:48:13.029458 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:13.029420    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls podName:eff66dae-0031-4fe2-907d-85f57640b231 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:14.029403852 +0000 UTC m=+78.288625997 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6f9td" (UID: "eff66dae-0031-4fe2-907d-85f57640b231") : secret "insights-runtime-extractor-tls" not found
May 28 18:48:13.634985 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:13.634951    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qqsdr" event={"ID":"df63be57-56fd-4e9a-b2a2-f16efc2ccf4d","Type":"ContainerStarted","Data":"f20a4f40c5fdc7baf33930e5fafda0e4f4c20376c6d1782cb56e3f661b02289a"}
May 28 18:48:13.658466 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:13.658416    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qqsdr" podStartSLOduration=2.16054993 podStartE2EDuration="3.658400622s" podCreationTimestamp="2026-05-28 18:48:10 +0000 UTC" firstStartedPulling="2026-05-28 18:48:11.040239073 +0000 UTC m=+75.299461221" lastFinishedPulling="2026-05-28 18:48:12.538089766 +0000 UTC m=+76.797311913" observedRunningTime="2026-05-28 18:48:13.658029875 +0000 UTC m=+77.917252041" watchObservedRunningTime="2026-05-28 18:48:13.658400622 +0000 UTC m=+77.917622789"
May 28 18:48:14.037741 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:14.037642    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:14.037918 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:14.037826    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
May 28 18:48:14.037918 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:14.037908    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls podName:eff66dae-0031-4fe2-907d-85f57640b231 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:16.037889791 +0000 UTC m=+80.297111934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6f9td" (UID: "eff66dae-0031-4fe2-907d-85f57640b231") : secret "insights-runtime-extractor-tls" not found
May 28 18:48:14.356850 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:14.356764    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:14.356850 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:14.356801    2577 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:14.357196 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:14.357180    2577 scope.go:117] "RemoveContainer" containerID="c0ea853193949135f7beb93004c7964f8b81c6ec277e671743365856d8f26b35"
May 28 18:48:14.357393 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:14.357374    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-kshzv_openshift-console-operator(3f6a673c-afbc-441f-a108-f66950a3558b)\"" pod="openshift-console-operator/console-operator-76dff48d59-kshzv" podUID="3f6a673c-afbc-441f-a108-f66950a3558b"
May 28 18:48:16.056222 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:16.056165    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:16.056664 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:16.056311    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
May 28 18:48:16.056664 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:16.056380    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls podName:eff66dae-0031-4fe2-907d-85f57640b231 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:20.056365671 +0000 UTC m=+84.315587815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6f9td" (UID: "eff66dae-0031-4fe2-907d-85f57640b231") : secret "insights-runtime-extractor-tls" not found
May 28 18:48:18.984191 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:18.984148    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls\") pod \"cluster-samples-operator-697587599-l24d8\" (UID: \"45953cd7-31ed-4062-9c3a-f45873502c85\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"
May 28 18:48:18.987097 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:18.987065    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45953cd7-31ed-4062-9c3a-f45873502c85-samples-operator-tls\") pod \"cluster-samples-operator-697587599-l24d8\" (UID: \"45953cd7-31ed-4062-9c3a-f45873502c85\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"
May 28 18:48:19.050982 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:19.050936    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"
May 28 18:48:19.177517 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:19.177485    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8"]
May 28 18:48:19.649797 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:19.649760    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8" event={"ID":"45953cd7-31ed-4062-9c3a-f45873502c85","Type":"ContainerStarted","Data":"e3083f46e8ac9cca1f18eac49e397ae879d703cd9ea56c5894c932aa60383c8e"}
May 28 18:48:20.092226 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:20.092130    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:20.092584 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:20.092275    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
May 28 18:48:20.092584 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:20.092342    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls podName:eff66dae-0031-4fe2-907d-85f57640b231 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:28.092326986 +0000 UTC m=+92.351549129 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6f9td" (UID: "eff66dae-0031-4fe2-907d-85f57640b231") : secret "insights-runtime-extractor-tls" not found
May 28 18:48:21.658123 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:21.658082    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8" event={"ID":"45953cd7-31ed-4062-9c3a-f45873502c85","Type":"ContainerStarted","Data":"6995d1e57a842bdbad5bec426fbda2e4816b29405d39137896fc5ce117c94f85"}
May 28 18:48:21.658123 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:21.658125    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8" event={"ID":"45953cd7-31ed-4062-9c3a-f45873502c85","Type":"ContainerStarted","Data":"5199c4f4345a20f502d26433906fb57899bf0e7ec0cf0a5d71fa07dbd179caf9"}
May 28 18:48:21.674073 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:21.674020    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-l24d8" podStartSLOduration=16.691681971 podStartE2EDuration="18.674006298s" podCreationTimestamp="2026-05-28 18:48:03 +0000 UTC" firstStartedPulling="2026-05-28 18:48:19.215465441 +0000 UTC m=+83.474687584" lastFinishedPulling="2026-05-28 18:48:21.197789768 +0000 UTC m=+85.457011911" observedRunningTime="2026-05-28 18:48:21.673254494 +0000 UTC m=+85.932476661" watchObservedRunningTime="2026-05-28 18:48:21.674006298 +0000 UTC m=+85.933228463"
May 28 18:48:25.354380 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:25.354350    2577 scope.go:117] "RemoveContainer" containerID="c0ea853193949135f7beb93004c7964f8b81c6ec277e671743365856d8f26b35"
May 28 18:48:25.670058 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:25.670026    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 18:48:25.670311 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:25.670108    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-kshzv" event={"ID":"3f6a673c-afbc-441f-a108-f66950a3558b","Type":"ContainerStarted","Data":"95302a3e872f1f6d5cc73c14a7ff9f39b50493555db43632b8af589a0b8ca1bc"}
May 28 18:48:25.670391 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:25.670365    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:25.687295 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:25.687246    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-76dff48d59-kshzv" podStartSLOduration=18.792659993 podStartE2EDuration="21.687231118s" podCreationTimestamp="2026-05-28 18:48:04 +0000 UTC" firstStartedPulling="2026-05-28 18:48:05.404705554 +0000 UTC m=+69.663927710" lastFinishedPulling="2026-05-28 18:48:08.299276687 +0000 UTC m=+72.558498835" observedRunningTime="2026-05-28 18:48:25.686791183 +0000 UTC m=+89.946013349" watchObservedRunningTime="2026-05-28 18:48:25.687231118 +0000 UTC m=+89.946453283"
May 28 18:48:25.970192 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:25.970101    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-76dff48d59-kshzv"
May 28 18:48:26.848044 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:26.848007    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:48:26.850557 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:26.850528    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/932039c7-8302-443b-9af2-30c993764ffa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-bk4zh\" (UID: \"932039c7-8302-443b-9af2-30c993764ffa\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:48:26.862765 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:26.862745    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vqdtm\""
May 28 18:48:26.871186 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:26.871172    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"
May 28 18:48:26.989256 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:26.989218    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh"]
May 28 18:48:26.993183 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:26.993153    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod932039c7_8302_443b_9af2_30c993764ffa.slice/crio-8ac3b0f7cb7831bcb41ed1b4d8d26c15d1e1b18f2c41a2fdb3db910da027d009 WatchSource:0}: Error finding container 8ac3b0f7cb7831bcb41ed1b4d8d26c15d1e1b18f2c41a2fdb3db910da027d009: Status 404 returned error can't find the container with id 8ac3b0f7cb7831bcb41ed1b4d8d26c15d1e1b18f2c41a2fdb3db910da027d009
May 28 18:48:27.678804 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:27.678754    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh" event={"ID":"932039c7-8302-443b-9af2-30c993764ffa","Type":"ContainerStarted","Data":"8ac3b0f7cb7831bcb41ed1b4d8d26c15d1e1b18f2c41a2fdb3db910da027d009"}
May 28 18:48:28.158196 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:28.158160    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:28.160658 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:28.160630    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eff66dae-0031-4fe2-907d-85f57640b231-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6f9td\" (UID: \"eff66dae-0031-4fe2-907d-85f57640b231\") " pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:28.264130 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:28.264099    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6f9td"
May 28 18:48:28.393820 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:28.393787    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6f9td"]
May 28 18:48:28.396805 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:28.396782    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff66dae_0031_4fe2_907d_85f57640b231.slice/crio-d12e6e1da81f8e17c5b149768ae43d6a8818d37d0cee025698de42451cd2c659 WatchSource:0}: Error finding container d12e6e1da81f8e17c5b149768ae43d6a8818d37d0cee025698de42451cd2c659: Status 404 returned error can't find the container with id d12e6e1da81f8e17c5b149768ae43d6a8818d37d0cee025698de42451cd2c659
May 28 18:48:28.683451 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:28.683363    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6f9td" event={"ID":"eff66dae-0031-4fe2-907d-85f57640b231","Type":"ContainerStarted","Data":"0470ea9803eac9dbf1c4c6923de42bafbec4c094464273ec97bd7416037461cf"}
May 28 18:48:28.683451 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:28.683401    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6f9td" event={"ID":"eff66dae-0031-4fe2-907d-85f57640b231","Type":"ContainerStarted","Data":"d12e6e1da81f8e17c5b149768ae43d6a8818d37d0cee025698de42451cd2c659"}
May 28 18:48:29.687727 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:29.687690    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6f9td" event={"ID":"eff66dae-0031-4fe2-907d-85f57640b231","Type":"ContainerStarted","Data":"b29aa9e56ab1d0fbb61301f161f141520a4ce44a639e748d726512ccbbfba7ed"}
May 28 18:48:29.688934 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:29.688908    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh" event={"ID":"932039c7-8302-443b-9af2-30c993764ffa","Type":"ContainerStarted","Data":"97bba9c858ef39391d957a78d4159f385ec19ffe7f782ae129821742985ef1df"}
May 28 18:48:29.705031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:29.704990    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-bk4zh" podStartSLOduration=33.488156287 podStartE2EDuration="35.704976831s" podCreationTimestamp="2026-05-28 18:47:54 +0000 UTC" firstStartedPulling="2026-05-28 18:48:26.995186951 +0000 UTC m=+91.254409096" lastFinishedPulling="2026-05-28 18:48:29.212007496 +0000 UTC m=+93.471229640" observedRunningTime="2026-05-28 18:48:29.704542218 +0000 UTC m=+93.963764383" watchObservedRunningTime="2026-05-28 18:48:29.704976831 +0000 UTC m=+93.964199046"
May 28 18:48:30.992583 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:30.992541    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4"]
May 28 18:48:30.994867 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:30.994843    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4"
May 28 18:48:30.998565 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:30.998540    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-82kb5\""
May 28 18:48:30.998565 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:30.998554    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
May 28 18:48:31.005824 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.005800    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4"]
May 28 18:48:31.042715 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.042687    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-676cdccb6-g86th"]
May 28 18:48:31.044527 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.044512    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.047062 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.047041    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 18:48:31.047181 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.047123    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 18:48:31.047222 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.047180    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 18:48:31.047521 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.047500    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-79vb8\""
May 28 18:48:31.051869 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.051847    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 18:48:31.056698 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.056671    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-676cdccb6-g86th"]
May 28 18:48:31.084941 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.084908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d80cb94e-6a08-46a3-8865-57b8b7029b3c-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4\" (UID: \"d80cb94e-6a08-46a3-8865-57b8b7029b3c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4"
May 28 18:48:31.088468 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.088440    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-57bd96b888-t4dk9"]
May 28 18:48:31.090533 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.090513    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-t4dk9"
May 28 18:48:31.093876 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.093856    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-f5r5h\""
May 28 18:48:31.093975 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.093933    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 28 18:48:31.096483 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.096463    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 28 18:48:31.109454 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.109432    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-t4dk9"]
May 28 18:48:31.185804 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.185775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdcf9\" (UniqueName: \"kubernetes.io/projected/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-kube-api-access-hdcf9\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.185804 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.185814    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-trusted-ca\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.186010 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.185832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2dfs\" (UniqueName: \"kubernetes.io/projected/addb0542-e63c-4184-bbe8-1e21d45a4f68-kube-api-access-m2dfs\") pod \"downloads-57bd96b888-t4dk9\" (UID: \"addb0542-e63c-4184-bbe8-1e21d45a4f68\") " pod="openshift-console/downloads-57bd96b888-t4dk9"
May 28 18:48:31.186010 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.185861    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d80cb94e-6a08-46a3-8865-57b8b7029b3c-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4\" (UID: \"d80cb94e-6a08-46a3-8865-57b8b7029b3c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4"
May 28 18:48:31.186010 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.185904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-ca-trust-extracted\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.186010 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.185930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-installation-pull-secrets\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.186010 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.185988    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-registry-tls\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.186161 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.186015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-image-registry-private-configuration\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.186161 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.186049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-bound-sa-token\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.186161 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.186115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-registry-certificates\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.188233 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.188206    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d80cb94e-6a08-46a3-8865-57b8b7029b3c-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4\" (UID: \"d80cb94e-6a08-46a3-8865-57b8b7029b3c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4"
May 28 18:48:31.286747 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.286698    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-registry-certificates\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.286747 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.286749    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hdcf9\" (UniqueName: \"kubernetes.io/projected/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-kube-api-access-hdcf9\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.286965 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.286775    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-trusted-ca\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.286965 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.286817    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m2dfs\" (UniqueName: \"kubernetes.io/projected/addb0542-e63c-4184-bbe8-1e21d45a4f68-kube-api-access-m2dfs\") pod \"downloads-57bd96b888-t4dk9\" (UID: \"addb0542-e63c-4184-bbe8-1e21d45a4f68\") " pod="openshift-console/downloads-57bd96b888-t4dk9"
May 28 18:48:31.287038 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.286963    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-ca-trust-extracted\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.287038 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.287029    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-installation-pull-secrets\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.287107 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.287077    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-registry-tls\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.287153 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.287109    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-image-registry-private-configuration\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.287205 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.287180    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-bound-sa-token\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.287389 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.287368    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-ca-trust-extracted\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.287615 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.287592    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-registry-certificates\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.287755 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.287738    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-trusted-ca\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.289558 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.289537    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-registry-tls\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.289558 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.289560    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-image-registry-private-configuration\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.289708 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.289633    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-installation-pull-secrets\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.295169 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.295146    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdcf9\" (UniqueName: \"kubernetes.io/projected/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-kube-api-access-hdcf9\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.295284 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.295247    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cea4760-a4e5-43f5-9a87-f1f2094eaddd-bound-sa-token\") pod \"image-registry-676cdccb6-g86th\" (UID: \"4cea4760-a4e5-43f5-9a87-f1f2094eaddd\") " pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.295630 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.295597    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2dfs\" (UniqueName: \"kubernetes.io/projected/addb0542-e63c-4184-bbe8-1e21d45a4f68-kube-api-access-m2dfs\") pod \"downloads-57bd96b888-t4dk9\" (UID: \"addb0542-e63c-4184-bbe8-1e21d45a4f68\") " pod="openshift-console/downloads-57bd96b888-t4dk9"
May 28 18:48:31.306198 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.306171    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4"
May 28 18:48:31.354486 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.354462    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.400262 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.400227    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-t4dk9"
May 28 18:48:31.443493 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.443389    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4"]
May 28 18:48:31.450088 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:31.450052    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd80cb94e_6a08_46a3_8865_57b8b7029b3c.slice/crio-bbcb14f0b915e208de489993d168a5792ce382cf3f010adb3ff667fa8d5e3367 WatchSource:0}: Error finding container bbcb14f0b915e208de489993d168a5792ce382cf3f010adb3ff667fa8d5e3367: Status 404 returned error can't find the container with id bbcb14f0b915e208de489993d168a5792ce382cf3f010adb3ff667fa8d5e3367
May 28 18:48:31.512452 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.512416    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-676cdccb6-g86th"]
May 28 18:48:31.515570 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:31.515533    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cea4760_a4e5_43f5_9a87_f1f2094eaddd.slice/crio-034b25609803696ac9d1a9e3344af164904bb997ff322ff4981f49eed1ad27bc WatchSource:0}: Error finding container 034b25609803696ac9d1a9e3344af164904bb997ff322ff4981f49eed1ad27bc: Status 404 returned error can't find the container with id 034b25609803696ac9d1a9e3344af164904bb997ff322ff4981f49eed1ad27bc
May 28 18:48:31.553271 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.553083    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-t4dk9"]
May 28 18:48:31.555901 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:31.555874    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaddb0542_e63c_4184_bbe8_1e21d45a4f68.slice/crio-d3d9df6e2d496119224adab71168e4ee40ced82c5f51e9bed6fe1c025f2ef827 WatchSource:0}: Error finding container d3d9df6e2d496119224adab71168e4ee40ced82c5f51e9bed6fe1c025f2ef827: Status 404 returned error can't find the container with id d3d9df6e2d496119224adab71168e4ee40ced82c5f51e9bed6fe1c025f2ef827
May 28 18:48:31.696830 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.696742    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6f9td" event={"ID":"eff66dae-0031-4fe2-907d-85f57640b231","Type":"ContainerStarted","Data":"a6d19e914c302d82da53508d4d93a26780c7fb765f0ee1456103e7cbac27ebaf"}
May 28 18:48:31.697859 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.697835    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-t4dk9" event={"ID":"addb0542-e63c-4184-bbe8-1e21d45a4f68","Type":"ContainerStarted","Data":"d3d9df6e2d496119224adab71168e4ee40ced82c5f51e9bed6fe1c025f2ef827"}
May 28 18:48:31.699103 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.699081    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676cdccb6-g86th" event={"ID":"4cea4760-a4e5-43f5-9a87-f1f2094eaddd","Type":"ContainerStarted","Data":"d48cd10e9789314ca21a8b464fdcb5d2fad718eee5e3c2f5e72ec7683c10f92a"}
May 28 18:48:31.699212 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.699111    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676cdccb6-g86th" event={"ID":"4cea4760-a4e5-43f5-9a87-f1f2094eaddd","Type":"ContainerStarted","Data":"034b25609803696ac9d1a9e3344af164904bb997ff322ff4981f49eed1ad27bc"}
May 28 18:48:31.699212 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.699190    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:31.700190 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.700171    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4" event={"ID":"d80cb94e-6a08-46a3-8865-57b8b7029b3c","Type":"ContainerStarted","Data":"bbcb14f0b915e208de489993d168a5792ce382cf3f010adb3ff667fa8d5e3367"}
May 28 18:48:31.713912 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.713871    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6f9td" podStartSLOduration=17.304665341 podStartE2EDuration="19.713857021s" podCreationTimestamp="2026-05-28 18:48:12 +0000 UTC" firstStartedPulling="2026-05-28 18:48:28.463985111 +0000 UTC m=+92.723207263" lastFinishedPulling="2026-05-28 18:48:30.873176799 +0000 UTC m=+95.132398943" observedRunningTime="2026-05-28 18:48:31.712698235 +0000 UTC m=+95.971920402" watchObservedRunningTime="2026-05-28 18:48:31.713857021 +0000 UTC m=+95.973079248"
May 28 18:48:31.731543 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:31.731500    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-676cdccb6-g86th" podStartSLOduration=0.731484897 podStartE2EDuration="731.484897ms" podCreationTimestamp="2026-05-28 18:48:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 18:48:31.730648304 +0000 UTC m=+95.989870471" watchObservedRunningTime="2026-05-28 18:48:31.731484897 +0000 UTC m=+95.990707064"
May 28 18:48:33.707914 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:33.707872    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4" event={"ID":"d80cb94e-6a08-46a3-8865-57b8b7029b3c","Type":"ContainerStarted","Data":"6604e03f007f9a9209f2d13487ccc25d9719ca42a934a7206eaf8b144dfecabc"}
May 28 18:48:33.708361 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:33.708092    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4"
May 28 18:48:33.713977 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:33.713951    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4"
May 28 18:48:33.723402 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:33.723350    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4" podStartSLOduration=2.426743212 podStartE2EDuration="3.723332659s" podCreationTimestamp="2026-05-28 18:48:30 +0000 UTC" firstStartedPulling="2026-05-28 18:48:31.452561381 +0000 UTC m=+95.711783525" lastFinishedPulling="2026-05-28 18:48:32.749150812 +0000 UTC m=+97.008372972" observedRunningTime="2026-05-28 18:48:33.722341512 +0000 UTC m=+97.981563680" watchObservedRunningTime="2026-05-28 18:48:33.723332659 +0000 UTC m=+97.982554830"
May 28 18:48:34.012273 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:34.012235    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:48:34.012443 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:34.012285    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert\") pod \"ingress-canary-hmmqn\" (UID: \"74ae6b85-71a4-4b53-a23e-4975b6464338\") " pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:48:34.014894 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:34.014868    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c79a026-b52c-4e1e-9720-1bbe00a9ed38-metrics-tls\") pod \"dns-default-4n69j\" (UID: \"0c79a026-b52c-4e1e-9720-1bbe00a9ed38\") " pod="openshift-dns/dns-default-4n69j"
May 28 18:48:34.015015 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:34.014917    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74ae6b85-71a4-4b53-a23e-4975b6464338-cert\") pod \"ingress-canary-hmmqn\" (UID: \"74ae6b85-71a4-4b53-a23e-4975b6464338\") " pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:48:34.020136 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:34.020115    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zpgsq\""
May 28 18:48:34.028378 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:34.028350    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hmmqn"
May 28 18:48:34.149953 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:34.149916    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hmmqn"]
May 28 18:48:34.153264 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:34.153237    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ae6b85_71a4_4b53_a23e_4975b6464338.slice/crio-ab72af01ba6765ee2c063567b6e86b1f8eaf147ed179c4f4d150a2002543de91 WatchSource:0}: Error finding container ab72af01ba6765ee2c063567b6e86b1f8eaf147ed179c4f4d150a2002543de91: Status 404 returned error can't find the container with id ab72af01ba6765ee2c063567b6e86b1f8eaf147ed179c4f4d150a2002543de91
May 28 18:48:34.305264 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:34.305169    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l78mj\""
May 28 18:48:34.313315 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:34.313289    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4n69j"
May 28 18:48:34.441256 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:34.441224    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4n69j"]
May 28 18:48:34.444426 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:34.444393    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c79a026_b52c_4e1e_9720_1bbe00a9ed38.slice/crio-5f7f988c40e1b76ef033b4be17cbd8615da2a468ece841c796eb745a1a8d13d2 WatchSource:0}: Error finding container 5f7f988c40e1b76ef033b4be17cbd8615da2a468ece841c796eb745a1a8d13d2: Status 404 returned error can't find the container with id 5f7f988c40e1b76ef033b4be17cbd8615da2a468ece841c796eb745a1a8d13d2
May 28 18:48:34.713337 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:34.713287    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hmmqn" event={"ID":"74ae6b85-71a4-4b53-a23e-4975b6464338","Type":"ContainerStarted","Data":"ab72af01ba6765ee2c063567b6e86b1f8eaf147ed179c4f4d150a2002543de91"}
May 28 18:48:34.714517 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:34.714475    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4n69j" event={"ID":"0c79a026-b52c-4e1e-9720-1bbe00a9ed38","Type":"ContainerStarted","Data":"5f7f988c40e1b76ef033b4be17cbd8615da2a468ece841c796eb745a1a8d13d2"}
May 28 18:48:35.466290 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.466256    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-bb69dd9f8-9z72z"]
May 28 18:48:35.469961 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.469933    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.473496 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.473471    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 28 18:48:35.473639 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.473521    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 28 18:48:35.473929 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.473471    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-tdtbf\""
May 28 18:48:35.473929 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.473791    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 28 18:48:35.473929 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.473875    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 28 18:48:35.474138 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.474079    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 28 18:48:35.478827 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.478645    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bb69dd9f8-9z72z"]
May 28 18:48:35.627075 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.627039    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-service-ca\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.627252 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.627107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-serving-cert\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.627252 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.627169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-oauth-config\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.627252 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.627201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-oauth-serving-cert\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.627252 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.627245    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-config\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.627434 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.627265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72qk8\" (UniqueName: \"kubernetes.io/projected/2fa17eb3-d549-44fc-a836-9b12b1a91aee-kube-api-access-72qk8\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.728283 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.728199    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-service-ca\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.728283 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.728270    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-serving-cert\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.728947 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.728310    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-oauth-config\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.728947 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.728348    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-oauth-serving-cert\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.728947 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.728388    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-config\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.728947 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.728414    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-72qk8\" (UniqueName: \"kubernetes.io/projected/2fa17eb3-d549-44fc-a836-9b12b1a91aee-kube-api-access-72qk8\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.729151 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.729023    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-service-ca\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.729151 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.729127    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-oauth-serving-cert\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.729371 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.729330    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-config\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.731834 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.731813    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-serving-cert\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.732604 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.732558    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-oauth-config\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.741100 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.741055    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72qk8\" (UniqueName: \"kubernetes.io/projected/2fa17eb3-d549-44fc-a836-9b12b1a91aee-kube-api-access-72qk8\") pod \"console-bb69dd9f8-9z72z\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") " pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:35.782822 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:35.782780    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:36.606214 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:36.606130    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-p24tw"
May 28 18:48:36.786994 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:36.786945    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bb69dd9f8-9z72z"]
May 28 18:48:36.790916 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:36.790886    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fa17eb3_d549_44fc_a836_9b12b1a91aee.slice/crio-8a4a94b7f4bb4c2554fc4455f78b7bc92553ec227a80ecdd6f54447807d712ac WatchSource:0}: Error finding container 8a4a94b7f4bb4c2554fc4455f78b7bc92553ec227a80ecdd6f54447807d712ac: Status 404 returned error can't find the container with id 8a4a94b7f4bb4c2554fc4455f78b7bc92553ec227a80ecdd6f54447807d712ac
May 28 18:48:37.725440 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:37.725380    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hmmqn" event={"ID":"74ae6b85-71a4-4b53-a23e-4975b6464338","Type":"ContainerStarted","Data":"691807a2ca9c86c9cfc614d5d5f37db44788d7e6c33db2aed02ff05be516ffd4"}
May 28 18:48:37.727678 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:37.727645    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4n69j" event={"ID":"0c79a026-b52c-4e1e-9720-1bbe00a9ed38","Type":"ContainerStarted","Data":"b9b3fd0b61bd0d2e3e8f31a1cc67e918413a7e8f9c63e096e7701ca3ed0aab11"}
May 28 18:48:37.727678 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:37.727678    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4n69j" event={"ID":"0c79a026-b52c-4e1e-9720-1bbe00a9ed38","Type":"ContainerStarted","Data":"89acdac630a91be569bbd960cbeb5783652acc26157e09e31607f981c17b19bf"}
May 28 18:48:37.727882 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:37.727794    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4n69j"
May 28 18:48:37.728986 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:37.728962    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb69dd9f8-9z72z" event={"ID":"2fa17eb3-d549-44fc-a836-9b12b1a91aee","Type":"ContainerStarted","Data":"8a4a94b7f4bb4c2554fc4455f78b7bc92553ec227a80ecdd6f54447807d712ac"}
May 28 18:48:37.743177 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:37.743086    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hmmqn" podStartSLOduration=65.265549324 podStartE2EDuration="1m7.743042795s" podCreationTimestamp="2026-05-28 18:47:30 +0000 UTC" firstStartedPulling="2026-05-28 18:48:34.155884837 +0000 UTC m=+98.415106998" lastFinishedPulling="2026-05-28 18:48:36.633378311 +0000 UTC m=+100.892600469" observedRunningTime="2026-05-28 18:48:37.740372373 +0000 UTC m=+101.999594552" watchObservedRunningTime="2026-05-28 18:48:37.743042795 +0000 UTC m=+102.002264971"
May 28 18:48:37.758003 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:37.757940    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-4n69j" podStartSLOduration=65.567091774 podStartE2EDuration="1m7.757919324s" podCreationTimestamp="2026-05-28 18:47:30 +0000 UTC" firstStartedPulling="2026-05-28 18:48:34.446452802 +0000 UTC m=+98.705674951" lastFinishedPulling="2026-05-28 18:48:36.637280344 +0000 UTC m=+100.896502501" observedRunningTime="2026-05-28 18:48:37.757911624 +0000 UTC m=+102.017133791" watchObservedRunningTime="2026-05-28 18:48:37.757919324 +0000 UTC m=+102.017141494"
May 28 18:48:39.184380 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.184341    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r4d59"]
May 28 18:48:39.207402 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.207370    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.210576 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.210393    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 18:48:39.210576 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.210412    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 18:48:39.210802 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.210714    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 18:48:39.210935 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.210914    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w4rs5\""
May 28 18:48:39.211756 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.211554    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 18:48:39.362310 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.362273    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-tls\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.362492 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.362328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8c10cda-fc2b-487c-8b95-44064737610d-sys\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.362492 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.362429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-wtmp\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.362752 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.362490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f8c10cda-fc2b-487c-8b95-44064737610d-root\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.362752 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.362526    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgnnv\" (UniqueName: \"kubernetes.io/projected/f8c10cda-fc2b-487c-8b95-44064737610d-kube-api-access-rgnnv\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.362752 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.362584    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.362752 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.362689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-textfile\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.362752 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.362718    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8c10cda-fc2b-487c-8b95-44064737610d-metrics-client-ca\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.363031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.362768    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-accelerators-collector-config\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.463714 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.463636    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-wtmp\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.463714 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.463699    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f8c10cda-fc2b-487c-8b95-44064737610d-root\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.463919 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.463729    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rgnnv\" (UniqueName: \"kubernetes.io/projected/f8c10cda-fc2b-487c-8b95-44064737610d-kube-api-access-rgnnv\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.463919 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.463754    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.463919 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.463822    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-wtmp\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.463919 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.463833    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f8c10cda-fc2b-487c-8b95-44064737610d-root\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.464143 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.463934    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-textfile\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.464143 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.463975    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8c10cda-fc2b-487c-8b95-44064737610d-metrics-client-ca\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.464143 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.464005    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-accelerators-collector-config\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.464143 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.464073    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-tls\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.464347 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:39.464155    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 28 18:48:39.464347 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:39.464209    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-tls podName:f8c10cda-fc2b-487c-8b95-44064737610d nodeName:}" failed. No retries permitted until 2026-05-28 18:48:39.964190985 +0000 UTC m=+104.223413129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-tls") pod "node-exporter-r4d59" (UID: "f8c10cda-fc2b-487c-8b95-44064737610d") : secret "node-exporter-tls" not found
May 28 18:48:39.464347 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.464236    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8c10cda-fc2b-487c-8b95-44064737610d-sys\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.464347 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.464299    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-textfile\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.464347 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.464323    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8c10cda-fc2b-487c-8b95-44064737610d-sys\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.464667 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.464624    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8c10cda-fc2b-487c-8b95-44064737610d-metrics-client-ca\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.464857 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.464837    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-accelerators-collector-config\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.466719 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.466687    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.473071 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.473024    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgnnv\" (UniqueName: \"kubernetes.io/projected/f8c10cda-fc2b-487c-8b95-44064737610d-kube-api-access-rgnnv\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.969098 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:39.969064    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-tls\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:39.969269 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:39.969229    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 28 18:48:39.969335 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:39.969309    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-tls podName:f8c10cda-fc2b-487c-8b95-44064737610d nodeName:}" failed. No retries permitted until 2026-05-28 18:48:40.969287946 +0000 UTC m=+105.228510113 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-tls") pod "node-exporter-r4d59" (UID: "f8c10cda-fc2b-487c-8b95-44064737610d") : secret "node-exporter-tls" not found
May 28 18:48:40.740678 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:40.740633    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb69dd9f8-9z72z" event={"ID":"2fa17eb3-d549-44fc-a836-9b12b1a91aee","Type":"ContainerStarted","Data":"42c7e9034f26b38fa193d9c09345bf3ee3cde3d6209def4293a71fa679e80025"}
May 28 18:48:40.758876 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:40.758539    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-bb69dd9f8-9z72z" podStartSLOduration=2.779097249 podStartE2EDuration="5.758521796s" podCreationTimestamp="2026-05-28 18:48:35 +0000 UTC" firstStartedPulling="2026-05-28 18:48:36.793445924 +0000 UTC m=+101.052668068" lastFinishedPulling="2026-05-28 18:48:39.772870458 +0000 UTC m=+104.032092615" observedRunningTime="2026-05-28 18:48:40.756939807 +0000 UTC m=+105.016161995" watchObservedRunningTime="2026-05-28 18:48:40.758521796 +0000 UTC m=+105.017743960"
May 28 18:48:40.978600 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:40.978557    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-tls\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:40.980841 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:40.980813    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f8c10cda-fc2b-487c-8b95-44064737610d-node-exporter-tls\") pod \"node-exporter-r4d59\" (UID: \"f8c10cda-fc2b-487c-8b95-44064737610d\") " pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:41.020762 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:41.020661    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r4d59"
May 28 18:48:41.030036 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:41.030007    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c10cda_fc2b_487c_8b95_44064737610d.slice/crio-d6034bf23261643701a3ab5d34fc4213be01a6d3e3ac616bf39e3b204e6135ca WatchSource:0}: Error finding container d6034bf23261643701a3ab5d34fc4213be01a6d3e3ac616bf39e3b204e6135ca: Status 404 returned error can't find the container with id d6034bf23261643701a3ab5d34fc4213be01a6d3e3ac616bf39e3b204e6135ca
May 28 18:48:41.745478 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:41.745438    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r4d59" event={"ID":"f8c10cda-fc2b-487c-8b95-44064737610d","Type":"ContainerStarted","Data":"d6034bf23261643701a3ab5d34fc4213be01a6d3e3ac616bf39e3b204e6135ca"}
May 28 18:48:42.750114 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:42.750078    2577 generic.go:358] "Generic (PLEG): container finished" podID="f8c10cda-fc2b-487c-8b95-44064737610d" containerID="e2277eab08df6ec2aab627ba2fbf905f282ae0a29688bea347c6cf68242d43ff" exitCode=0
May 28 18:48:42.750114 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:42.750118    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r4d59" event={"ID":"f8c10cda-fc2b-487c-8b95-44064737610d","Type":"ContainerDied","Data":"e2277eab08df6ec2aab627ba2fbf905f282ae0a29688bea347c6cf68242d43ff"}
May 28 18:48:43.440136 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.440093    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-577976b98c-s56wz"]
May 28 18:48:43.443253 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.443228    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.446074 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.445918    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-wpbwp\""
May 28 18:48:43.446074 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.445949    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
May 28 18:48:43.446074 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.445974    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
May 28 18:48:43.446074 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.446025    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 28 18:48:43.447089 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.447066    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
May 28 18:48:43.447234 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.447149    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ep3q4p696v2mp\""
May 28 18:48:43.453892 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.453868    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-577976b98c-s56wz"]
May 28 18:48:43.603695 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.603653    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbd2029-9fd3-45cc-a807-98245e5b0d75-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.603884 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.603739    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8cbd2029-9fd3-45cc-a807-98245e5b0d75-secret-metrics-server-tls\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.603884 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.603798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8cbd2029-9fd3-45cc-a807-98245e5b0d75-client-ca-bundle\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.603884 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.603871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8cbd2029-9fd3-45cc-a807-98245e5b0d75-audit-log\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.604027 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.603906    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8cbd2029-9fd3-45cc-a807-98245e5b0d75-secret-metrics-server-client-certs\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.604027 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.603989    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhtj8\" (UniqueName: \"kubernetes.io/projected/8cbd2029-9fd3-45cc-a807-98245e5b0d75-kube-api-access-nhtj8\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.604121 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.604030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8cbd2029-9fd3-45cc-a807-98245e5b0d75-metrics-server-audit-profiles\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.705315 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.705217    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbd2029-9fd3-45cc-a807-98245e5b0d75-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.705315 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.705280    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8cbd2029-9fd3-45cc-a807-98245e5b0d75-secret-metrics-server-tls\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.705315 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.705315    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8cbd2029-9fd3-45cc-a807-98245e5b0d75-client-ca-bundle\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.705596 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.705355    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8cbd2029-9fd3-45cc-a807-98245e5b0d75-audit-log\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.705596 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.705385    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8cbd2029-9fd3-45cc-a807-98245e5b0d75-secret-metrics-server-client-certs\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.705596 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.705421    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nhtj8\" (UniqueName: \"kubernetes.io/projected/8cbd2029-9fd3-45cc-a807-98245e5b0d75-kube-api-access-nhtj8\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.705596 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.705453    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8cbd2029-9fd3-45cc-a807-98245e5b0d75-metrics-server-audit-profiles\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.706051 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.706012    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8cbd2029-9fd3-45cc-a807-98245e5b0d75-audit-log\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.706178 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.706101    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbd2029-9fd3-45cc-a807-98245e5b0d75-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.706544 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.706523    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8cbd2029-9fd3-45cc-a807-98245e5b0d75-metrics-server-audit-profiles\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.708453 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.708389    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8cbd2029-9fd3-45cc-a807-98245e5b0d75-client-ca-bundle\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.708453 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.708410    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8cbd2029-9fd3-45cc-a807-98245e5b0d75-secret-metrics-server-tls\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.708704 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.708683    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8cbd2029-9fd3-45cc-a807-98245e5b0d75-secret-metrics-server-client-certs\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.713768 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.713747    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhtj8\" (UniqueName: \"kubernetes.io/projected/8cbd2029-9fd3-45cc-a807-98245e5b0d75-kube-api-access-nhtj8\") pod \"metrics-server-577976b98c-s56wz\" (UID: \"8cbd2029-9fd3-45cc-a807-98245e5b0d75\") " pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.755399 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.755361    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:48:43.759948 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.759914    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r4d59" event={"ID":"f8c10cda-fc2b-487c-8b95-44064737610d","Type":"ContainerStarted","Data":"ce20056af3460f9798de8e62f8049835d380eed30ebea65ba8753c8966ac4790"}
May 28 18:48:43.760095 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.759959    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r4d59" event={"ID":"f8c10cda-fc2b-487c-8b95-44064737610d","Type":"ContainerStarted","Data":"bfc475cdfb5d7d850afbced4a91bc878709c808499c027980c3b3c53fdaeb326"}
May 28 18:48:43.781132 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.781079    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r4d59" podStartSLOduration=3.889055592 podStartE2EDuration="4.781064398s" podCreationTimestamp="2026-05-28 18:48:39 +0000 UTC" firstStartedPulling="2026-05-28 18:48:41.031702836 +0000 UTC m=+105.290924980" lastFinishedPulling="2026-05-28 18:48:41.923711629 +0000 UTC m=+106.182933786" observedRunningTime="2026-05-28 18:48:43.779778128 +0000 UTC m=+108.039000296" watchObservedRunningTime="2026-05-28 18:48:43.781064398 +0000 UTC m=+108.040286563"
May 28 18:48:43.896863 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.896827    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt"]
May 28 18:48:43.900255 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.900232    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt"
May 28 18:48:43.902673 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.902641    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
May 28 18:48:43.902778 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.902724    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-x7zwl\""
May 28 18:48:43.907291 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:43.907249    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt"]
May 28 18:48:44.009179 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.009090    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/89ab8170-7fd7-4238-9213-ba216b213b30-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-w2mxt\" (UID: \"89ab8170-7fd7-4238-9213-ba216b213b30\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt"
May 28 18:48:44.110205 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.110154    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/89ab8170-7fd7-4238-9213-ba216b213b30-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-w2mxt\" (UID: \"89ab8170-7fd7-4238-9213-ba216b213b30\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt"
May 28 18:48:44.110389 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:44.110307    2577 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
May 28 18:48:44.110389 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:48:44.110385    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ab8170-7fd7-4238-9213-ba216b213b30-monitoring-plugin-cert podName:89ab8170-7fd7-4238-9213-ba216b213b30 nodeName:}" failed. No retries permitted until 2026-05-28 18:48:44.610365117 +0000 UTC m=+108.869587340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/89ab8170-7fd7-4238-9213-ba216b213b30-monitoring-plugin-cert") pod "monitoring-plugin-665c9c8584-w2mxt" (UID: "89ab8170-7fd7-4238-9213-ba216b213b30") : secret "monitoring-plugin-cert" not found
May 28 18:48:44.344280 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.344202    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"]
May 28 18:48:44.346806 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.346782    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.349523 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.349494    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
May 28 18:48:44.349733 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.349714    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
May 28 18:48:44.349839 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.349769    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
May 28 18:48:44.349959 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.349943    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-dw9xk\""
May 28 18:48:44.350055 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.350041    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
May 28 18:48:44.350106 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.350045    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
May 28 18:48:44.354593 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.354562    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
May 28 18:48:44.360488 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.360465    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"]
May 28 18:48:44.513818 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.513776    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5537b1f2-9cad-4fd5-8953-213489037a0d-federate-client-tls\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.513989 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.513827    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5537b1f2-9cad-4fd5-8953-213489037a0d-metrics-client-ca\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.513989 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.513861    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5537b1f2-9cad-4fd5-8953-213489037a0d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.513989 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.513915    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5537b1f2-9cad-4fd5-8953-213489037a0d-serving-certs-ca-bundle\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.513989 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.513979    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5537b1f2-9cad-4fd5-8953-213489037a0d-secret-telemeter-client\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.514237 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.514041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5537b1f2-9cad-4fd5-8953-213489037a0d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.514237 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.514095    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5537b1f2-9cad-4fd5-8953-213489037a0d-telemeter-client-tls\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.514237 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.514171    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5c4t\" (UniqueName: \"kubernetes.io/projected/5537b1f2-9cad-4fd5-8953-213489037a0d-kube-api-access-g5c4t\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.615312 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.615224    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/89ab8170-7fd7-4238-9213-ba216b213b30-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-w2mxt\" (UID: \"89ab8170-7fd7-4238-9213-ba216b213b30\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt"
May 28 18:48:44.615312 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.615267    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5537b1f2-9cad-4fd5-8953-213489037a0d-federate-client-tls\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.615540 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.615331    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5537b1f2-9cad-4fd5-8953-213489037a0d-metrics-client-ca\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.615540 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.615366    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5537b1f2-9cad-4fd5-8953-213489037a0d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.615540 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.615396    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5537b1f2-9cad-4fd5-8953-213489037a0d-serving-certs-ca-bundle\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.615540 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.615436    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5537b1f2-9cad-4fd5-8953-213489037a0d-secret-telemeter-client\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.615540 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.615467    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5537b1f2-9cad-4fd5-8953-213489037a0d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.615540 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.615514    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5537b1f2-9cad-4fd5-8953-213489037a0d-telemeter-client-tls\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.616653 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.615551    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g5c4t\" (UniqueName: \"kubernetes.io/projected/5537b1f2-9cad-4fd5-8953-213489037a0d-kube-api-access-g5c4t\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.616653 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.616187    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5537b1f2-9cad-4fd5-8953-213489037a0d-metrics-client-ca\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.616827 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.616780    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5537b1f2-9cad-4fd5-8953-213489037a0d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.616963 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.616937    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5537b1f2-9cad-4fd5-8953-213489037a0d-serving-certs-ca-bundle\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.618766 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.618738    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5537b1f2-9cad-4fd5-8953-213489037a0d-secret-telemeter-client\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.618875 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.618750    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5537b1f2-9cad-4fd5-8953-213489037a0d-federate-client-tls\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.618875 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.618851    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5537b1f2-9cad-4fd5-8953-213489037a0d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.618989 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.618906    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/89ab8170-7fd7-4238-9213-ba216b213b30-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-w2mxt\" (UID: \"89ab8170-7fd7-4238-9213-ba216b213b30\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt"
May 28 18:48:44.619170 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.619150    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5537b1f2-9cad-4fd5-8953-213489037a0d-telemeter-client-tls\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.624251 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.624227    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5c4t\" (UniqueName: \"kubernetes.io/projected/5537b1f2-9cad-4fd5-8953-213489037a0d-kube-api-access-g5c4t\") pod \"telemeter-client-77b4bfff95-6xj9h\" (UID: \"5537b1f2-9cad-4fd5-8953-213489037a0d\") " pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.659246 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.659206    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"
May 28 18:48:44.811995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:44.811957    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt"
May 28 18:48:45.365675 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.365636    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 18:48:45.369452 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.369425    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.372100 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.372077    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 28 18:48:45.372314 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.372084    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 28 18:48:45.372405 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.372334    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 28 18:48:45.372405 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.372347    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 28 18:48:45.372405 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.372349    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 28 18:48:45.372564 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.372149    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 28 18:48:45.372751 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.372732    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8vfhd\""
May 28 18:48:45.372844 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.372735    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 28 18:48:45.373561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.373432    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 28 18:48:45.373561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.373437    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 28 18:48:45.373561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.373463    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-egtupvir1lvri\""
May 28 18:48:45.373561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.373511    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 28 18:48:45.376602 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.376570    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 28 18:48:45.378932 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.378912    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 28 18:48:45.383861 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.383839    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 18:48:45.524575 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.524526    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.524794 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.524587    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.524794 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.524645    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.524794 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.524675    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.524794 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.524702    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.524794 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.524751    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.525031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.524878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.525031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.524913    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.525031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.524944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d8e0b49-aa50-4c67-a189-41497d66fb5a-config-out\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.525031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.524998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-config\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.525031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.525013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.525242 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.525058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l2mb\" (UniqueName: \"kubernetes.io/projected/4d8e0b49-aa50-4c67-a189-41497d66fb5a-kube-api-access-4l2mb\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.525242 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.525077    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.525242 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.525097    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.525242 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.525143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-web-config\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.525242 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.525187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.525242 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.525229    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.525471 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.525268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d8e0b49-aa50-4c67-a189-41497d66fb5a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.625884 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.625793    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.625884 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.625838    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d8e0b49-aa50-4c67-a189-41497d66fb5a-config-out\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.625884 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.625885    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-config\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626170 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.625908    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626170 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.625939    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4l2mb\" (UniqueName: \"kubernetes.io/projected/4d8e0b49-aa50-4c67-a189-41497d66fb5a-kube-api-access-4l2mb\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626170 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.625965    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626170 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.625992    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626170 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.626018    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-web-config\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626170 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.626058    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626170 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.626082    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626170 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.626118    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d8e0b49-aa50-4c67-a189-41497d66fb5a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626170 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.626169    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626560 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.626204    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626560 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.626229    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626560 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.626263    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626560 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.626290    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626560 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.626323    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.626560 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.626371    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.628421 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.627088    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.628421 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.627284    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.628421 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.627819    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.629883 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.629379    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-config\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.629883 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.629453    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.630255 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.630098    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.630730 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.630708    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.630907 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.630879    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.631275 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.631166    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.631275 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.631232    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.631275 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.631239    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.631491 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.631409    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d8e0b49-aa50-4c67-a189-41497d66fb5a-config-out\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.632137 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.632089    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d8e0b49-aa50-4c67-a189-41497d66fb5a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.633973 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.633931    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.633973 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.633963    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-web-config\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.634157 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.634132    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.634262 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.634231    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.636487 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.636465    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l2mb\" (UniqueName: \"kubernetes.io/projected/4d8e0b49-aa50-4c67-a189-41497d66fb5a-kube-api-access-4l2mb\") pod \"prometheus-k8s-0\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.681540 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.681506    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:48:45.783523 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.783480    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:45.783523 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.783523    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:45.789894 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:45.789867    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:46.774037 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:46.774003    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:48:47.735398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:47.735363    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4n69j"
May 28 18:48:50.403729 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:50.403660    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt"]
May 28 18:48:50.407705 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:50.407671    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ab8170_7fd7_4238_9213_ba216b213b30.slice/crio-a01b37d74d3a720a52ddde01c7694f5a76157333a106b0ec0d34aee457b0eb8e WatchSource:0}: Error finding container a01b37d74d3a720a52ddde01c7694f5a76157333a106b0ec0d34aee457b0eb8e: Status 404 returned error can't find the container with id a01b37d74d3a720a52ddde01c7694f5a76157333a106b0ec0d34aee457b0eb8e
May 28 18:48:50.426908 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:50.426875    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-577976b98c-s56wz"]
May 28 18:48:50.432747 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:50.432671    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cbd2029_9fd3_45cc_a807_98245e5b0d75.slice/crio-1adad9d3bf721a3f03393344593e9f98c518aef2d3d906ab4d6b045c0e3778d2 WatchSource:0}: Error finding container 1adad9d3bf721a3f03393344593e9f98c518aef2d3d906ab4d6b045c0e3778d2: Status 404 returned error can't find the container with id 1adad9d3bf721a3f03393344593e9f98c518aef2d3d906ab4d6b045c0e3778d2
May 28 18:48:50.453149 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:50.453108    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 18:48:50.457995 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:50.457966    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d8e0b49_aa50_4c67_a189_41497d66fb5a.slice/crio-15c4b9110971b1f6a2054cf888539cd9fce9fdec170ba4d11074bba1b9632cda WatchSource:0}: Error finding container 15c4b9110971b1f6a2054cf888539cd9fce9fdec170ba4d11074bba1b9632cda: Status 404 returned error can't find the container with id 15c4b9110971b1f6a2054cf888539cd9fce9fdec170ba4d11074bba1b9632cda
May 28 18:48:50.469687 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:50.469597    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77b4bfff95-6xj9h"]
May 28 18:48:50.472067 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:48:50.472039    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5537b1f2_9cad_4fd5_8953_213489037a0d.slice/crio-140932729cfd3dc314c8129a5665287631aaadaeadc21d91d25f3ec23f2cf44a WatchSource:0}: Error finding container 140932729cfd3dc314c8129a5665287631aaadaeadc21d91d25f3ec23f2cf44a: Status 404 returned error can't find the container with id 140932729cfd3dc314c8129a5665287631aaadaeadc21d91d25f3ec23f2cf44a
May 28 18:48:50.783764 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:50.783721    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h" event={"ID":"5537b1f2-9cad-4fd5-8953-213489037a0d","Type":"ContainerStarted","Data":"140932729cfd3dc314c8129a5665287631aaadaeadc21d91d25f3ec23f2cf44a"}
May 28 18:48:50.785051 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:50.785017    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-577976b98c-s56wz" event={"ID":"8cbd2029-9fd3-45cc-a807-98245e5b0d75","Type":"ContainerStarted","Data":"1adad9d3bf721a3f03393344593e9f98c518aef2d3d906ab4d6b045c0e3778d2"}
May 28 18:48:50.786303 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:50.786260    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerStarted","Data":"15c4b9110971b1f6a2054cf888539cd9fce9fdec170ba4d11074bba1b9632cda"}
May 28 18:48:50.787475 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:50.787445    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt" event={"ID":"89ab8170-7fd7-4238-9213-ba216b213b30","Type":"ContainerStarted","Data":"a01b37d74d3a720a52ddde01c7694f5a76157333a106b0ec0d34aee457b0eb8e"}
May 28 18:48:50.788920 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:50.788881    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-t4dk9" event={"ID":"addb0542-e63c-4184-bbe8-1e21d45a4f68","Type":"ContainerStarted","Data":"b1f5894168b05e6ac41d143071efe05c820be62357e55aaddcec0bf694078394"}
May 28 18:48:50.789133 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:50.789116    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-57bd96b888-t4dk9"
May 28 18:48:50.805437 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:50.805277    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-57bd96b888-t4dk9"
May 28 18:48:50.806853 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:50.806808    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-57bd96b888-t4dk9" podStartSLOduration=1.04111707 podStartE2EDuration="19.80679407s" podCreationTimestamp="2026-05-28 18:48:31 +0000 UTC" firstStartedPulling="2026-05-28 18:48:31.558124073 +0000 UTC m=+95.817346217" lastFinishedPulling="2026-05-28 18:48:50.323801057 +0000 UTC m=+114.583023217" observedRunningTime="2026-05-28 18:48:50.805130682 +0000 UTC m=+115.064352871" watchObservedRunningTime="2026-05-28 18:48:50.80679407 +0000 UTC m=+115.066016236"
May 28 18:48:52.707907 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:52.707876    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-676cdccb6-g86th"
May 28 18:48:55.809392 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:55.809354    2577 generic.go:358] "Generic (PLEG): container finished" podID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerID="0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40" exitCode=0
May 28 18:48:55.809873 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:55.809431    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerDied","Data":"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"}
May 28 18:48:55.811158 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:55.811122    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt" event={"ID":"89ab8170-7fd7-4238-9213-ba216b213b30","Type":"ContainerStarted","Data":"9d6ef9602e0ff080561e19d076d0953e6e58c81e27916692003944dce544f251"}
May 28 18:48:55.811344 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:55.811235    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt"
May 28 18:48:55.813560 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:55.813533    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h" event={"ID":"5537b1f2-9cad-4fd5-8953-213489037a0d","Type":"ContainerStarted","Data":"b4b2d58aad41dbccc5d9b0d86d271d09200a4074a101fd0dbbf443b188a149c1"}
May 28 18:48:55.813680 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:55.813567    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h" event={"ID":"5537b1f2-9cad-4fd5-8953-213489037a0d","Type":"ContainerStarted","Data":"d36c825e80d27bd344369317ddacf952e3243aa417fae0e96845653e1deecf58"}
May 28 18:48:55.813680 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:55.813582    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h" event={"ID":"5537b1f2-9cad-4fd5-8953-213489037a0d","Type":"ContainerStarted","Data":"6e4e0a10d7f677a33e7624c81a0b69a22fa2d29faad1458494dae1921e5741a9"}
May 28 18:48:55.815047 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:55.815011    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-577976b98c-s56wz" event={"ID":"8cbd2029-9fd3-45cc-a807-98245e5b0d75","Type":"ContainerStarted","Data":"2eb73a20a48e99d099e4082387479f5a153ef9e3c54fd90cff4102401696a056"}
May 28 18:48:55.817653 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:55.817634    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt"
May 28 18:48:55.855276 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:55.855221    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-665c9c8584-w2mxt" podStartSLOduration=8.504207592 podStartE2EDuration="12.855205029s" podCreationTimestamp="2026-05-28 18:48:43 +0000 UTC" firstStartedPulling="2026-05-28 18:48:50.410228822 +0000 UTC m=+114.669450967" lastFinishedPulling="2026-05-28 18:48:54.761226259 +0000 UTC m=+119.020448404" observedRunningTime="2026-05-28 18:48:55.854295186 +0000 UTC m=+120.113517353" watchObservedRunningTime="2026-05-28 18:48:55.855205029 +0000 UTC m=+120.114427194"
May 28 18:48:55.878013 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:55.874764    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-77b4bfff95-6xj9h" podStartSLOduration=7.580148455 podStartE2EDuration="11.874751s" podCreationTimestamp="2026-05-28 18:48:44 +0000 UTC" firstStartedPulling="2026-05-28 18:48:50.473942312 +0000 UTC m=+114.733164456" lastFinishedPulling="2026-05-28 18:48:54.768544852 +0000 UTC m=+119.027767001" observedRunningTime="2026-05-28 18:48:55.87319399 +0000 UTC m=+120.132416158" watchObservedRunningTime="2026-05-28 18:48:55.874751 +0000 UTC m=+120.133973185"
May 28 18:48:55.890137 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:48:55.890086    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-577976b98c-s56wz" podStartSLOduration=8.563773216 podStartE2EDuration="12.890071119s" podCreationTimestamp="2026-05-28 18:48:43 +0000 UTC" firstStartedPulling="2026-05-28 18:48:50.434864871 +0000 UTC m=+114.694087015" lastFinishedPulling="2026-05-28 18:48:54.76116276 +0000 UTC m=+119.020384918" observedRunningTime="2026-05-28 18:48:55.888199104 +0000 UTC m=+120.147421270" watchObservedRunningTime="2026-05-28 18:48:55.890071119 +0000 UTC m=+120.149293285"
May 28 18:49:00.838310 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:00.838274    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerStarted","Data":"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"}
May 28 18:49:01.843635 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:01.843581    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerStarted","Data":"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"}
May 28 18:49:03.756073 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:03.755970    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:49:03.756437 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:03.756065    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:49:03.852904 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:03.852870    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerStarted","Data":"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"}
May 28 18:49:03.852904 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:03.852908    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerStarted","Data":"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"}
May 28 18:49:03.853116 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:03.852919    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerStarted","Data":"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"}
May 28 18:49:03.853116 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:03.852928    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerStarted","Data":"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"}
May 28 18:49:03.881630 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:03.881571    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.16643958 podStartE2EDuration="18.881557771s" podCreationTimestamp="2026-05-28 18:48:45 +0000 UTC" firstStartedPulling="2026-05-28 18:48:50.45995936 +0000 UTC m=+114.719181505" lastFinishedPulling="2026-05-28 18:49:03.17507754 +0000 UTC m=+127.434299696" observedRunningTime="2026-05-28 18:49:03.879681627 +0000 UTC m=+128.138903794" watchObservedRunningTime="2026-05-28 18:49:03.881557771 +0000 UTC m=+128.140779988"
May 28 18:49:05.682043 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:05.682011    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:49:06.139016 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:06.138983    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:49:06.141351 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:06.141332    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1625d5d8-1437-4fb6-b18e-320acb890173-metrics-certs\") pod \"network-metrics-daemon-9xhpt\" (UID: \"1625d5d8-1437-4fb6-b18e-320acb890173\") " pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:49:06.175021 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:06.174991    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-25f4v\""
May 28 18:49:06.183093 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:06.183073    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xhpt"
May 28 18:49:06.306816 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:06.306789    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9xhpt"]
May 28 18:49:06.308920 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:49:06.308895    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1625d5d8_1437_4fb6_b18e_320acb890173.slice/crio-245488c0dc987a6dab5992cf2a6f6cd043ac3e33307a36851a56a7a1083357b5 WatchSource:0}: Error finding container 245488c0dc987a6dab5992cf2a6f6cd043ac3e33307a36851a56a7a1083357b5: Status 404 returned error can't find the container with id 245488c0dc987a6dab5992cf2a6f6cd043ac3e33307a36851a56a7a1083357b5
May 28 18:49:06.862445 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:06.862397    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9xhpt" event={"ID":"1625d5d8-1437-4fb6-b18e-320acb890173","Type":"ContainerStarted","Data":"245488c0dc987a6dab5992cf2a6f6cd043ac3e33307a36851a56a7a1083357b5"}
May 28 18:49:07.676532 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:07.676497    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bb69dd9f8-9z72z"]
May 28 18:49:07.867063 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:07.867030    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9xhpt" event={"ID":"1625d5d8-1437-4fb6-b18e-320acb890173","Type":"ContainerStarted","Data":"2dd5c19d7d061815077de07938da47425b24db22dab78416c7b992856866cd60"}
May 28 18:49:08.871142 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:08.871104    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9xhpt" event={"ID":"1625d5d8-1437-4fb6-b18e-320acb890173","Type":"ContainerStarted","Data":"5044e0553306a7c87b72ef6e1f62292772932a04d7c4d5d9db78e7221362ef89"}
May 28 18:49:14.891151 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:14.891113    2577 generic.go:358] "Generic (PLEG): container finished" podID="eedbec9e-ba9d-464c-91be-9f9d8e3688ae" containerID="9470b0542e0d0bef2d6282a8d4b126f48889cbc66bc5f29226545d4c1648086e" exitCode=0
May 28 18:49:14.891644 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:14.891169    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn" event={"ID":"eedbec9e-ba9d-464c-91be-9f9d8e3688ae","Type":"ContainerDied","Data":"9470b0542e0d0bef2d6282a8d4b126f48889cbc66bc5f29226545d4c1648086e"}
May 28 18:49:14.891644 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:14.891489    2577 scope.go:117] "RemoveContainer" containerID="9470b0542e0d0bef2d6282a8d4b126f48889cbc66bc5f29226545d4c1648086e"
May 28 18:49:14.907377 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:14.907331    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9xhpt" podStartSLOduration=137.592872246 podStartE2EDuration="2m18.907317486s" podCreationTimestamp="2026-05-28 18:46:56 +0000 UTC" firstStartedPulling="2026-05-28 18:49:06.310938791 +0000 UTC m=+130.570160938" lastFinishedPulling="2026-05-28 18:49:07.625384031 +0000 UTC m=+131.884606178" observedRunningTime="2026-05-28 18:49:08.886548274 +0000 UTC m=+133.145770440" watchObservedRunningTime="2026-05-28 18:49:14.907317486 +0000 UTC m=+139.166539659"
May 28 18:49:15.896785 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:15.896751    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-lngkn" event={"ID":"eedbec9e-ba9d-464c-91be-9f9d8e3688ae","Type":"ContainerStarted","Data":"3a4d7b118eb11b34244d3b2318661640cdc082958b0544227bdd14adbf464eb0"}
May 28 18:49:23.761090 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:23.761059    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:49:23.765149 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:23.765126    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-577976b98c-s56wz"
May 28 18:49:23.926125 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:23.926093    2577 generic.go:358] "Generic (PLEG): container finished" podID="3a0962b8-484f-46c3-8f88-4eaf3130145e" containerID="2442a03d3e007f2fc4364280143d11322cfe66241746b59d911e00b4400265a5" exitCode=0
May 28 18:49:23.926297 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:23.926166    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-jx2bq" event={"ID":"3a0962b8-484f-46c3-8f88-4eaf3130145e","Type":"ContainerDied","Data":"2442a03d3e007f2fc4364280143d11322cfe66241746b59d911e00b4400265a5"}
May 28 18:49:23.926577 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:23.926562    2577 scope.go:117] "RemoveContainer" containerID="2442a03d3e007f2fc4364280143d11322cfe66241746b59d911e00b4400265a5"
May 28 18:49:24.930121 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:24.930077    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-jx2bq" event={"ID":"3a0962b8-484f-46c3-8f88-4eaf3130145e","Type":"ContainerStarted","Data":"71c1d579458ef5ffe37e99cff86ba179df17e495a91c6ec5e5f8532b5887d88c"}
May 28 18:49:32.697333 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:32.697268    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-bb69dd9f8-9z72z" podUID="2fa17eb3-d549-44fc-a836-9b12b1a91aee" containerName="console" containerID="cri-o://42c7e9034f26b38fa193d9c09345bf3ee3cde3d6209def4293a71fa679e80025" gracePeriod=15
May 28 18:49:32.960128 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:32.960054    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-bb69dd9f8-9z72z_2fa17eb3-d549-44fc-a836-9b12b1a91aee/console/0.log"
May 28 18:49:32.960128 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:32.960092    2577 generic.go:358] "Generic (PLEG): container finished" podID="2fa17eb3-d549-44fc-a836-9b12b1a91aee" containerID="42c7e9034f26b38fa193d9c09345bf3ee3cde3d6209def4293a71fa679e80025" exitCode=2
May 28 18:49:32.960321 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:32.960180    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb69dd9f8-9z72z" event={"ID":"2fa17eb3-d549-44fc-a836-9b12b1a91aee","Type":"ContainerDied","Data":"42c7e9034f26b38fa193d9c09345bf3ee3cde3d6209def4293a71fa679e80025"}
May 28 18:49:32.984158 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:32.984128    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-bb69dd9f8-9z72z_2fa17eb3-d549-44fc-a836-9b12b1a91aee/console/0.log"
May 28 18:49:32.984306 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:32.984202    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:49:33.075344 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.075305    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-serving-cert\") pod \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") "
May 28 18:49:33.075344 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.075347    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-oauth-config\") pod \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") "
May 28 18:49:33.075583 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.075366    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-config\") pod \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") "
May 28 18:49:33.075583 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.075413    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-oauth-serving-cert\") pod \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") "
May 28 18:49:33.075583 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.075439    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72qk8\" (UniqueName: \"kubernetes.io/projected/2fa17eb3-d549-44fc-a836-9b12b1a91aee-kube-api-access-72qk8\") pod \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") "
May 28 18:49:33.075583 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.075455    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-service-ca\") pod \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\" (UID: \"2fa17eb3-d549-44fc-a836-9b12b1a91aee\") "
May 28 18:49:33.075933 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.075904    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-config" (OuterVolumeSpecName: "console-config") pod "2fa17eb3-d549-44fc-a836-9b12b1a91aee" (UID: "2fa17eb3-d549-44fc-a836-9b12b1a91aee"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 18:49:33.076025 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.075927    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-service-ca" (OuterVolumeSpecName: "service-ca") pod "2fa17eb3-d549-44fc-a836-9b12b1a91aee" (UID: "2fa17eb3-d549-44fc-a836-9b12b1a91aee"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 18:49:33.076025 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.075933    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2fa17eb3-d549-44fc-a836-9b12b1a91aee" (UID: "2fa17eb3-d549-44fc-a836-9b12b1a91aee"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 18:49:33.077800 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.077775    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2fa17eb3-d549-44fc-a836-9b12b1a91aee" (UID: "2fa17eb3-d549-44fc-a836-9b12b1a91aee"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 18:49:33.077908 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.077775    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa17eb3-d549-44fc-a836-9b12b1a91aee-kube-api-access-72qk8" (OuterVolumeSpecName: "kube-api-access-72qk8") pod "2fa17eb3-d549-44fc-a836-9b12b1a91aee" (UID: "2fa17eb3-d549-44fc-a836-9b12b1a91aee"). InnerVolumeSpecName "kube-api-access-72qk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 18:49:33.077908 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.077856    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2fa17eb3-d549-44fc-a836-9b12b1a91aee" (UID: "2fa17eb3-d549-44fc-a836-9b12b1a91aee"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 18:49:33.176878 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.176840    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-serving-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:49:33.176878 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.176871    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-oauth-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:49:33.176878 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.176882    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-console-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:49:33.176878 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.176891    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-oauth-serving-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:49:33.177142 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.176900    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72qk8\" (UniqueName: \"kubernetes.io/projected/2fa17eb3-d549-44fc-a836-9b12b1a91aee-kube-api-access-72qk8\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:49:33.177142 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.176909    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2fa17eb3-d549-44fc-a836-9b12b1a91aee-service-ca\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:49:33.963797 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.963773    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-bb69dd9f8-9z72z_2fa17eb3-d549-44fc-a836-9b12b1a91aee/console/0.log"
May 28 18:49:33.964304 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.963825    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb69dd9f8-9z72z" event={"ID":"2fa17eb3-d549-44fc-a836-9b12b1a91aee","Type":"ContainerDied","Data":"8a4a94b7f4bb4c2554fc4455f78b7bc92553ec227a80ecdd6f54447807d712ac"}
May 28 18:49:33.964304 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.963862    2577 scope.go:117] "RemoveContainer" containerID="42c7e9034f26b38fa193d9c09345bf3ee3cde3d6209def4293a71fa679e80025"
May 28 18:49:33.964304 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.963888    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb69dd9f8-9z72z"
May 28 18:49:33.984761 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.984737    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bb69dd9f8-9z72z"]
May 28 18:49:33.990384 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:33.990364    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bb69dd9f8-9z72z"]
May 28 18:49:34.358990 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:34.358900    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fa17eb3-d549-44fc-a836-9b12b1a91aee" path="/var/lib/kubelet/pods/2fa17eb3-d549-44fc-a836-9b12b1a91aee/volumes"
May 28 18:49:38.978974 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:38.978941    2577 generic.go:358] "Generic (PLEG): container finished" podID="4f285834-d577-4e96-b077-8d242366862e" containerID="a8b153894155bfb33b4052acffb6e9d22ebc082aa8a35b574770d14c0b917e04" exitCode=0
May 28 18:49:38.979389 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:38.979022    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9" event={"ID":"4f285834-d577-4e96-b077-8d242366862e","Type":"ContainerDied","Data":"a8b153894155bfb33b4052acffb6e9d22ebc082aa8a35b574770d14c0b917e04"}
May 28 18:49:38.979389 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:38.979345    2577 scope.go:117] "RemoveContainer" containerID="a8b153894155bfb33b4052acffb6e9d22ebc082aa8a35b574770d14c0b917e04"
May 28 18:49:39.983724 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:39.983686    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-h85g9" event={"ID":"4f285834-d577-4e96-b077-8d242366862e","Type":"ContainerStarted","Data":"1aa0acbc28d99cb210bc4f142e6dfb11bf068bc5de4eb4e004e58d6d65c15432"}
May 28 18:49:45.682240 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:45.682208    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:49:45.702698 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:45.702663    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:49:46.022111 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:49:46.022028    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:03.740051 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:03.740011    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 18:50:03.740506 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:03.740461    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="prometheus" containerID="cri-o://ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a" gracePeriod=600
May 28 18:50:03.740579 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:03.740493    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="kube-rbac-proxy" containerID="cri-o://b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9" gracePeriod=600
May 28 18:50:03.740668 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:03.740545    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="thanos-sidecar" containerID="cri-o://2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25" gracePeriod=600
May 28 18:50:03.740668 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:03.740604    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="config-reloader" containerID="cri-o://f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8" gracePeriod=600
May 28 18:50:03.740668 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:03.740644    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a" gracePeriod=600
May 28 18:50:03.740830 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:03.740678    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="kube-rbac-proxy-web" containerID="cri-o://44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f" gracePeriod=600
May 28 18:50:03.979143 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:03.979118    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.040365 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040262    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-trusted-ca-bundle\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.040365 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040302    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d8e0b49-aa50-4c67-a189-41497d66fb5a-tls-assets\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.040365 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040322    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-kubelet-serving-ca-bundle\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.040365 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040345    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-tls\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.040724 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040504    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l2mb\" (UniqueName: \"kubernetes.io/projected/4d8e0b49-aa50-4c67-a189-41497d66fb5a-kube-api-access-4l2mb\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.040724 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040546    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-thanos-prometheus-http-client-file\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.040724 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040580    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.040724 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040628    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-serving-certs-ca-bundle\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.040724 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040653    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-grpc-tls\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.040724 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040692    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-metrics-client-certs\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.041031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040744    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.041031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040771    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-k8s-rulefiles-0\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.041031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040817    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-k8s-db\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.041031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040820    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 18:50:04.041031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040829    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 18:50:04.041031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040842    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-config\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.041031 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.040904    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-metrics-client-ca\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.041422 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.041048    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d8e0b49-aa50-4c67-a189-41497d66fb5a-config-out\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.041422 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.041077    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-web-config\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.041422 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.041106    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-kube-rbac-proxy\") pod \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\" (UID: \"4d8e0b49-aa50-4c67-a189-41497d66fb5a\") "
May 28 18:50:04.041422 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.041101    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 18:50:04.041422 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.041366    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.041422 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.041386    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.041422 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.041402    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.044413 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.044159    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 18:50:04.044747 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.044720    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 18:50:04.044880 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.044857    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 18:50:04.044993 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.044889    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 18:50:04.044993 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.044981    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d8e0b49-aa50-4c67-a189-41497d66fb5a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 18:50:04.045177 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.045062    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 18:50:04.045177 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.045071    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 18:50:04.045284 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.045189    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 18:50:04.045560 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.045533    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d8e0b49-aa50-4c67-a189-41497d66fb5a-kube-api-access-4l2mb" (OuterVolumeSpecName: "kube-api-access-4l2mb") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "kube-api-access-4l2mb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 18:50:04.045711 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.045680    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 18:50:04.045787 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.045692    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 18:50:04.046099 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.046065    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-config" (OuterVolumeSpecName: "config") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 18:50:04.046217 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.046127    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d8e0b49-aa50-4c67-a189-41497d66fb5a-config-out" (OuterVolumeSpecName: "config-out") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 18:50:04.046594 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.046569    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 18:50:04.058728 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058684    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-web-config" (OuterVolumeSpecName: "web-config") pod "4d8e0b49-aa50-4c67-a189-41497d66fb5a" (UID: "4d8e0b49-aa50-4c67-a189-41497d66fb5a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 18:50:04.058892 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058831    2577 generic.go:358] "Generic (PLEG): container finished" podID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerID="340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a" exitCode=0
May 28 18:50:04.058892 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058852    2577 generic.go:358] "Generic (PLEG): container finished" podID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerID="b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9" exitCode=0
May 28 18:50:04.058892 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058861    2577 generic.go:358] "Generic (PLEG): container finished" podID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerID="44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f" exitCode=0
May 28 18:50:04.058892 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058867    2577 generic.go:358] "Generic (PLEG): container finished" podID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerID="2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25" exitCode=0
May 28 18:50:04.058892 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058856    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerDied","Data":"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"}
May 28 18:50:04.059146 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058898    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerDied","Data":"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"}
May 28 18:50:04.059146 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058909    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerDied","Data":"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"}
May 28 18:50:04.059146 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058919    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerDied","Data":"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"}
May 28 18:50:04.059146 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058930    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerDied","Data":"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"}
May 28 18:50:04.059146 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058872    2577 generic.go:358] "Generic (PLEG): container finished" podID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerID="f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8" exitCode=0
May 28 18:50:04.059146 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058953    2577 generic.go:358] "Generic (PLEG): container finished" podID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerID="ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a" exitCode=0
May 28 18:50:04.059146 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058969    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.059146 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058991    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerDied","Data":"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"}
May 28 18:50:04.059146 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.058945    2577 scope.go:117] "RemoveContainer" containerID="340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"
May 28 18:50:04.059146 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.059015    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4d8e0b49-aa50-4c67-a189-41497d66fb5a","Type":"ContainerDied","Data":"15c4b9110971b1f6a2054cf888539cd9fce9fdec170ba4d11074bba1b9632cda"}
May 28 18:50:04.067852 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.067828    2577 scope.go:117] "RemoveContainer" containerID="b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"
May 28 18:50:04.076803 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.076781    2577 scope.go:117] "RemoveContainer" containerID="44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"
May 28 18:50:04.084279 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.084257    2577 scope.go:117] "RemoveContainer" containerID="2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"
May 28 18:50:04.088877 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.088848    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 18:50:04.092497 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.092476    2577 scope.go:117] "RemoveContainer" containerID="f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"
May 28 18:50:04.093665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.093642    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 18:50:04.099595 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.099577    2577 scope.go:117] "RemoveContainer" containerID="ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"
May 28 18:50:04.107338 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.107316    2577 scope.go:117] "RemoveContainer" containerID="0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"
May 28 18:50:04.114394 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.114369    2577 scope.go:117] "RemoveContainer" containerID="340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"
May 28 18:50:04.114718 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:50:04.114693    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a\": container with ID starting with 340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a not found: ID does not exist" containerID="340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"
May 28 18:50:04.114790 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.114731    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"} err="failed to get container status \"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a\": rpc error: code = NotFound desc = could not find container \"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a\": container with ID starting with 340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a not found: ID does not exist"
May 28 18:50:04.114790 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.114788    2577 scope.go:117] "RemoveContainer" containerID="b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"
May 28 18:50:04.115045 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:50:04.115030    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9\": container with ID starting with b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9 not found: ID does not exist" containerID="b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"
May 28 18:50:04.115092 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.115048    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"} err="failed to get container status \"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9\": rpc error: code = NotFound desc = could not find container \"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9\": container with ID starting with b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9 not found: ID does not exist"
May 28 18:50:04.115092 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.115061    2577 scope.go:117] "RemoveContainer" containerID="44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"
May 28 18:50:04.115267 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:50:04.115249    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f\": container with ID starting with 44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f not found: ID does not exist" containerID="44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"
May 28 18:50:04.115306 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.115273    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"} err="failed to get container status \"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f\": rpc error: code = NotFound desc = could not find container \"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f\": container with ID starting with 44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f not found: ID does not exist"
May 28 18:50:04.115306 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.115288    2577 scope.go:117] "RemoveContainer" containerID="2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"
May 28 18:50:04.115542 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:50:04.115522    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25\": container with ID starting with 2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25 not found: ID does not exist" containerID="2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"
May 28 18:50:04.115660 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.115560    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"} err="failed to get container status \"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25\": rpc error: code = NotFound desc = could not find container \"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25\": container with ID starting with 2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25 not found: ID does not exist"
May 28 18:50:04.115660 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.115582    2577 scope.go:117] "RemoveContainer" containerID="f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"
May 28 18:50:04.116057 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:50:04.116037    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8\": container with ID starting with f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8 not found: ID does not exist" containerID="f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"
May 28 18:50:04.116143 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.116059    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"} err="failed to get container status \"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8\": rpc error: code = NotFound desc = could not find container \"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8\": container with ID starting with f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8 not found: ID does not exist"
May 28 18:50:04.116143 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.116074    2577 scope.go:117] "RemoveContainer" containerID="ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"
May 28 18:50:04.116307 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:50:04.116291    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a\": container with ID starting with ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a not found: ID does not exist" containerID="ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"
May 28 18:50:04.116350 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.116311    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"} err="failed to get container status \"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a\": rpc error: code = NotFound desc = could not find container \"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a\": container with ID starting with ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a not found: ID does not exist"
May 28 18:50:04.116350 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.116325    2577 scope.go:117] "RemoveContainer" containerID="0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"
May 28 18:50:04.116563 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:50:04.116547    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40\": container with ID starting with 0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40 not found: ID does not exist" containerID="0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"
May 28 18:50:04.116653 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.116572    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"} err="failed to get container status \"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40\": rpc error: code = NotFound desc = could not find container \"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40\": container with ID starting with 0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40 not found: ID does not exist"
May 28 18:50:04.116653 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.116585    2577 scope.go:117] "RemoveContainer" containerID="340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"
May 28 18:50:04.116841 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.116821    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"} err="failed to get container status \"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a\": rpc error: code = NotFound desc = could not find container \"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a\": container with ID starting with 340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a not found: ID does not exist"
May 28 18:50:04.116882 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.116842    2577 scope.go:117] "RemoveContainer" containerID="b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"
May 28 18:50:04.117080 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117059    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"} err="failed to get container status \"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9\": rpc error: code = NotFound desc = could not find container \"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9\": container with ID starting with b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9 not found: ID does not exist"
May 28 18:50:04.117080 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117073    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 18:50:04.117195 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117080    2577 scope.go:117] "RemoveContainer" containerID="44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"
May 28 18:50:04.117371 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117345    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"} err="failed to get container status \"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f\": rpc error: code = NotFound desc = could not find container \"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f\": container with ID starting with 44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f not found: ID does not exist"
May 28 18:50:04.117416 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117371    2577 scope.go:117] "RemoveContainer" containerID="2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"
May 28 18:50:04.117490 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117461    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fa17eb3-d549-44fc-a836-9b12b1a91aee" containerName="console"
May 28 18:50:04.117490 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117480    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fa17eb3-d549-44fc-a836-9b12b1a91aee" containerName="console"
May 28 18:50:04.117665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117496    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="kube-rbac-proxy"
May 28 18:50:04.117665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117505    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="kube-rbac-proxy"
May 28 18:50:04.117665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117526    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="thanos-sidecar"
May 28 18:50:04.117665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117535    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="thanos-sidecar"
May 28 18:50:04.117665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117544    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="kube-rbac-proxy-thanos"
May 28 18:50:04.117665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117553    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="kube-rbac-proxy-thanos"
May 28 18:50:04.117665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117566    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="config-reloader"
May 28 18:50:04.117665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117573    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="config-reloader"
May 28 18:50:04.117665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117582    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="init-config-reloader"
May 28 18:50:04.117665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117589    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="init-config-reloader"
May 28 18:50:04.117665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117633    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="kube-rbac-proxy-web"
May 28 18:50:04.117665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117624    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"} err="failed to get container status \"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25\": rpc error: code = NotFound desc = could not find container \"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25\": container with ID starting with 2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25 not found: ID does not exist"
May 28 18:50:04.117665 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117654    2577 scope.go:117] "RemoveContainer" containerID="f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"
May 28 18:50:04.118363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117642    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="kube-rbac-proxy-web"
May 28 18:50:04.118363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117724    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="prometheus"
May 28 18:50:04.118363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117735    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="prometheus"
May 28 18:50:04.118363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117818    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="2fa17eb3-d549-44fc-a836-9b12b1a91aee" containerName="console"
May 28 18:50:04.118363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117831    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="thanos-sidecar"
May 28 18:50:04.118363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117844    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="kube-rbac-proxy-web"
May 28 18:50:04.118363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117856    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="prometheus"
May 28 18:50:04.118363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117866    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="kube-rbac-proxy-thanos"
May 28 18:50:04.118363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117878    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="config-reloader"
May 28 18:50:04.118363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.117888    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" containerName="kube-rbac-proxy"
May 28 18:50:04.118363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.118283    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"} err="failed to get container status \"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8\": rpc error: code = NotFound desc = could not find container \"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8\": container with ID starting with f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8 not found: ID does not exist"
May 28 18:50:04.118363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.118299    2577 scope.go:117] "RemoveContainer" containerID="ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"
May 28 18:50:04.119056 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.118496    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"} err="failed to get container status \"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a\": rpc error: code = NotFound desc = could not find container \"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a\": container with ID starting with ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a not found: ID does not exist"
May 28 18:50:04.119056 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.118512    2577 scope.go:117] "RemoveContainer" containerID="0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"
May 28 18:50:04.119056 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.118678    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"} err="failed to get container status \"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40\": rpc error: code = NotFound desc = could not find container \"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40\": container with ID starting with 0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40 not found: ID does not exist"
May 28 18:50:04.119056 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.118703    2577 scope.go:117] "RemoveContainer" containerID="340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"
May 28 18:50:04.119056 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.118906    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"} err="failed to get container status \"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a\": rpc error: code = NotFound desc = could not find container \"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a\": container with ID starting with 340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a not found: ID does not exist"
May 28 18:50:04.119056 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.118922    2577 scope.go:117] "RemoveContainer" containerID="b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"
May 28 18:50:04.119338 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.119140    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"} err="failed to get container status \"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9\": rpc error: code = NotFound desc = could not find container \"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9\": container with ID starting with b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9 not found: ID does not exist"
May 28 18:50:04.119338 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.119153    2577 scope.go:117] "RemoveContainer" containerID="44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"
May 28 18:50:04.119338 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.119313    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"} err="failed to get container status \"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f\": rpc error: code = NotFound desc = could not find container \"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f\": container with ID starting with 44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f not found: ID does not exist"
May 28 18:50:04.119338 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.119325    2577 scope.go:117] "RemoveContainer" containerID="2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"
May 28 18:50:04.119529 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.119497    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"} err="failed to get container status \"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25\": rpc error: code = NotFound desc = could not find container \"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25\": container with ID starting with 2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25 not found: ID does not exist"
May 28 18:50:04.119529 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.119509    2577 scope.go:117] "RemoveContainer" containerID="f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"
May 28 18:50:04.119702 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.119684    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"} err="failed to get container status \"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8\": rpc error: code = NotFound desc = could not find container \"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8\": container with ID starting with f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8 not found: ID does not exist"
May 28 18:50:04.119702 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.119701    2577 scope.go:117] "RemoveContainer" containerID="ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"
May 28 18:50:04.119884 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.119869    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"} err="failed to get container status \"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a\": rpc error: code = NotFound desc = could not find container \"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a\": container with ID starting with ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a not found: ID does not exist"
May 28 18:50:04.119918 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.119886    2577 scope.go:117] "RemoveContainer" containerID="0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"
May 28 18:50:04.120057 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.120031    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"} err="failed to get container status \"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40\": rpc error: code = NotFound desc = could not find container \"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40\": container with ID starting with 0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40 not found: ID does not exist"
May 28 18:50:04.120112 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.120060    2577 scope.go:117] "RemoveContainer" containerID="340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"
May 28 18:50:04.120245 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.120230    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"} err="failed to get container status \"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a\": rpc error: code = NotFound desc = could not find container \"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a\": container with ID starting with 340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a not found: ID does not exist"
May 28 18:50:04.120289 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.120244    2577 scope.go:117] "RemoveContainer" containerID="b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"
May 28 18:50:04.120475 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.120457    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"} err="failed to get container status \"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9\": rpc error: code = NotFound desc = could not find container \"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9\": container with ID starting with b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9 not found: ID does not exist"
May 28 18:50:04.120522 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.120475    2577 scope.go:117] "RemoveContainer" containerID="44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"
May 28 18:50:04.120673 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.120655    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"} err="failed to get container status \"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f\": rpc error: code = NotFound desc = could not find container \"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f\": container with ID starting with 44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f not found: ID does not exist"
May 28 18:50:04.120734 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.120674    2577 scope.go:117] "RemoveContainer" containerID="2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"
May 28 18:50:04.120834 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.120817    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"} err="failed to get container status \"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25\": rpc error: code = NotFound desc = could not find container \"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25\": container with ID starting with 2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25 not found: ID does not exist"
May 28 18:50:04.120877 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.120836    2577 scope.go:117] "RemoveContainer" containerID="f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"
May 28 18:50:04.121039 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.121020    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"} err="failed to get container status \"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8\": rpc error: code = NotFound desc = could not find container \"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8\": container with ID starting with f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8 not found: ID does not exist"
May 28 18:50:04.121082 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.121039    2577 scope.go:117] "RemoveContainer" containerID="ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"
May 28 18:50:04.121236 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.121219    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"} err="failed to get container status \"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a\": rpc error: code = NotFound desc = could not find container \"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a\": container with ID starting with ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a not found: ID does not exist"
May 28 18:50:04.121282 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.121235    2577 scope.go:117] "RemoveContainer" containerID="0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"
May 28 18:50:04.121446 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.121429    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"} err="failed to get container status \"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40\": rpc error: code = NotFound desc = could not find container \"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40\": container with ID starting with 0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40 not found: ID does not exist"
May 28 18:50:04.121446 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.121445    2577 scope.go:117] "RemoveContainer" containerID="340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"
May 28 18:50:04.121767 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.121748    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"} err="failed to get container status \"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a\": rpc error: code = NotFound desc = could not find container \"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a\": container with ID starting with 340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a not found: ID does not exist"
May 28 18:50:04.121767 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.121763    2577 scope.go:117] "RemoveContainer" containerID="b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"
May 28 18:50:04.121933 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.121917    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"} err="failed to get container status \"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9\": rpc error: code = NotFound desc = could not find container \"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9\": container with ID starting with b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9 not found: ID does not exist"
May 28 18:50:04.121933 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.121932    2577 scope.go:117] "RemoveContainer" containerID="44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"
May 28 18:50:04.122011 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.121939    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.122121 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.122092    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"} err="failed to get container status \"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f\": rpc error: code = NotFound desc = could not find container \"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f\": container with ID starting with 44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f not found: ID does not exist"
May 28 18:50:04.122121 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.122118    2577 scope.go:117] "RemoveContainer" containerID="2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"
May 28 18:50:04.122495 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.122473    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"} err="failed to get container status \"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25\": rpc error: code = NotFound desc = could not find container \"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25\": container with ID starting with 2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25 not found: ID does not exist"
May 28 18:50:04.122495 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.122495    2577 scope.go:117] "RemoveContainer" containerID="f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"
May 28 18:50:04.122775 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.122755    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"} err="failed to get container status \"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8\": rpc error: code = NotFound desc = could not find container \"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8\": container with ID starting with f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8 not found: ID does not exist"
May 28 18:50:04.122819 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.122777    2577 scope.go:117] "RemoveContainer" containerID="ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"
May 28 18:50:04.122982 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.122963    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"} err="failed to get container status \"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a\": rpc error: code = NotFound desc = could not find container \"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a\": container with ID starting with ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a not found: ID does not exist"
May 28 18:50:04.123054 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.122984    2577 scope.go:117] "RemoveContainer" containerID="0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"
May 28 18:50:04.123196 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.123180    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"} err="failed to get container status \"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40\": rpc error: code = NotFound desc = could not find container \"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40\": container with ID starting with 0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40 not found: ID does not exist"
May 28 18:50:04.123263 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.123198    2577 scope.go:117] "RemoveContainer" containerID="340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"
May 28 18:50:04.123389 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.123371    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a"} err="failed to get container status \"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a\": rpc error: code = NotFound desc = could not find container \"340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a\": container with ID starting with 340a873a08b64d2c9941b7d6553e2577b1f24a75fc27535c6c8b59b193eba46a not found: ID does not exist"
May 28 18:50:04.123454 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.123392    2577 scope.go:117] "RemoveContainer" containerID="b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"
May 28 18:50:04.123629 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.123591    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9"} err="failed to get container status \"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9\": rpc error: code = NotFound desc = could not find container \"b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9\": container with ID starting with b2d1c63208bfc19208fbe5817ef14da180ec96b32e56b75b59af16bf50553ff9 not found: ID does not exist"
May 28 18:50:04.123629 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.123623    2577 scope.go:117] "RemoveContainer" containerID="44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"
May 28 18:50:04.123892 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.123876    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f"} err="failed to get container status \"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f\": rpc error: code = NotFound desc = could not find container \"44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f\": container with ID starting with 44cee189064eef8f5b49f752fe4191b266ce41a26da8bccdd8a572402b363b0f not found: ID does not exist"
May 28 18:50:04.123892 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.123891    2577 scope.go:117] "RemoveContainer" containerID="2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"
May 28 18:50:04.124060 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.124046    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25"} err="failed to get container status \"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25\": rpc error: code = NotFound desc = could not find container \"2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25\": container with ID starting with 2e91bde0ea29f42dcb13cfb1108c3b63140026f33a4479dccfe736e0e3537b25 not found: ID does not exist"
May 28 18:50:04.124108 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.124060    2577 scope.go:117] "RemoveContainer" containerID="f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"
May 28 18:50:04.124276 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.124257    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8"} err="failed to get container status \"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8\": rpc error: code = NotFound desc = could not find container \"f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8\": container with ID starting with f8d2965be28938083fab3e535176681a9771c49150677728c38ffa88a6e39ad8 not found: ID does not exist"
May 28 18:50:04.124323 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.124277    2577 scope.go:117] "RemoveContainer" containerID="ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"
May 28 18:50:04.124480 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.124459    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a"} err="failed to get container status \"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a\": rpc error: code = NotFound desc = could not find container \"ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a\": container with ID starting with ac26e986b1ca3adea275479a75d9e5e825c5d692c9b60e17ca4b4e1d75f2c73a not found: ID does not exist"
May 28 18:50:04.124480 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.124477    2577 scope.go:117] "RemoveContainer" containerID="0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"
May 28 18:50:04.124700 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.124680    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 28 18:50:04.124812 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.124793    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40"} err="failed to get container status \"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40\": rpc error: code = NotFound desc = could not find container \"0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40\": container with ID starting with 0b4b1a1427c2d43ac66b95da6f5d3f1d271a92e5adf8d183195417d2bbbf4c40 not found: ID does not exist"
May 28 18:50:04.126041 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.125978    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-egtupvir1lvri\""
May 28 18:50:04.126041 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.126027    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 28 18:50:04.126193 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.126051    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8vfhd\""
May 28 18:50:04.126193 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.126105    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 28 18:50:04.126193 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.126124    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 28 18:50:04.126193 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.126054    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 28 18:50:04.126368 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.126345    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 28 18:50:04.126417 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.126382    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 28 18:50:04.126493 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.126479    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 28 18:50:04.126553 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.126509    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 28 18:50:04.127577 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.127561    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 28 18:50:04.133283 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.133259    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 28 18:50:04.136562 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.136537    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 28 18:50:04.138101 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.138070    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 18:50:04.142348 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142328    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142450 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142351    2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-grpc-tls\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142450 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142367    2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-metrics-client-certs\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142450 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142383    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142450 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142398    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142450 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142411    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4d8e0b49-aa50-4c67-a189-41497d66fb5a-prometheus-k8s-db\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142450 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142424    2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142450 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142437    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d8e0b49-aa50-4c67-a189-41497d66fb5a-configmap-metrics-client-ca\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142450 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142450    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d8e0b49-aa50-4c67-a189-41497d66fb5a-config-out\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142758 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142463    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-web-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142758 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142476    2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-kube-rbac-proxy\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142758 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142490    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d8e0b49-aa50-4c67-a189-41497d66fb5a-tls-assets\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142758 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142506    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-secret-prometheus-k8s-tls\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142758 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142519    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l2mb\" (UniqueName: \"kubernetes.io/projected/4d8e0b49-aa50-4c67-a189-41497d66fb5a-kube-api-access-4l2mb\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.142758 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.142534    2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4d8e0b49-aa50-4c67-a189-41497d66fb5a-thanos-prometheus-http-client-file\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:50:04.243488 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.243692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243492    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.243692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243531    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.243692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243554    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.243692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243579    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.243692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.243692 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243677    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-config\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.244005 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243696    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.244005 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243754    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.244005 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243807    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf6gw\" (UniqueName: \"kubernetes.io/projected/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-kube-api-access-gf6gw\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.244005 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243856    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.244005 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.244005 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-web-config\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.244005 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.244005 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.243991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.244301 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.244020    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.244301 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.244054    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.244301 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.244086    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-config-out\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345237 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345140    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345237 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345185    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345436 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345311    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345436 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345357    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345436 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345391    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345436 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345418    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345657 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345452    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-config\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345657 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345579    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345768 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345663    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345768 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345702    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345768 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345728    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gf6gw\" (UniqueName: \"kubernetes.io/projected/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-kube-api-access-gf6gw\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345932 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345775    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345932 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345809    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345932 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345857    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-web-config\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345932 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345884    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.345932 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345921    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.346176 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345953    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.346176 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.345996    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.346176 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.346036    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-config-out\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.346328 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.346180    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.346328 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.346251    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.347287 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.347223    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.348484 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.348454    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-config-out\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.348603 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.348549    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-config\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.348603 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.348591    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.348760 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.348689    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.349079 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.348980    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.349178 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.349108    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.349244 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.349210    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.349480 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.349402    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.349869 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.349849    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.350623 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.350596    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.351108 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.351093    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-web-config\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.351437 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.351422    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.352022 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.352004    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.356543 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.356524    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf6gw\" (UniqueName: \"kubernetes.io/projected/57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc-kube-api-access-gf6gw\") pod \"prometheus-k8s-0\" (UID: \"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc\") " pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.358574 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.358551    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d8e0b49-aa50-4c67-a189-41497d66fb5a" path="/var/lib/kubelet/pods/4d8e0b49-aa50-4c67-a189-41497d66fb5a/volumes"
May 28 18:50:04.432173 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.432139    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:04.567363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:04.567335    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 28 18:50:04.569650 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:50:04.569621    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ee20ae_0bc5_4acb_bf1a_43c0d0d4f3cc.slice/crio-7d995a04d9716ffb84510acf22b386ca3e483eb4028c21bc81df3ee53acad658 WatchSource:0}: Error finding container 7d995a04d9716ffb84510acf22b386ca3e483eb4028c21bc81df3ee53acad658: Status 404 returned error can't find the container with id 7d995a04d9716ffb84510acf22b386ca3e483eb4028c21bc81df3ee53acad658
May 28 18:50:05.063484 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:05.063443    2577 generic.go:358] "Generic (PLEG): container finished" podID="57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc" containerID="b17724223037be9d9f5b109b180e898fcb7556d9499f313299d60a32e56f4c13" exitCode=0
May 28 18:50:05.063932 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:05.063528    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc","Type":"ContainerDied","Data":"b17724223037be9d9f5b109b180e898fcb7556d9499f313299d60a32e56f4c13"}
May 28 18:50:05.063932 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:05.063568    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc","Type":"ContainerStarted","Data":"7d995a04d9716ffb84510acf22b386ca3e483eb4028c21bc81df3ee53acad658"}
May 28 18:50:06.069469 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:06.069431    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc","Type":"ContainerStarted","Data":"da96118b32782eafe820dfc0500ad50d45a385129a59fbee261c4bd940c13efe"}
May 28 18:50:06.069469 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:06.069470    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc","Type":"ContainerStarted","Data":"288da2a5987e0720f679712ae425f2c9f3430adb25dde33fc48e773930bc0ccc"}
May 28 18:50:06.069959 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:06.069486    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc","Type":"ContainerStarted","Data":"148c20ddcb6c844652172413f68089db2b7bd80a97756a67eb7efa2929346002"}
May 28 18:50:06.069959 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:06.069498    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc","Type":"ContainerStarted","Data":"55369769350e55d2ccf721bb18b747939543c64f1479fc6a43eba691e3e62d43"}
May 28 18:50:06.069959 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:06.069509    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc","Type":"ContainerStarted","Data":"52e4640de5c50138c4ea6e114ffa46a0d472898f2bc6e80ab176f4775aa50eca"}
May 28 18:50:06.069959 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:06.069519    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc","Type":"ContainerStarted","Data":"7ec1e8fc5c7701c2888e2acd237c685f4f7dc6320328dff9f28d8980637d2008"}
May 28 18:50:06.097398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:06.097343    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.097328342 podStartE2EDuration="2.097328342s" podCreationTimestamp="2026-05-28 18:50:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 18:50:06.095945987 +0000 UTC m=+190.355168154" watchObservedRunningTime="2026-05-28 18:50:06.097328342 +0000 UTC m=+190.356550529"
May 28 18:50:09.432318 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:09.432279    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:50:55.371410 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.371367    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-bd8c2"]
May 28 18:50:55.373848 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.373831    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bd8c2"
May 28 18:50:55.376510 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.376487    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 18:50:55.381603 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.381277    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bd8c2"]
May 28 18:50:55.442989 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.442942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5474ed71-aeca-4921-9680-b338954bf5fd-original-pull-secret\") pod \"global-pull-secret-syncer-bd8c2\" (UID: \"5474ed71-aeca-4921-9680-b338954bf5fd\") " pod="kube-system/global-pull-secret-syncer-bd8c2"
May 28 18:50:55.442989 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.442994    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5474ed71-aeca-4921-9680-b338954bf5fd-dbus\") pod \"global-pull-secret-syncer-bd8c2\" (UID: \"5474ed71-aeca-4921-9680-b338954bf5fd\") " pod="kube-system/global-pull-secret-syncer-bd8c2"
May 28 18:50:55.443195 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.443016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5474ed71-aeca-4921-9680-b338954bf5fd-kubelet-config\") pod \"global-pull-secret-syncer-bd8c2\" (UID: \"5474ed71-aeca-4921-9680-b338954bf5fd\") " pod="kube-system/global-pull-secret-syncer-bd8c2"
May 28 18:50:55.544106 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.544064    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5474ed71-aeca-4921-9680-b338954bf5fd-original-pull-secret\") pod \"global-pull-secret-syncer-bd8c2\" (UID: \"5474ed71-aeca-4921-9680-b338954bf5fd\") " pod="kube-system/global-pull-secret-syncer-bd8c2"
May 28 18:50:55.544106 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.544111    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5474ed71-aeca-4921-9680-b338954bf5fd-dbus\") pod \"global-pull-secret-syncer-bd8c2\" (UID: \"5474ed71-aeca-4921-9680-b338954bf5fd\") " pod="kube-system/global-pull-secret-syncer-bd8c2"
May 28 18:50:55.544315 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.544242    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5474ed71-aeca-4921-9680-b338954bf5fd-dbus\") pod \"global-pull-secret-syncer-bd8c2\" (UID: \"5474ed71-aeca-4921-9680-b338954bf5fd\") " pod="kube-system/global-pull-secret-syncer-bd8c2"
May 28 18:50:55.544315 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.544286    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5474ed71-aeca-4921-9680-b338954bf5fd-kubelet-config\") pod \"global-pull-secret-syncer-bd8c2\" (UID: \"5474ed71-aeca-4921-9680-b338954bf5fd\") " pod="kube-system/global-pull-secret-syncer-bd8c2"
May 28 18:50:55.544378 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.544360    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5474ed71-aeca-4921-9680-b338954bf5fd-kubelet-config\") pod \"global-pull-secret-syncer-bd8c2\" (UID: \"5474ed71-aeca-4921-9680-b338954bf5fd\") " pod="kube-system/global-pull-secret-syncer-bd8c2"
May 28 18:50:55.546378 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.546359    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5474ed71-aeca-4921-9680-b338954bf5fd-original-pull-secret\") pod \"global-pull-secret-syncer-bd8c2\" (UID: \"5474ed71-aeca-4921-9680-b338954bf5fd\") " pod="kube-system/global-pull-secret-syncer-bd8c2"
May 28 18:50:55.684316 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.684280    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bd8c2"
May 28 18:50:55.808377 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:55.808351    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bd8c2"]
May 28 18:50:55.810627 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:50:55.810584    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5474ed71_aeca_4921_9680_b338954bf5fd.slice/crio-de92b675039cda89fc473c22d77c8623b32e38cc50c538988ac9575241c67067 WatchSource:0}: Error finding container de92b675039cda89fc473c22d77c8623b32e38cc50c538988ac9575241c67067: Status 404 returned error can't find the container with id de92b675039cda89fc473c22d77c8623b32e38cc50c538988ac9575241c67067
May 28 18:50:56.228127 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:50:56.228092    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bd8c2" event={"ID":"5474ed71-aeca-4921-9680-b338954bf5fd","Type":"ContainerStarted","Data":"de92b675039cda89fc473c22d77c8623b32e38cc50c538988ac9575241c67067"}
May 28 18:51:01.247792 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:01.247749    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bd8c2" event={"ID":"5474ed71-aeca-4921-9680-b338954bf5fd","Type":"ContainerStarted","Data":"5a59f9bc608f9aad3569cb5847701cec326c70d719cd3c60626a1bc321cb1b83"}
May 28 18:51:01.261978 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:01.261909    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bd8c2" podStartSLOduration=1.750462133 podStartE2EDuration="6.261892861s" podCreationTimestamp="2026-05-28 18:50:55 +0000 UTC" firstStartedPulling="2026-05-28 18:50:55.812156963 +0000 UTC m=+240.071379111" lastFinishedPulling="2026-05-28 18:51:00.323587696 +0000 UTC m=+244.582809839" observedRunningTime="2026-05-28 18:51:01.26181263 +0000 UTC m=+245.521034797" watchObservedRunningTime="2026-05-28 18:51:01.261892861 +0000 UTC m=+245.521115027"
May 28 18:51:04.432418 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:04.432361    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:51:04.448076 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:04.448049    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:51:05.274249 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:05.274214    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 28 18:51:56.243532 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:56.243502    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 18:51:56.243532 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:56.243525    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 18:51:56.256010 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:56.255980    2577 kubelet.go:1628] "Image garbage collection succeeded"
May 28 18:51:57.296772 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.296736    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-w8wv6"]
May 28 18:51:57.298943 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.298927    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-w8wv6"
May 28 18:51:57.301751 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.301724    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
May 28 18:51:57.302819 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.302797    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-pwcj9\""
May 28 18:51:57.302923 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.302807    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
May 28 18:51:57.311217 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.311191    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-w8wv6"]
May 28 18:51:57.368666 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.368633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9-bound-sa-token\") pod \"cert-manager-79c8d999ff-w8wv6\" (UID: \"d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9\") " pod="cert-manager/cert-manager-79c8d999ff-w8wv6"
May 28 18:51:57.368840 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.368690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4ttc\" (UniqueName: \"kubernetes.io/projected/d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9-kube-api-access-n4ttc\") pod \"cert-manager-79c8d999ff-w8wv6\" (UID: \"d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9\") " pod="cert-manager/cert-manager-79c8d999ff-w8wv6"
May 28 18:51:57.470109 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.470068    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9-bound-sa-token\") pod \"cert-manager-79c8d999ff-w8wv6\" (UID: \"d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9\") " pod="cert-manager/cert-manager-79c8d999ff-w8wv6"
May 28 18:51:57.470326 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.470174    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n4ttc\" (UniqueName: \"kubernetes.io/projected/d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9-kube-api-access-n4ttc\") pod \"cert-manager-79c8d999ff-w8wv6\" (UID: \"d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9\") " pod="cert-manager/cert-manager-79c8d999ff-w8wv6"
May 28 18:51:57.477651 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.477627    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9-bound-sa-token\") pod \"cert-manager-79c8d999ff-w8wv6\" (UID: \"d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9\") " pod="cert-manager/cert-manager-79c8d999ff-w8wv6"
May 28 18:51:57.477780 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.477695    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4ttc\" (UniqueName: \"kubernetes.io/projected/d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9-kube-api-access-n4ttc\") pod \"cert-manager-79c8d999ff-w8wv6\" (UID: \"d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9\") " pod="cert-manager/cert-manager-79c8d999ff-w8wv6"
May 28 18:51:57.618218 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.618111    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-w8wv6"
May 28 18:51:57.743694 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.743586    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-w8wv6"]
May 28 18:51:57.746674 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:51:57.746646    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd56ce9bb_eab7_412e_ba8c_20b1ace3dcc9.slice/crio-3149b204811e9d71c19827377c2844cef4b0bf354f6700ee232dc3c7008400f8 WatchSource:0}: Error finding container 3149b204811e9d71c19827377c2844cef4b0bf354f6700ee232dc3c7008400f8: Status 404 returned error can't find the container with id 3149b204811e9d71c19827377c2844cef4b0bf354f6700ee232dc3c7008400f8
May 28 18:51:57.748894 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:57.748878    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 18:51:58.410219 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:51:58.410181    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-w8wv6" event={"ID":"d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9","Type":"ContainerStarted","Data":"3149b204811e9d71c19827377c2844cef4b0bf354f6700ee232dc3c7008400f8"}
May 28 18:52:01.422262 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:01.422223    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-w8wv6" event={"ID":"d56ce9bb-eab7-412e-ba8c-20b1ace3dcc9","Type":"ContainerStarted","Data":"2cb39be03678364fa0d9d0193e324a008dc757d9609c06b14f9a1f14819e0122"}
May 28 18:52:01.438476 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:01.438365    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-w8wv6" podStartSLOduration=1.004102234 podStartE2EDuration="4.438347878s" podCreationTimestamp="2026-05-28 18:51:57 +0000 UTC" firstStartedPulling="2026-05-28 18:51:57.749009868 +0000 UTC m=+302.008232012" lastFinishedPulling="2026-05-28 18:52:01.183255512 +0000 UTC m=+305.442477656" observedRunningTime="2026-05-28 18:52:01.437165594 +0000 UTC m=+305.696387760" watchObservedRunningTime="2026-05-28 18:52:01.438347878 +0000 UTC m=+305.697570044"
May 28 18:52:15.363534 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.363489    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"]
May 28 18:52:15.366852 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.366830    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"
May 28 18:52:15.370430 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.370411    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
May 28 18:52:15.370714 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.370698    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gtp98\""
May 28 18:52:15.371008 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.370994    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
May 28 18:52:15.371057 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.370996    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
May 28 18:52:15.371301 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.371288    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
May 28 18:52:15.383957 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.383933    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"]
May 28 18:52:15.523893 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.523857    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lgbk\" (UniqueName: \"kubernetes.io/projected/62ae530e-ee79-4380-98a0-4ef991090cef-kube-api-access-7lgbk\") pod \"opendatahub-operator-controller-manager-75cc899db8-kpb7q\" (UID: \"62ae530e-ee79-4380-98a0-4ef991090cef\") " pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"
May 28 18:52:15.524075 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.523901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62ae530e-ee79-4380-98a0-4ef991090cef-webhook-cert\") pod \"opendatahub-operator-controller-manager-75cc899db8-kpb7q\" (UID: \"62ae530e-ee79-4380-98a0-4ef991090cef\") " pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"
May 28 18:52:15.524075 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.523924    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/62ae530e-ee79-4380-98a0-4ef991090cef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75cc899db8-kpb7q\" (UID: \"62ae530e-ee79-4380-98a0-4ef991090cef\") " pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"
May 28 18:52:15.624962 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.624871    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7lgbk\" (UniqueName: \"kubernetes.io/projected/62ae530e-ee79-4380-98a0-4ef991090cef-kube-api-access-7lgbk\") pod \"opendatahub-operator-controller-manager-75cc899db8-kpb7q\" (UID: \"62ae530e-ee79-4380-98a0-4ef991090cef\") " pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"
May 28 18:52:15.624962 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.624915    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62ae530e-ee79-4380-98a0-4ef991090cef-webhook-cert\") pod \"opendatahub-operator-controller-manager-75cc899db8-kpb7q\" (UID: \"62ae530e-ee79-4380-98a0-4ef991090cef\") " pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"
May 28 18:52:15.624962 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.624934    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/62ae530e-ee79-4380-98a0-4ef991090cef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75cc899db8-kpb7q\" (UID: \"62ae530e-ee79-4380-98a0-4ef991090cef\") " pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"
May 28 18:52:15.627419 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.627389    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62ae530e-ee79-4380-98a0-4ef991090cef-webhook-cert\") pod \"opendatahub-operator-controller-manager-75cc899db8-kpb7q\" (UID: \"62ae530e-ee79-4380-98a0-4ef991090cef\") " pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"
May 28 18:52:15.627529 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.627421    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/62ae530e-ee79-4380-98a0-4ef991090cef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75cc899db8-kpb7q\" (UID: \"62ae530e-ee79-4380-98a0-4ef991090cef\") " pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"
May 28 18:52:15.633824 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.633798    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lgbk\" (UniqueName: \"kubernetes.io/projected/62ae530e-ee79-4380-98a0-4ef991090cef-kube-api-access-7lgbk\") pod \"opendatahub-operator-controller-manager-75cc899db8-kpb7q\" (UID: \"62ae530e-ee79-4380-98a0-4ef991090cef\") " pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"
May 28 18:52:15.676545 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.676518    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"
May 28 18:52:15.806117 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:15.806074    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"]
May 28 18:52:15.809418 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:52:15.809390    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ae530e_ee79_4380_98a0_4ef991090cef.slice/crio-baa6080e69a4e14f4adb9107acd49584534b27ace6485476bf0b8d68ef1c6ac4 WatchSource:0}: Error finding container baa6080e69a4e14f4adb9107acd49584534b27ace6485476bf0b8d68ef1c6ac4: Status 404 returned error can't find the container with id baa6080e69a4e14f4adb9107acd49584534b27ace6485476bf0b8d68ef1c6ac4
May 28 18:52:16.472460 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:16.472417    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q" event={"ID":"62ae530e-ee79-4380-98a0-4ef991090cef","Type":"ContainerStarted","Data":"baa6080e69a4e14f4adb9107acd49584534b27ace6485476bf0b8d68ef1c6ac4"}
May 28 18:52:18.480966 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:18.480917    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q" event={"ID":"62ae530e-ee79-4380-98a0-4ef991090cef","Type":"ContainerStarted","Data":"04526b5209e6e7e1d3ccc4a1714f43064c08cb41b7521a98a0224842dbeff31b"}
May 28 18:52:18.481365 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:18.481055    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"
May 28 18:52:18.502214 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:18.502163    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q" podStartSLOduration=1.049336378 podStartE2EDuration="3.502148345s" podCreationTimestamp="2026-05-28 18:52:15 +0000 UTC" firstStartedPulling="2026-05-28 18:52:15.811176246 +0000 UTC m=+320.070398390" lastFinishedPulling="2026-05-28 18:52:18.263988202 +0000 UTC m=+322.523210357" observedRunningTime="2026-05-28 18:52:18.50151557 +0000 UTC m=+322.760737735" watchObservedRunningTime="2026-05-28 18:52:18.502148345 +0000 UTC m=+322.761370511"
May 28 18:52:29.486781 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:29.486749    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-75cc899db8-kpb7q"
May 28 18:52:32.772691 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.772656    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"]
May 28 18:52:32.777372 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.777352    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"
May 28 18:52:32.780333 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.780310    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 28 18:52:32.780663 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.780645    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
May 28 18:52:32.780751 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.780652    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
May 28 18:52:32.781542 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.781517    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-62tds\""
May 28 18:52:32.781638 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.781525    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 28 18:52:32.791279 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.791259    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"]
May 28 18:52:32.879424 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.879375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bca077fc-d6dc-4412-aa92-c606a758230a-tls-certs\") pod \"kube-auth-proxy-5cf74c98fc-vmhgz\" (UID: \"bca077fc-d6dc-4412-aa92-c606a758230a\") " pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"
May 28 18:52:32.879602 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.879498    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mf9z\" (UniqueName: \"kubernetes.io/projected/bca077fc-d6dc-4412-aa92-c606a758230a-kube-api-access-5mf9z\") pod \"kube-auth-proxy-5cf74c98fc-vmhgz\" (UID: \"bca077fc-d6dc-4412-aa92-c606a758230a\") " pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"
May 28 18:52:32.879602 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.879540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bca077fc-d6dc-4412-aa92-c606a758230a-tmp\") pod \"kube-auth-proxy-5cf74c98fc-vmhgz\" (UID: \"bca077fc-d6dc-4412-aa92-c606a758230a\") " pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"
May 28 18:52:32.981007 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.980965    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bca077fc-d6dc-4412-aa92-c606a758230a-tls-certs\") pod \"kube-auth-proxy-5cf74c98fc-vmhgz\" (UID: \"bca077fc-d6dc-4412-aa92-c606a758230a\") " pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"
May 28 18:52:32.981213 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.981042    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5mf9z\" (UniqueName: \"kubernetes.io/projected/bca077fc-d6dc-4412-aa92-c606a758230a-kube-api-access-5mf9z\") pod \"kube-auth-proxy-5cf74c98fc-vmhgz\" (UID: \"bca077fc-d6dc-4412-aa92-c606a758230a\") " pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"
May 28 18:52:32.981213 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.981076    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bca077fc-d6dc-4412-aa92-c606a758230a-tmp\") pod \"kube-auth-proxy-5cf74c98fc-vmhgz\" (UID: \"bca077fc-d6dc-4412-aa92-c606a758230a\") " pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"
May 28 18:52:32.983380 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.983355    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bca077fc-d6dc-4412-aa92-c606a758230a-tmp\") pod \"kube-auth-proxy-5cf74c98fc-vmhgz\" (UID: \"bca077fc-d6dc-4412-aa92-c606a758230a\") " pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"
May 28 18:52:32.983675 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.983659    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bca077fc-d6dc-4412-aa92-c606a758230a-tls-certs\") pod \"kube-auth-proxy-5cf74c98fc-vmhgz\" (UID: \"bca077fc-d6dc-4412-aa92-c606a758230a\") " pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"
May 28 18:52:32.989156 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:32.989132    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mf9z\" (UniqueName: \"kubernetes.io/projected/bca077fc-d6dc-4412-aa92-c606a758230a-kube-api-access-5mf9z\") pod \"kube-auth-proxy-5cf74c98fc-vmhgz\" (UID: \"bca077fc-d6dc-4412-aa92-c606a758230a\") " pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"
May 28 18:52:33.086788 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:33.086701    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"
May 28 18:52:33.213309 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:33.213275    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz"]
May 28 18:52:33.216994 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:52:33.216958    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca077fc_d6dc_4412_aa92_c606a758230a.slice/crio-02d593179c90a8d256111ca8d5c4146450e3a85c804b36fb824d34bcbd0fc8fa WatchSource:0}: Error finding container 02d593179c90a8d256111ca8d5c4146450e3a85c804b36fb824d34bcbd0fc8fa: Status 404 returned error can't find the container with id 02d593179c90a8d256111ca8d5c4146450e3a85c804b36fb824d34bcbd0fc8fa
May 28 18:52:33.537100 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:33.537062    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz" event={"ID":"bca077fc-d6dc-4412-aa92-c606a758230a","Type":"ContainerStarted","Data":"02d593179c90a8d256111ca8d5c4146450e3a85c804b36fb824d34bcbd0fc8fa"}
May 28 18:52:35.979892 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:35.979855    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-jtf76"]
May 28 18:52:35.982186 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:35.982168    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76"
May 28 18:52:35.984683 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:35.984663    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
May 28 18:52:35.984782 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:35.984709    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-sr8zs\""
May 28 18:52:35.993484 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:35.993462    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-jtf76"]
May 28 18:52:36.004446 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:36.004422    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b033be53-342a-4f0d-a7da-76327e02eef5-cert\") pod \"odh-model-controller-858dbf95b8-jtf76\" (UID: \"b033be53-342a-4f0d-a7da-76327e02eef5\") " pod="opendatahub/odh-model-controller-858dbf95b8-jtf76"
May 28 18:52:36.004579 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:36.004460    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrmk9\" (UniqueName: \"kubernetes.io/projected/b033be53-342a-4f0d-a7da-76327e02eef5-kube-api-access-hrmk9\") pod \"odh-model-controller-858dbf95b8-jtf76\" (UID: \"b033be53-342a-4f0d-a7da-76327e02eef5\") " pod="opendatahub/odh-model-controller-858dbf95b8-jtf76"
May 28 18:52:36.106033 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:36.105993    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b033be53-342a-4f0d-a7da-76327e02eef5-cert\") pod \"odh-model-controller-858dbf95b8-jtf76\" (UID: \"b033be53-342a-4f0d-a7da-76327e02eef5\") " pod="opendatahub/odh-model-controller-858dbf95b8-jtf76"
May 28 18:52:36.106233 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:36.106042    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hrmk9\" (UniqueName: \"kubernetes.io/projected/b033be53-342a-4f0d-a7da-76327e02eef5-kube-api-access-hrmk9\") pod \"odh-model-controller-858dbf95b8-jtf76\" (UID: \"b033be53-342a-4f0d-a7da-76327e02eef5\") " pod="opendatahub/odh-model-controller-858dbf95b8-jtf76"
May 28 18:52:36.106233 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:52:36.106160    2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
May 28 18:52:36.106349 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:52:36.106237    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b033be53-342a-4f0d-a7da-76327e02eef5-cert podName:b033be53-342a-4f0d-a7da-76327e02eef5 nodeName:}" failed. No retries permitted until 2026-05-28 18:52:36.606215338 +0000 UTC m=+340.865437487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b033be53-342a-4f0d-a7da-76327e02eef5-cert") pod "odh-model-controller-858dbf95b8-jtf76" (UID: "b033be53-342a-4f0d-a7da-76327e02eef5") : secret "odh-model-controller-webhook-cert" not found
May 28 18:52:36.115704 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:36.115670    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrmk9\" (UniqueName: \"kubernetes.io/projected/b033be53-342a-4f0d-a7da-76327e02eef5-kube-api-access-hrmk9\") pod \"odh-model-controller-858dbf95b8-jtf76\" (UID: \"b033be53-342a-4f0d-a7da-76327e02eef5\") " pod="opendatahub/odh-model-controller-858dbf95b8-jtf76"
May 28 18:52:36.610499 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:36.610445    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b033be53-342a-4f0d-a7da-76327e02eef5-cert\") pod \"odh-model-controller-858dbf95b8-jtf76\" (UID: \"b033be53-342a-4f0d-a7da-76327e02eef5\") " pod="opendatahub/odh-model-controller-858dbf95b8-jtf76"
May 28 18:52:36.613574 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:36.613543    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b033be53-342a-4f0d-a7da-76327e02eef5-cert\") pod \"odh-model-controller-858dbf95b8-jtf76\" (UID: \"b033be53-342a-4f0d-a7da-76327e02eef5\") " pod="opendatahub/odh-model-controller-858dbf95b8-jtf76"
May 28 18:52:36.893402 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:36.893362    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76"
May 28 18:52:37.081708 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:37.081630    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-jtf76"]
May 28 18:52:37.083813 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:52:37.083786    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb033be53_342a_4f0d_a7da_76327e02eef5.slice/crio-74b06c52b6bd1a914545a1d3f58ccaf1d464a7104289553d63b15739c7bf1991 WatchSource:0}: Error finding container 74b06c52b6bd1a914545a1d3f58ccaf1d464a7104289553d63b15739c7bf1991: Status 404 returned error can't find the container with id 74b06c52b6bd1a914545a1d3f58ccaf1d464a7104289553d63b15739c7bf1991
May 28 18:52:37.554348 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:37.554308    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz" event={"ID":"bca077fc-d6dc-4412-aa92-c606a758230a","Type":"ContainerStarted","Data":"dd7e5a8ae439a946596eb25fcc045da85a6e98dd33bd13b9772a4e74e57238f9"}
May 28 18:52:37.555363 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:37.555342    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76" event={"ID":"b033be53-342a-4f0d-a7da-76327e02eef5","Type":"ContainerStarted","Data":"74b06c52b6bd1a914545a1d3f58ccaf1d464a7104289553d63b15739c7bf1991"}
May 28 18:52:37.573512 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:37.573470    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5cf74c98fc-vmhgz" podStartSLOduration=1.814121664 podStartE2EDuration="5.573458361s" podCreationTimestamp="2026-05-28 18:52:32 +0000 UTC" firstStartedPulling="2026-05-28 18:52:33.219150831 +0000 UTC m=+337.478372974" lastFinishedPulling="2026-05-28 18:52:36.978487517 +0000 UTC m=+341.237709671" observedRunningTime="2026-05-28 18:52:37.571467351 +0000 UTC m=+341.830689520" watchObservedRunningTime="2026-05-28 18:52:37.573458361 +0000 UTC m=+341.832680527"
May 28 18:52:40.568509 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:40.568471    2577 generic.go:358] "Generic (PLEG): container finished" podID="b033be53-342a-4f0d-a7da-76327e02eef5" containerID="8a6180ee359250e608271e0ad82fb7f4e72ed6b211b8ad6e6faae7919f75edb5" exitCode=1
May 28 18:52:40.568920 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:40.568522    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76" event={"ID":"b033be53-342a-4f0d-a7da-76327e02eef5","Type":"ContainerDied","Data":"8a6180ee359250e608271e0ad82fb7f4e72ed6b211b8ad6e6faae7919f75edb5"}
May 28 18:52:40.568920 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:40.568769    2577 scope.go:117] "RemoveContainer" containerID="8a6180ee359250e608271e0ad82fb7f4e72ed6b211b8ad6e6faae7919f75edb5"
May 28 18:52:41.573267 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:41.573230    2577 generic.go:358] "Generic (PLEG): container finished" podID="b033be53-342a-4f0d-a7da-76327e02eef5" containerID="e7e214daa2ca91d3bece147cb2b1e1b232ed2845d652d0c3159c3d018b768649" exitCode=1
May 28 18:52:41.573783 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:41.573318    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76" event={"ID":"b033be53-342a-4f0d-a7da-76327e02eef5","Type":"ContainerDied","Data":"e7e214daa2ca91d3bece147cb2b1e1b232ed2845d652d0c3159c3d018b768649"}
May 28 18:52:41.573783 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:41.573369    2577 scope.go:117] "RemoveContainer" containerID="8a6180ee359250e608271e0ad82fb7f4e72ed6b211b8ad6e6faae7919f75edb5"
May 28 18:52:41.573783 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:41.573632    2577 scope.go:117] "RemoveContainer" containerID="e7e214daa2ca91d3bece147cb2b1e1b232ed2845d652d0c3159c3d018b768649"
May 28 18:52:41.573940 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:52:41.573860    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-jtf76_opendatahub(b033be53-342a-4f0d-a7da-76327e02eef5)\"" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76" podUID="b033be53-342a-4f0d-a7da-76327e02eef5"
May 28 18:52:41.966074 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:41.966043    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-gxxq7"]
May 28 18:52:41.970400 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:41.970379    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-gxxq7"
May 28 18:52:41.973291 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:41.973264    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
May 28 18:52:41.973490 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:41.973344    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-kb6mm\""
May 28 18:52:41.977950 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:41.977920    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-gxxq7"]
May 28 18:52:42.055703 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:42.055655    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz8fp\" (UniqueName: \"kubernetes.io/projected/ee73c29d-b2f3-4876-be80-8345b5bf2e71-kube-api-access-zz8fp\") pod \"kserve-controller-manager-856948b99f-gxxq7\" (UID: \"ee73c29d-b2f3-4876-be80-8345b5bf2e71\") " pod="opendatahub/kserve-controller-manager-856948b99f-gxxq7"
May 28 18:52:42.055703 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:42.055716    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee73c29d-b2f3-4876-be80-8345b5bf2e71-cert\") pod \"kserve-controller-manager-856948b99f-gxxq7\" (UID: \"ee73c29d-b2f3-4876-be80-8345b5bf2e71\") " pod="opendatahub/kserve-controller-manager-856948b99f-gxxq7"
May 28 18:52:42.156852 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:42.156819    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zz8fp\" (UniqueName: \"kubernetes.io/projected/ee73c29d-b2f3-4876-be80-8345b5bf2e71-kube-api-access-zz8fp\") pod \"kserve-controller-manager-856948b99f-gxxq7\" (UID: \"ee73c29d-b2f3-4876-be80-8345b5bf2e71\") " pod="opendatahub/kserve-controller-manager-856948b99f-gxxq7"
May 28 18:52:42.156852 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:42.156860    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee73c29d-b2f3-4876-be80-8345b5bf2e71-cert\") pod \"kserve-controller-manager-856948b99f-gxxq7\" (UID: \"ee73c29d-b2f3-4876-be80-8345b5bf2e71\") " pod="opendatahub/kserve-controller-manager-856948b99f-gxxq7"
May 28 18:52:42.159218 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:42.159196    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee73c29d-b2f3-4876-be80-8345b5bf2e71-cert\") pod \"kserve-controller-manager-856948b99f-gxxq7\" (UID: \"ee73c29d-b2f3-4876-be80-8345b5bf2e71\") " pod="opendatahub/kserve-controller-manager-856948b99f-gxxq7"
May 28 18:52:42.166775 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:42.166745    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz8fp\" (UniqueName: \"kubernetes.io/projected/ee73c29d-b2f3-4876-be80-8345b5bf2e71-kube-api-access-zz8fp\") pod \"kserve-controller-manager-856948b99f-gxxq7\" (UID: \"ee73c29d-b2f3-4876-be80-8345b5bf2e71\") " pod="opendatahub/kserve-controller-manager-856948b99f-gxxq7"
May 28 18:52:42.285453 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:42.285357    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-gxxq7"
May 28 18:52:42.416028 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:42.416004    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-gxxq7"]
May 28 18:52:42.418017 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:52:42.417989    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee73c29d_b2f3_4876_be80_8345b5bf2e71.slice/crio-42af8f96b409cfac3985a43a2deeaa95f44e612e95a88a6ae57697e66ec69520 WatchSource:0}: Error finding container 42af8f96b409cfac3985a43a2deeaa95f44e612e95a88a6ae57697e66ec69520: Status 404 returned error can't find the container with id 42af8f96b409cfac3985a43a2deeaa95f44e612e95a88a6ae57697e66ec69520
May 28 18:52:42.578637 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:42.578507    2577 scope.go:117] "RemoveContainer" containerID="e7e214daa2ca91d3bece147cb2b1e1b232ed2845d652d0c3159c3d018b768649"
May 28 18:52:42.579060 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:52:42.578725    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-jtf76_opendatahub(b033be53-342a-4f0d-a7da-76327e02eef5)\"" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76" podUID="b033be53-342a-4f0d-a7da-76327e02eef5"
May 28 18:52:42.579269 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:42.579246    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-gxxq7" event={"ID":"ee73c29d-b2f3-4876-be80-8345b5bf2e71","Type":"ContainerStarted","Data":"42af8f96b409cfac3985a43a2deeaa95f44e612e95a88a6ae57697e66ec69520"}
May 28 18:52:46.893744 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:46.893702    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76"
May 28 18:52:46.894247 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:46.894230    2577 scope.go:117] "RemoveContainer" containerID="e7e214daa2ca91d3bece147cb2b1e1b232ed2845d652d0c3159c3d018b768649"
May 28 18:52:46.894472 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:52:46.894451    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-jtf76_opendatahub(b033be53-342a-4f0d-a7da-76327e02eef5)\"" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76" podUID="b033be53-342a-4f0d-a7da-76327e02eef5"
May 28 18:52:47.598011 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:47.597971    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-gxxq7" event={"ID":"ee73c29d-b2f3-4876-be80-8345b5bf2e71","Type":"ContainerStarted","Data":"17171b7d0d8a5c6e4d83f879cba0b8efc83b2cd28423194183c84efc1509daf0"}
May 28 18:52:47.598209 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:47.598025    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-gxxq7"
May 28 18:52:47.616035 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:47.615966    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-gxxq7" podStartSLOduration=1.99119969 podStartE2EDuration="6.615946544s" podCreationTimestamp="2026-05-28 18:52:41 +0000 UTC" firstStartedPulling="2026-05-28 18:52:42.419852541 +0000 UTC m=+346.679074690" lastFinishedPulling="2026-05-28 18:52:47.0445994 +0000 UTC m=+351.303821544" observedRunningTime="2026-05-28 18:52:47.615737159 +0000 UTC m=+351.874959324" watchObservedRunningTime="2026-05-28 18:52:47.615946544 +0000 UTC m=+351.875168711"
May 28 18:52:47.852706 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:47.852597    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv"]
May 28 18:52:47.856330 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:47.856310    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv"
May 28 18:52:47.858884 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:47.858863    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\""
May 28 18:52:47.859074 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:47.859047    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-ddhzs\""
May 28 18:52:47.859173 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:47.859054    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\""
May 28 18:52:47.870604 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:47.870575    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv"]
May 28 18:52:47.909722 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:47.909680    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4b4q\" (UniqueName: \"kubernetes.io/projected/e4e43a40-3187-43fe-9f8a-b64569e115d4-kube-api-access-j4b4q\") pod \"servicemesh-operator3-55f49c5f94-5vmgv\" (UID: \"e4e43a40-3187-43fe-9f8a-b64569e115d4\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv"
May 28 18:52:47.910104 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:47.909829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/e4e43a40-3187-43fe-9f8a-b64569e115d4-operator-config\") pod \"servicemesh-operator3-55f49c5f94-5vmgv\" (UID: \"e4e43a40-3187-43fe-9f8a-b64569e115d4\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv"
May 28 18:52:48.010806 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:48.010768    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/e4e43a40-3187-43fe-9f8a-b64569e115d4-operator-config\") pod \"servicemesh-operator3-55f49c5f94-5vmgv\" (UID: \"e4e43a40-3187-43fe-9f8a-b64569e115d4\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv"
May 28 18:52:48.010806 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:48.010808    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j4b4q\" (UniqueName: \"kubernetes.io/projected/e4e43a40-3187-43fe-9f8a-b64569e115d4-kube-api-access-j4b4q\") pod \"servicemesh-operator3-55f49c5f94-5vmgv\" (UID: \"e4e43a40-3187-43fe-9f8a-b64569e115d4\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv"
May 28 18:52:48.013325 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:48.013294    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/e4e43a40-3187-43fe-9f8a-b64569e115d4-operator-config\") pod \"servicemesh-operator3-55f49c5f94-5vmgv\" (UID: \"e4e43a40-3187-43fe-9f8a-b64569e115d4\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv"
May 28 18:52:48.019906 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:48.019875    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4b4q\" (UniqueName: \"kubernetes.io/projected/e4e43a40-3187-43fe-9f8a-b64569e115d4-kube-api-access-j4b4q\") pod \"servicemesh-operator3-55f49c5f94-5vmgv\" (UID: \"e4e43a40-3187-43fe-9f8a-b64569e115d4\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv"
May 28 18:52:48.166278 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:48.166236    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv"
May 28 18:52:48.301517 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:48.301492    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv"]
May 28 18:52:48.304137 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:52:48.304107    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4e43a40_3187_43fe_9f8a_b64569e115d4.slice/crio-34d8187135a119d7e8fe1d445c833d82754861fdc145d158e3870f054cf6ed49 WatchSource:0}: Error finding container 34d8187135a119d7e8fe1d445c833d82754861fdc145d158e3870f054cf6ed49: Status 404 returned error can't find the container with id 34d8187135a119d7e8fe1d445c833d82754861fdc145d158e3870f054cf6ed49
May 28 18:52:48.609288 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:48.609194    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv" event={"ID":"e4e43a40-3187-43fe-9f8a-b64569e115d4","Type":"ContainerStarted","Data":"34d8187135a119d7e8fe1d445c833d82754861fdc145d158e3870f054cf6ed49"}
May 28 18:52:53.627627 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:53.627565    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv" event={"ID":"e4e43a40-3187-43fe-9f8a-b64569e115d4","Type":"ContainerStarted","Data":"e842ee88dbc78304d76da0f3cf85982dabe0cacd89b1a153b5691a1de84053b4"}
May 28 18:52:53.628026 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:53.627704    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv"
May 28 18:52:53.650785 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:53.650715    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv" podStartSLOduration=1.467548896 podStartE2EDuration="6.650695723s" podCreationTimestamp="2026-05-28 18:52:47 +0000 UTC" firstStartedPulling="2026-05-28 18:52:48.306698747 +0000 UTC m=+352.565920894" lastFinishedPulling="2026-05-28 18:52:53.489845568 +0000 UTC m=+357.749067721" observedRunningTime="2026-05-28 18:52:53.649130425 +0000 UTC m=+357.908352593" watchObservedRunningTime="2026-05-28 18:52:53.650695723 +0000 UTC m=+357.909917890"
May 28 18:52:54.560758 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.560717    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"]
May 28 18:52:54.564221 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.564200    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.566948 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.566923    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
May 28 18:52:54.566948 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.566938    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
May 28 18:52:54.567120 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.566938    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
May 28 18:52:54.567120 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.566938    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qbztd\""
May 28 18:52:54.567120 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.566942    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
May 28 18:52:54.574169 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.574148    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"]
May 28 18:52:54.674977 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.674933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c5bd532-1000-400d-a898-859a9488ab5f-local-certs\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.674977 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.674982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c5bd532-1000-400d-a898-859a9488ab5f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.675561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.675006    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c5bd532-1000-400d-a898-859a9488ab5f-cacerts\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.675561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.675076    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbxw2\" (UniqueName: \"kubernetes.io/projected/1c5bd532-1000-400d-a898-859a9488ab5f-kube-api-access-wbxw2\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.675561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.675119    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c5bd532-1000-400d-a898-859a9488ab5f-istio-token\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.675561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.675180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c5bd532-1000-400d-a898-859a9488ab5f-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.675561 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.675209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c5bd532-1000-400d-a898-859a9488ab5f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.776512 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.776471    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c5bd532-1000-400d-a898-859a9488ab5f-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.776716 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.776520    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c5bd532-1000-400d-a898-859a9488ab5f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.776716 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.776565    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c5bd532-1000-400d-a898-859a9488ab5f-local-certs\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.776716 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.776599    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c5bd532-1000-400d-a898-859a9488ab5f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.776716 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.776653    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c5bd532-1000-400d-a898-859a9488ab5f-cacerts\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.776716 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.776696    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wbxw2\" (UniqueName: \"kubernetes.io/projected/1c5bd532-1000-400d-a898-859a9488ab5f-kube-api-access-wbxw2\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.776990 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.776728    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c5bd532-1000-400d-a898-859a9488ab5f-istio-token\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.777296 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.777264    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c5bd532-1000-400d-a898-859a9488ab5f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.779123 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.779100    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c5bd532-1000-400d-a898-859a9488ab5f-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.779338 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.779314    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c5bd532-1000-400d-a898-859a9488ab5f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.779398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.779350    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c5bd532-1000-400d-a898-859a9488ab5f-local-certs\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.779398 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.779377    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c5bd532-1000-400d-a898-859a9488ab5f-cacerts\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.784756 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.784730    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c5bd532-1000-400d-a898-859a9488ab5f-istio-token\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.784875 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.784809    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbxw2\" (UniqueName: \"kubernetes.io/projected/1c5bd532-1000-400d-a898-859a9488ab5f-kube-api-access-wbxw2\") pod \"istiod-openshift-gateway-798958bb55-bg9lx\" (UID: \"1c5bd532-1000-400d-a898-859a9488ab5f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:54.875306 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:54.875200    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:55.018512 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:55.015262    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"]
May 28 18:52:55.020260 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:52:55.020220    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c5bd532_1000_400d_a898_859a9488ab5f.slice/crio-944df6e16b9d82fa4bf17a163fa0ff0d1d6369d1fbf0f60a813ea694e584b0cd WatchSource:0}: Error finding container 944df6e16b9d82fa4bf17a163fa0ff0d1d6369d1fbf0f60a813ea694e584b0cd: Status 404 returned error can't find the container with id 944df6e16b9d82fa4bf17a163fa0ff0d1d6369d1fbf0f60a813ea694e584b0cd
May 28 18:52:55.638945 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:55.638907    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx" event={"ID":"1c5bd532-1000-400d-a898-859a9488ab5f","Type":"ContainerStarted","Data":"944df6e16b9d82fa4bf17a163fa0ff0d1d6369d1fbf0f60a813ea694e584b0cd"}
May 28 18:52:56.894481 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:56.894444    2577 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76"
May 28 18:52:56.894973 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:56.894956    2577 scope.go:117] "RemoveContainer" containerID="e7e214daa2ca91d3bece147cb2b1e1b232ed2845d652d0c3159c3d018b768649"
May 28 18:52:57.459576 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:57.459529    2577 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
May 28 18:52:57.459695 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:57.459628    2577 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
May 28 18:52:57.648366 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:57.648323    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76" event={"ID":"b033be53-342a-4f0d-a7da-76327e02eef5","Type":"ContainerStarted","Data":"1a65ac43c6eb8572ff44e2778ff453997de06c76ebc782e258d19e587836db7d"}
May 28 18:52:57.649179 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:57.649151    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76"
May 28 18:52:57.650860 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:57.650836    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx" event={"ID":"1c5bd532-1000-400d-a898-859a9488ab5f","Type":"ContainerStarted","Data":"8c1e5341be0a0c7422e1794142bab756551585542c33e58dd11b5ab2cfb2249e"}
May 28 18:52:57.650987 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:57.650975    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:52:57.667334 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:57.667286    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76" podStartSLOduration=2.293253243 podStartE2EDuration="22.667272888s" podCreationTimestamp="2026-05-28 18:52:35 +0000 UTC" firstStartedPulling="2026-05-28 18:52:37.08509317 +0000 UTC m=+341.344315314" lastFinishedPulling="2026-05-28 18:52:57.459112801 +0000 UTC m=+361.718334959" observedRunningTime="2026-05-28 18:52:57.666473792 +0000 UTC m=+361.925695956" watchObservedRunningTime="2026-05-28 18:52:57.667272888 +0000 UTC m=+361.926495053"
May 28 18:52:57.689033 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:57.688970    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx" podStartSLOduration=1.25179245 podStartE2EDuration="3.688951267s" podCreationTimestamp="2026-05-28 18:52:54 +0000 UTC" firstStartedPulling="2026-05-28 18:52:55.022139866 +0000 UTC m=+359.281362009" lastFinishedPulling="2026-05-28 18:52:57.459298666 +0000 UTC m=+361.718520826" observedRunningTime="2026-05-28 18:52:57.686529532 +0000 UTC m=+361.945751724" watchObservedRunningTime="2026-05-28 18:52:57.688951267 +0000 UTC m=+361.948173434"
May 28 18:52:58.657374 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:58.657328    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-bg9lx container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
May 28 18:52:58.657837 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:52:58.657398    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx" podUID="1c5bd532-1000-400d-a898-859a9488ab5f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 28 18:53:01.656724 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:53:01.656694    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bg9lx"
May 28 18:53:04.633509 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:53:04.633474    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5vmgv"
May 28 18:53:08.657719 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:53:08.657685    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-jtf76"
May 28 18:53:18.615511 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:53:18.615479    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-gxxq7"
May 28 18:54:08.361929 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.361896    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"]
May 28 18:54:08.365399 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.365351    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"
May 28 18:54:08.368207 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.368170    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zzds7\""
May 28 18:54:08.368382 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.368364    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 28 18:54:08.368595 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.368581    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 28 18:54:08.376371 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.376348    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"]
May 28 18:54:08.449781 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.449740    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a359d2f-a7cf-4587-89b2-b998e55bb3ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ss5bt\" (UID: \"2a359d2f-a7cf-4587-89b2-b998e55bb3ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"
May 28 18:54:08.449958 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.449822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rf4h\" (UniqueName: \"kubernetes.io/projected/2a359d2f-a7cf-4587-89b2-b998e55bb3ae-kube-api-access-9rf4h\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ss5bt\" (UID: \"2a359d2f-a7cf-4587-89b2-b998e55bb3ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"
May 28 18:54:08.550517 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.550478    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9rf4h\" (UniqueName: \"kubernetes.io/projected/2a359d2f-a7cf-4587-89b2-b998e55bb3ae-kube-api-access-9rf4h\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ss5bt\" (UID: \"2a359d2f-a7cf-4587-89b2-b998e55bb3ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"
May 28 18:54:08.550720 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.550658    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a359d2f-a7cf-4587-89b2-b998e55bb3ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ss5bt\" (UID: \"2a359d2f-a7cf-4587-89b2-b998e55bb3ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"
May 28 18:54:08.551012 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.550995    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a359d2f-a7cf-4587-89b2-b998e55bb3ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ss5bt\" (UID: \"2a359d2f-a7cf-4587-89b2-b998e55bb3ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"
May 28 18:54:08.565377 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.565345    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rf4h\" (UniqueName: \"kubernetes.io/projected/2a359d2f-a7cf-4587-89b2-b998e55bb3ae-kube-api-access-9rf4h\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ss5bt\" (UID: \"2a359d2f-a7cf-4587-89b2-b998e55bb3ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"
May 28 18:54:08.676228 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.676183    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"
May 28 18:54:08.805076 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.805047    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"]
May 28 18:54:08.807395 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:54:08.807366    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a359d2f_a7cf_4587_89b2_b998e55bb3ae.slice/crio-ff23e37672b91c7d9228fcb215457458b3fd42f7c3c052efc8b4eb8b3580b35d WatchSource:0}: Error finding container ff23e37672b91c7d9228fcb215457458b3fd42f7c3c052efc8b4eb8b3580b35d: Status 404 returned error can't find the container with id ff23e37672b91c7d9228fcb215457458b3fd42f7c3c052efc8b4eb8b3580b35d
May 28 18:54:08.898067 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:08.898030    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt" event={"ID":"2a359d2f-a7cf-4587-89b2-b998e55bb3ae","Type":"ContainerStarted","Data":"ff23e37672b91c7d9228fcb215457458b3fd42f7c3c052efc8b4eb8b3580b35d"}
May 28 18:54:14.375323 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.375278    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-lbm6s"]
May 28 18:54:14.378896 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.378873    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-lbm6s"
May 28 18:54:14.381579 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.381549    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-6zv8p\""
May 28 18:54:14.381711 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.381605    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
May 28 18:54:14.386735 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.386708    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-lbm6s"]
May 28 18:54:14.511429 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.511339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2gt2\" (UniqueName: \"kubernetes.io/projected/8eb1be56-a272-40d1-b758-84a238337f53-kube-api-access-t2gt2\") pod \"dns-operator-controller-manager-648d5c98bc-lbm6s\" (UID: \"8eb1be56-a272-40d1-b758-84a238337f53\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-lbm6s"
May 28 18:54:14.612638 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.612567    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t2gt2\" (UniqueName: \"kubernetes.io/projected/8eb1be56-a272-40d1-b758-84a238337f53-kube-api-access-t2gt2\") pod \"dns-operator-controller-manager-648d5c98bc-lbm6s\" (UID: \"8eb1be56-a272-40d1-b758-84a238337f53\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-lbm6s"
May 28 18:54:14.626746 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.626715    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2gt2\" (UniqueName: \"kubernetes.io/projected/8eb1be56-a272-40d1-b758-84a238337f53-kube-api-access-t2gt2\") pod \"dns-operator-controller-manager-648d5c98bc-lbm6s\" (UID: \"8eb1be56-a272-40d1-b758-84a238337f53\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-lbm6s"
May 28 18:54:14.690311 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.690273    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-lbm6s"
May 28 18:54:14.829161 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.829134    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-lbm6s"]
May 28 18:54:14.831531 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:54:14.831504    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eb1be56_a272_40d1_b758_84a238337f53.slice/crio-50160d17ac11d1a07f32fcc2c5ba697b6732e9967d0a35eb0978136c3d954629 WatchSource:0}: Error finding container 50160d17ac11d1a07f32fcc2c5ba697b6732e9967d0a35eb0978136c3d954629: Status 404 returned error can't find the container with id 50160d17ac11d1a07f32fcc2c5ba697b6732e9967d0a35eb0978136c3d954629
May 28 18:54:14.923095 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.923055    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-lbm6s" event={"ID":"8eb1be56-a272-40d1-b758-84a238337f53","Type":"ContainerStarted","Data":"50160d17ac11d1a07f32fcc2c5ba697b6732e9967d0a35eb0978136c3d954629"}
May 28 18:54:14.924478 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.924443    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt" event={"ID":"2a359d2f-a7cf-4587-89b2-b998e55bb3ae","Type":"ContainerStarted","Data":"eeac00fec18f1c79b43337ee196f48791334945205c69592bfb26c105e475845"}
May 28 18:54:14.924638 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.924620    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"
May 28 18:54:14.944841 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:14.944798    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt" podStartSLOduration=1.55503333 podStartE2EDuration="6.944784586s" podCreationTimestamp="2026-05-28 18:54:08 +0000 UTC" firstStartedPulling="2026-05-28 18:54:08.809762432 +0000 UTC m=+433.068984579" lastFinishedPulling="2026-05-28 18:54:14.19951369 +0000 UTC m=+438.458735835" observedRunningTime="2026-05-28 18:54:14.943961157 +0000 UTC m=+439.203183326" watchObservedRunningTime="2026-05-28 18:54:14.944784586 +0000 UTC m=+439.204006774"
May 28 18:54:16.932775 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:16.932738    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-lbm6s" event={"ID":"8eb1be56-a272-40d1-b758-84a238337f53","Type":"ContainerStarted","Data":"770c07d140d2b16ad3c47dafff114fbd7f73edec55a390d5ee8293393ef77162"}
May 28 18:54:16.933139 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:16.932801    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-lbm6s"
May 28 18:54:16.949216 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:16.949163    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-lbm6s" podStartSLOduration=0.929465497 podStartE2EDuration="2.94914776s" podCreationTimestamp="2026-05-28 18:54:14 +0000 UTC" firstStartedPulling="2026-05-28 18:54:14.833317482 +0000 UTC m=+439.092539629" lastFinishedPulling="2026-05-28 18:54:16.852999736 +0000 UTC m=+441.112221892" observedRunningTime="2026-05-28 18:54:16.949107553 +0000 UTC m=+441.208329731" watchObservedRunningTime="2026-05-28 18:54:16.94914776 +0000 UTC m=+441.208369926"
May 28 18:54:25.931444 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:25.931412    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"
May 28 18:54:27.709636 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.709578    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"]
May 28 18:54:27.710054 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.709797    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt" podUID="2a359d2f-a7cf-4587-89b2-b998e55bb3ae" containerName="manager" containerID="cri-o://eeac00fec18f1c79b43337ee196f48791334945205c69592bfb26c105e475845" gracePeriod=2
May 28 18:54:27.715574 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.715543    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"]
May 28 18:54:27.736933 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.736905    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"]
May 28 18:54:27.737316 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.737288    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a359d2f-a7cf-4587-89b2-b998e55bb3ae" containerName="manager"
May 28 18:54:27.737316 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.737301    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a359d2f-a7cf-4587-89b2-b998e55bb3ae" containerName="manager"
May 28 18:54:27.737397 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.737366    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="2a359d2f-a7cf-4587-89b2-b998e55bb3ae" containerName="manager"
May 28 18:54:27.741959 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.741925    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"
May 28 18:54:27.744720 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.744679    2577 status_manager.go:895] "Failed to get status for pod" podUID="2a359d2f-a7cf-4587-89b2-b998e55bb3ae" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ss5bt\" is forbidden: User \"system:node:ip-10-0-137-226.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-226.ec2.internal' and this object"
May 28 18:54:27.748543 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.748517    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bx8ll"]
May 28 18:54:27.754049 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.754021    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"]
May 28 18:54:27.754188 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.754125    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bx8ll"
May 28 18:54:27.757077 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.757055    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-f2h6z\""
May 28 18:54:27.760644 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.760602    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bx8ll"]
May 28 18:54:27.768603 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.768568    2577 status_manager.go:895] "Failed to get status for pod" podUID="2a359d2f-a7cf-4587-89b2-b998e55bb3ae" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ss5bt\" is forbidden: User \"system:node:ip-10-0-137-226.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-226.ec2.internal' and this object"
May 28 18:54:27.831822 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.831787    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94gr8\" (UniqueName: \"kubernetes.io/projected/81e45e24-a72d-4641-8446-19262b1012cb-kube-api-access-94gr8\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-hn9s5\" (UID: \"81e45e24-a72d-4641-8446-19262b1012cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"
May 28 18:54:27.831961 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.831856    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81e45e24-a72d-4641-8446-19262b1012cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-hn9s5\" (UID: \"81e45e24-a72d-4641-8446-19262b1012cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"
May 28 18:54:27.831961 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.831912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64vkd\" (UniqueName: \"kubernetes.io/projected/3731f783-5532-41bc-9341-043b56fcb1d7-kube-api-access-64vkd\") pod \"limitador-operator-controller-manager-85c4996f8c-bx8ll\" (UID: \"3731f783-5532-41bc-9341-043b56fcb1d7\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bx8ll"
May 28 18:54:27.933017 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.932990    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81e45e24-a72d-4641-8446-19262b1012cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-hn9s5\" (UID: \"81e45e24-a72d-4641-8446-19262b1012cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"
May 28 18:54:27.933136 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.933050    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-64vkd\" (UniqueName: \"kubernetes.io/projected/3731f783-5532-41bc-9341-043b56fcb1d7-kube-api-access-64vkd\") pod \"limitador-operator-controller-manager-85c4996f8c-bx8ll\" (UID: \"3731f783-5532-41bc-9341-043b56fcb1d7\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bx8ll"
May 28 18:54:27.933136 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.933097    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-94gr8\" (UniqueName: \"kubernetes.io/projected/81e45e24-a72d-4641-8446-19262b1012cb-kube-api-access-94gr8\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-hn9s5\" (UID: \"81e45e24-a72d-4641-8446-19262b1012cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"
May 28 18:54:27.933334 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.933315    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81e45e24-a72d-4641-8446-19262b1012cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-hn9s5\" (UID: \"81e45e24-a72d-4641-8446-19262b1012cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"
May 28 18:54:27.939091 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.939066    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-lbm6s"
May 28 18:54:27.941148 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.941119    2577 status_manager.go:895] "Failed to get status for pod" podUID="2a359d2f-a7cf-4587-89b2-b998e55bb3ae" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ss5bt\" is forbidden: User \"system:node:ip-10-0-137-226.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-226.ec2.internal' and this object"
May 28 18:54:27.950580 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.950558    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64vkd\" (UniqueName: \"kubernetes.io/projected/3731f783-5532-41bc-9341-043b56fcb1d7-kube-api-access-64vkd\") pod \"limitador-operator-controller-manager-85c4996f8c-bx8ll\" (UID: \"3731f783-5532-41bc-9341-043b56fcb1d7\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bx8ll"
May 28 18:54:27.950706 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.950685    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94gr8\" (UniqueName: \"kubernetes.io/projected/81e45e24-a72d-4641-8446-19262b1012cb-kube-api-access-94gr8\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-hn9s5\" (UID: \"81e45e24-a72d-4641-8446-19262b1012cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"
May 28 18:54:27.951228 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.951213    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"
May 28 18:54:27.963803 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.963733    2577 status_manager.go:895] "Failed to get status for pod" podUID="2a359d2f-a7cf-4587-89b2-b998e55bb3ae" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ss5bt\" is forbidden: User \"system:node:ip-10-0-137-226.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-226.ec2.internal' and this object"
May 28 18:54:27.972526 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.972494    2577 generic.go:358] "Generic (PLEG): container finished" podID="2a359d2f-a7cf-4587-89b2-b998e55bb3ae" containerID="eeac00fec18f1c79b43337ee196f48791334945205c69592bfb26c105e475845" exitCode=0
May 28 18:54:27.972681 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.972543    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt"
May 28 18:54:27.972681 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.972595    2577 scope.go:117] "RemoveContainer" containerID="eeac00fec18f1c79b43337ee196f48791334945205c69592bfb26c105e475845"
May 28 18:54:27.974797 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.974763    2577 status_manager.go:895] "Failed to get status for pod" podUID="2a359d2f-a7cf-4587-89b2-b998e55bb3ae" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ss5bt\" is forbidden: User \"system:node:ip-10-0-137-226.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-226.ec2.internal' and this object"
May 28 18:54:27.983152 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.983131    2577 scope.go:117] "RemoveContainer" containerID="eeac00fec18f1c79b43337ee196f48791334945205c69592bfb26c105e475845"
May 28 18:54:27.983455 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:54:27.983433    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeac00fec18f1c79b43337ee196f48791334945205c69592bfb26c105e475845\": container with ID starting with eeac00fec18f1c79b43337ee196f48791334945205c69592bfb26c105e475845 not found: ID does not exist" containerID="eeac00fec18f1c79b43337ee196f48791334945205c69592bfb26c105e475845"
May 28 18:54:27.983506 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:27.983467    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeac00fec18f1c79b43337ee196f48791334945205c69592bfb26c105e475845"} err="failed to get container status \"eeac00fec18f1c79b43337ee196f48791334945205c69592bfb26c105e475845\": rpc error: code = NotFound desc = could not find container \"eeac00fec18f1c79b43337ee196f48791334945205c69592bfb26c105e475845\": container with ID starting with eeac00fec18f1c79b43337ee196f48791334945205c69592bfb26c105e475845 not found: ID does not exist"
May 28 18:54:28.033573 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.033542    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a359d2f-a7cf-4587-89b2-b998e55bb3ae-extensions-socket-volume\") pod \"2a359d2f-a7cf-4587-89b2-b998e55bb3ae\" (UID: \"2a359d2f-a7cf-4587-89b2-b998e55bb3ae\") "
May 28 18:54:28.033792 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.033597    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rf4h\" (UniqueName: \"kubernetes.io/projected/2a359d2f-a7cf-4587-89b2-b998e55bb3ae-kube-api-access-9rf4h\") pod \"2a359d2f-a7cf-4587-89b2-b998e55bb3ae\" (UID: \"2a359d2f-a7cf-4587-89b2-b998e55bb3ae\") "
May 28 18:54:28.034105 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.034076    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a359d2f-a7cf-4587-89b2-b998e55bb3ae-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2a359d2f-a7cf-4587-89b2-b998e55bb3ae" (UID: "2a359d2f-a7cf-4587-89b2-b998e55bb3ae"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 18:54:28.035804 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.035784    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a359d2f-a7cf-4587-89b2-b998e55bb3ae-kube-api-access-9rf4h" (OuterVolumeSpecName: "kube-api-access-9rf4h") pod "2a359d2f-a7cf-4587-89b2-b998e55bb3ae" (UID: "2a359d2f-a7cf-4587-89b2-b998e55bb3ae"). InnerVolumeSpecName "kube-api-access-9rf4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 18:54:28.097689 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.097651    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"
May 28 18:54:28.103485 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.103459    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bx8ll"
May 28 18:54:28.135230 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.135194    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a359d2f-a7cf-4587-89b2-b998e55bb3ae-extensions-socket-volume\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:54:28.135230 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.135225    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rf4h\" (UniqueName: \"kubernetes.io/projected/2a359d2f-a7cf-4587-89b2-b998e55bb3ae-kube-api-access-9rf4h\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:54:28.234880 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.234848    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"]
May 28 18:54:28.237985 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:54:28.237955    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e45e24_a72d_4641_8446_19262b1012cb.slice/crio-bbaab3b828e8e0b82f7a9122ec73e66b739c76a6086690c3787c2b1bf0904ec0 WatchSource:0}: Error finding container bbaab3b828e8e0b82f7a9122ec73e66b739c76a6086690c3787c2b1bf0904ec0: Status 404 returned error can't find the container with id bbaab3b828e8e0b82f7a9122ec73e66b739c76a6086690c3787c2b1bf0904ec0
May 28 18:54:28.255295 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.255269    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bx8ll"]
May 28 18:54:28.257450 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:54:28.257423    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3731f783_5532_41bc_9341_043b56fcb1d7.slice/crio-0099df560fb3f68bd15cc0118771a8ba6d75ec8ee84f0fe3dc07ea248dc6ec37 WatchSource:0}: Error finding container 0099df560fb3f68bd15cc0118771a8ba6d75ec8ee84f0fe3dc07ea248dc6ec37: Status 404 returned error can't find the container with id 0099df560fb3f68bd15cc0118771a8ba6d75ec8ee84f0fe3dc07ea248dc6ec37
May 28 18:54:28.296151 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.296119    2577 status_manager.go:895] "Failed to get status for pod" podUID="2a359d2f-a7cf-4587-89b2-b998e55bb3ae" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ss5bt" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-ss5bt\" is forbidden: User \"system:node:ip-10-0-137-226.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-226.ec2.internal' and this object"
May 28 18:54:28.359496 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.359464    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a359d2f-a7cf-4587-89b2-b998e55bb3ae" path="/var/lib/kubelet/pods/2a359d2f-a7cf-4587-89b2-b998e55bb3ae/volumes"
May 28 18:54:28.980572 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.980450    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5" event={"ID":"81e45e24-a72d-4641-8446-19262b1012cb","Type":"ContainerStarted","Data":"26742feac5ddcb4f795ec285d1ca614f12ef9ba5bd1d2cc3d296f7df976570cb"}
May 28 18:54:28.980572 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.980492    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5" event={"ID":"81e45e24-a72d-4641-8446-19262b1012cb","Type":"ContainerStarted","Data":"bbaab3b828e8e0b82f7a9122ec73e66b739c76a6086690c3787c2b1bf0904ec0"}
May 28 18:54:28.980572 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.980524    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"
May 28 18:54:28.981752 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:28.981719    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bx8ll" event={"ID":"3731f783-5532-41bc-9341-043b56fcb1d7","Type":"ContainerStarted","Data":"0099df560fb3f68bd15cc0118771a8ba6d75ec8ee84f0fe3dc07ea248dc6ec37"}
May 28 18:54:29.006428 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:29.006261    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5" podStartSLOduration=2.006238289 podStartE2EDuration="2.006238289s" podCreationTimestamp="2026-05-28 18:54:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 18:54:29.002931223 +0000 UTC m=+453.262153401" watchObservedRunningTime="2026-05-28 18:54:29.006238289 +0000 UTC m=+453.265460435"
May 28 18:54:29.986483 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:29.986443    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bx8ll" event={"ID":"3731f783-5532-41bc-9341-043b56fcb1d7","Type":"ContainerStarted","Data":"1e8b410378364700d5c32a83bcbcac80980da04eda8e0fdc3ceba5c4285988b5"}
May 28 18:54:29.987002 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:29.986527    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bx8ll"
May 28 18:54:30.013349 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:30.013292    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bx8ll" podStartSLOduration=1.7730399239999999 podStartE2EDuration="3.013277022s" podCreationTimestamp="2026-05-28 18:54:27 +0000 UTC" firstStartedPulling="2026-05-28 18:54:28.259810885 +0000 UTC m=+452.519033034" lastFinishedPulling="2026-05-28 18:54:29.500047985 +0000 UTC m=+453.759270132" observedRunningTime="2026-05-28 18:54:30.010793576 +0000 UTC m=+454.270015742" watchObservedRunningTime="2026-05-28 18:54:30.013277022 +0000 UTC m=+454.272499187"
May 28 18:54:39.988761 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:39.988728    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"
May 28 18:54:40.992025 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:40.991996    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bx8ll"
May 28 18:54:43.909626 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:43.909583    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"]
May 28 18:54:43.909993 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:43.909831    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5" podUID="81e45e24-a72d-4641-8446-19262b1012cb" containerName="manager" containerID="cri-o://26742feac5ddcb4f795ec285d1ca614f12ef9ba5bd1d2cc3d296f7df976570cb" gracePeriod=10
May 28 18:54:44.040951 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:44.040913    2577 generic.go:358] "Generic (PLEG): container finished" podID="81e45e24-a72d-4641-8446-19262b1012cb" containerID="26742feac5ddcb4f795ec285d1ca614f12ef9ba5bd1d2cc3d296f7df976570cb" exitCode=0
May 28 18:54:44.041101 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:44.040982    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5" event={"ID":"81e45e24-a72d-4641-8446-19262b1012cb","Type":"ContainerDied","Data":"26742feac5ddcb4f795ec285d1ca614f12ef9ba5bd1d2cc3d296f7df976570cb"}
May 28 18:54:44.151961 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:44.151937    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"
May 28 18:54:44.287040 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:44.286942    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94gr8\" (UniqueName: \"kubernetes.io/projected/81e45e24-a72d-4641-8446-19262b1012cb-kube-api-access-94gr8\") pod \"81e45e24-a72d-4641-8446-19262b1012cb\" (UID: \"81e45e24-a72d-4641-8446-19262b1012cb\") "
May 28 18:54:44.287040 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:44.286999    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81e45e24-a72d-4641-8446-19262b1012cb-extensions-socket-volume\") pod \"81e45e24-a72d-4641-8446-19262b1012cb\" (UID: \"81e45e24-a72d-4641-8446-19262b1012cb\") "
May 28 18:54:44.287382 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:44.287359    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81e45e24-a72d-4641-8446-19262b1012cb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "81e45e24-a72d-4641-8446-19262b1012cb" (UID: "81e45e24-a72d-4641-8446-19262b1012cb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 18:54:44.289036 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:44.289016    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81e45e24-a72d-4641-8446-19262b1012cb-kube-api-access-94gr8" (OuterVolumeSpecName: "kube-api-access-94gr8") pod "81e45e24-a72d-4641-8446-19262b1012cb" (UID: "81e45e24-a72d-4641-8446-19262b1012cb"). InnerVolumeSpecName "kube-api-access-94gr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 18:54:44.388351 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:44.388314    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94gr8\" (UniqueName: \"kubernetes.io/projected/81e45e24-a72d-4641-8446-19262b1012cb-kube-api-access-94gr8\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:54:44.388351 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:44.388347    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81e45e24-a72d-4641-8446-19262b1012cb-extensions-socket-volume\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:54:45.046126 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:45.046083    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5" event={"ID":"81e45e24-a72d-4641-8446-19262b1012cb","Type":"ContainerDied","Data":"bbaab3b828e8e0b82f7a9122ec73e66b739c76a6086690c3787c2b1bf0904ec0"}
May 28 18:54:45.046559 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:45.046139    2577 scope.go:117] "RemoveContainer" containerID="26742feac5ddcb4f795ec285d1ca614f12ef9ba5bd1d2cc3d296f7df976570cb"
May 28 18:54:45.046559 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:45.046141    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"
May 28 18:54:45.067139 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:45.067086    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"]
May 28 18:54:45.071892 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:45.071866    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-hn9s5"]
May 28 18:54:46.359372 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:54:46.359335    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81e45e24-a72d-4641-8446-19262b1012cb" path="/var/lib/kubelet/pods/81e45e24-a72d-4641-8446-19262b1012cb/volumes"
May 28 18:55:04.321991 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.321905    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-vlxnv"]
May 28 18:55:04.322540 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.322518    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81e45e24-a72d-4641-8446-19262b1012cb" containerName="manager"
May 28 18:55:04.322651 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.322542    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="81e45e24-a72d-4641-8446-19262b1012cb" containerName="manager"
May 28 18:55:04.322719 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.322693    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="81e45e24-a72d-4641-8446-19262b1012cb" containerName="manager"
May 28 18:55:04.328091 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.328066    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv"
May 28 18:55:04.330522 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.330497    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hnfz8\""
May 28 18:55:04.330661 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.330520    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
May 28 18:55:04.334270 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.334244    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-vlxnv"]
May 28 18:55:04.365173 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.365136    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b-config-file\") pod \"limitador-limitador-7d549b5b-vlxnv\" (UID: \"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b\") " pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv"
May 28 18:55:04.365173 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.365175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swspw\" (UniqueName: \"kubernetes.io/projected/4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b-kube-api-access-swspw\") pod \"limitador-limitador-7d549b5b-vlxnv\" (UID: \"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b\") " pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv"
May 28 18:55:04.425247 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.425215    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-vlxnv"]
May 28 18:55:04.466257 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.466220    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b-config-file\") pod \"limitador-limitador-7d549b5b-vlxnv\" (UID: \"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b\") " pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv"
May 28 18:55:04.466407 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.466268    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-swspw\" (UniqueName: \"kubernetes.io/projected/4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b-kube-api-access-swspw\") pod \"limitador-limitador-7d549b5b-vlxnv\" (UID: \"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b\") " pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv"
May 28 18:55:04.466859 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.466834    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b-config-file\") pod \"limitador-limitador-7d549b5b-vlxnv\" (UID: \"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b\") " pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv"
May 28 18:55:04.474640 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.474599    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swspw\" (UniqueName: \"kubernetes.io/projected/4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b-kube-api-access-swspw\") pod \"limitador-limitador-7d549b5b-vlxnv\" (UID: \"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b\") " pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv"
May 28 18:55:04.640531 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.640486    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv"
May 28 18:55:04.777413 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:04.777383    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-vlxnv"]
May 28 18:55:04.779572 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:55:04.779539    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec4c7f6_303f_4dad_b868_3c6ccf9eeb9b.slice/crio-6c0ec4c88ec27e823dfa9102bc726b608edb7de99e98e272a4e262ac323ac740 WatchSource:0}: Error finding container 6c0ec4c88ec27e823dfa9102bc726b608edb7de99e98e272a4e262ac323ac740: Status 404 returned error can't find the container with id 6c0ec4c88ec27e823dfa9102bc726b608edb7de99e98e272a4e262ac323ac740
May 28 18:55:05.122744 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:05.122658    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv" event={"ID":"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b","Type":"ContainerStarted","Data":"6c0ec4c88ec27e823dfa9102bc726b608edb7de99e98e272a4e262ac323ac740"}
May 28 18:55:08.137880 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:08.137835    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv" event={"ID":"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b","Type":"ContainerStarted","Data":"255edd63cbc29a1b8ba55e3e8c0fd4261f5ae3ed22828cebfd51439523297725"}
May 28 18:55:08.138302 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:08.137969    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv"
May 28 18:55:08.155859 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:08.155806    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv" podStartSLOduration=1.327505327 podStartE2EDuration="4.155791491s" podCreationTimestamp="2026-05-28 18:55:04 +0000 UTC" firstStartedPulling="2026-05-28 18:55:04.781283539 +0000 UTC m=+489.040505686" lastFinishedPulling="2026-05-28 18:55:07.609569696 +0000 UTC m=+491.868791850" observedRunningTime="2026-05-28 18:55:08.154145484 +0000 UTC m=+492.413367650" watchObservedRunningTime="2026-05-28 18:55:08.155791491 +0000 UTC m=+492.415013657"
May 28 18:55:19.143172 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:19.143137    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv"
May 28 18:55:22.714909 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:22.714871    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-vlxnv"]
May 28 18:55:22.715312 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:22.715120    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv" podUID="4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b" containerName="limitador" containerID="cri-o://255edd63cbc29a1b8ba55e3e8c0fd4261f5ae3ed22828cebfd51439523297725" gracePeriod=30
May 28 18:55:23.196822 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:23.196781    2577 generic.go:358] "Generic (PLEG): container finished" podID="4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b" containerID="255edd63cbc29a1b8ba55e3e8c0fd4261f5ae3ed22828cebfd51439523297725" exitCode=0
May 28 18:55:23.196822 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:23.196820    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv" event={"ID":"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b","Type":"ContainerDied","Data":"255edd63cbc29a1b8ba55e3e8c0fd4261f5ae3ed22828cebfd51439523297725"}
May 28 18:55:23.663101 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:23.663076    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv"
May 28 18:55:23.734017 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:23.733983    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swspw\" (UniqueName: \"kubernetes.io/projected/4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b-kube-api-access-swspw\") pod \"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b\" (UID: \"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b\") "
May 28 18:55:23.734421 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:23.734076    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b-config-file\") pod \"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b\" (UID: \"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b\") "
May 28 18:55:23.734421 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:23.734391    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b-config-file" (OuterVolumeSpecName: "config-file") pod "4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b" (UID: "4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 18:55:23.736063 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:23.736040    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b-kube-api-access-swspw" (OuterVolumeSpecName: "kube-api-access-swspw") pod "4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b" (UID: "4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b"). InnerVolumeSpecName "kube-api-access-swspw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 18:55:23.835798 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:23.835705    2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b-config-file\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:55:23.835798 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:23.835743    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swspw\" (UniqueName: \"kubernetes.io/projected/4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b-kube-api-access-swspw\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:55:24.201726 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:24.201686    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv" event={"ID":"4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b","Type":"ContainerDied","Data":"6c0ec4c88ec27e823dfa9102bc726b608edb7de99e98e272a4e262ac323ac740"}
May 28 18:55:24.201726 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:24.201725    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-vlxnv"
May 28 18:55:24.201940 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:24.201741    2577 scope.go:117] "RemoveContainer" containerID="255edd63cbc29a1b8ba55e3e8c0fd4261f5ae3ed22828cebfd51439523297725"
May 28 18:55:24.221858 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:24.221834    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-vlxnv"]
May 28 18:55:24.228411 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:24.228387    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-vlxnv"]
May 28 18:55:24.358868 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:24.358828    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b" path="/var/lib/kubelet/pods/4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b/volumes"
May 28 18:55:25.501637 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.501588    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-8bd5w"]
May 28 18:55:25.502099 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.501973    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b" containerName="limitador"
May 28 18:55:25.502099 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.501986    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b" containerName="limitador"
May 28 18:55:25.502099 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.502059    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="4ec4c7f6-303f-4dad-b868-3c6ccf9eeb9b" containerName="limitador"
May 28 18:55:25.506433 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.506406    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-8bd5w"
May 28 18:55:25.508944 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.508917    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
May 28 18:55:25.509089 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.508962    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dzjdz\""
May 28 18:55:25.513347 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.513324    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-8bd5w"]
May 28 18:55:25.552333 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.552295    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/937a17ac-4b76-486e-b6f6-f86c98175b28-data\") pod \"postgres-68c4fbbd6f-8bd5w\" (UID: \"937a17ac-4b76-486e-b6f6-f86c98175b28\") " pod="opendatahub/postgres-68c4fbbd6f-8bd5w"
May 28 18:55:25.552333 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.552333    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2z46\" (UniqueName: \"kubernetes.io/projected/937a17ac-4b76-486e-b6f6-f86c98175b28-kube-api-access-z2z46\") pod \"postgres-68c4fbbd6f-8bd5w\" (UID: \"937a17ac-4b76-486e-b6f6-f86c98175b28\") " pod="opendatahub/postgres-68c4fbbd6f-8bd5w"
May 28 18:55:25.652975 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.652936    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/937a17ac-4b76-486e-b6f6-f86c98175b28-data\") pod \"postgres-68c4fbbd6f-8bd5w\" (UID: \"937a17ac-4b76-486e-b6f6-f86c98175b28\") " pod="opendatahub/postgres-68c4fbbd6f-8bd5w"
May 28 18:55:25.652975 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.652978    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z2z46\" (UniqueName: \"kubernetes.io/projected/937a17ac-4b76-486e-b6f6-f86c98175b28-kube-api-access-z2z46\") pod \"postgres-68c4fbbd6f-8bd5w\" (UID: \"937a17ac-4b76-486e-b6f6-f86c98175b28\") " pod="opendatahub/postgres-68c4fbbd6f-8bd5w"
May 28 18:55:25.653307 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.653285    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/937a17ac-4b76-486e-b6f6-f86c98175b28-data\") pod \"postgres-68c4fbbd6f-8bd5w\" (UID: \"937a17ac-4b76-486e-b6f6-f86c98175b28\") " pod="opendatahub/postgres-68c4fbbd6f-8bd5w"
May 28 18:55:25.661161 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.661136    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2z46\" (UniqueName: \"kubernetes.io/projected/937a17ac-4b76-486e-b6f6-f86c98175b28-kube-api-access-z2z46\") pod \"postgres-68c4fbbd6f-8bd5w\" (UID: \"937a17ac-4b76-486e-b6f6-f86c98175b28\") " pod="opendatahub/postgres-68c4fbbd6f-8bd5w"
May 28 18:55:25.821333 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.821252    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-8bd5w"
May 28 18:55:25.953808 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:25.953782    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-8bd5w"]
May 28 18:55:25.955364 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:55:25.955334    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod937a17ac_4b76_486e_b6f6_f86c98175b28.slice/crio-038210156c998d3276adafc78850a33e450f549f81e064dbaeeb74a8fcff9287 WatchSource:0}: Error finding container 038210156c998d3276adafc78850a33e450f549f81e064dbaeeb74a8fcff9287: Status 404 returned error can't find the container with id 038210156c998d3276adafc78850a33e450f549f81e064dbaeeb74a8fcff9287
May 28 18:55:26.210299 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:26.210264    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-8bd5w" event={"ID":"937a17ac-4b76-486e-b6f6-f86c98175b28","Type":"ContainerStarted","Data":"038210156c998d3276adafc78850a33e450f549f81e064dbaeeb74a8fcff9287"}
May 28 18:55:31.231865 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:31.231827    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-8bd5w" event={"ID":"937a17ac-4b76-486e-b6f6-f86c98175b28","Type":"ContainerStarted","Data":"57ec28e07835cd9c9b2798814bef3da230ccfa1cae1d139ce29e434665a29892"}
May 28 18:55:31.232309 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:31.231886    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-8bd5w"
May 28 18:55:31.248913 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:31.248854    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-8bd5w" podStartSLOduration=1.450531346 podStartE2EDuration="6.248840073s" podCreationTimestamp="2026-05-28 18:55:25 +0000 UTC" firstStartedPulling="2026-05-28 18:55:25.956570193 +0000 UTC m=+510.215792340" lastFinishedPulling="2026-05-28 18:55:30.75487892 +0000 UTC m=+515.014101067" observedRunningTime="2026-05-28 18:55:31.247504447 +0000 UTC m=+515.506726616" watchObservedRunningTime="2026-05-28 18:55:31.248840073 +0000 UTC m=+515.508062239"
May 28 18:55:37.264136 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:37.264107    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-8bd5w"
May 28 18:55:42.263004 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.262930    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56c767b8c7-mh9fw"]
May 28 18:55:42.270723 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.270697    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c767b8c7-mh9fw"
May 28 18:55:42.273502 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.273475    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wr45w\""
May 28 18:55:42.273502 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.273490    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
May 28 18:55:42.273732 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.273475    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
May 28 18:55:42.273874 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.273850    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56c767b8c7-mh9fw"]
May 28 18:55:42.421011 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.420964    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j788\" (UniqueName: \"kubernetes.io/projected/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-kube-api-access-9j788\") pod \"maas-controller-56c767b8c7-mh9fw\" (UID: \"46f56c04-c3da-4bb2-bbff-f098a08e4cd5\") " pod="opendatahub/maas-controller-56c767b8c7-mh9fw"
May 28 18:55:42.421194 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.421093    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-webhook-cert\") pod \"maas-controller-56c767b8c7-mh9fw\" (UID: \"46f56c04-c3da-4bb2-bbff-f098a08e4cd5\") " pod="opendatahub/maas-controller-56c767b8c7-mh9fw"
May 28 18:55:42.427323 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.427295    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-cb7848758-qxnmd"]
May 28 18:55:42.430231 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.430207    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-cb7848758-qxnmd"
May 28 18:55:42.439091 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.439067    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-cb7848758-qxnmd"]
May 28 18:55:42.522486 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.522389    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-webhook-cert\") pod \"maas-controller-56c767b8c7-mh9fw\" (UID: \"46f56c04-c3da-4bb2-bbff-f098a08e4cd5\") " pod="opendatahub/maas-controller-56c767b8c7-mh9fw"
May 28 18:55:42.522683 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.522529    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9j788\" (UniqueName: \"kubernetes.io/projected/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-kube-api-access-9j788\") pod \"maas-controller-56c767b8c7-mh9fw\" (UID: \"46f56c04-c3da-4bb2-bbff-f098a08e4cd5\") " pod="opendatahub/maas-controller-56c767b8c7-mh9fw"
May 28 18:55:42.522683 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:55:42.522537    2577 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
May 28 18:55:42.522683 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:55:42.522645    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-webhook-cert podName:46f56c04-c3da-4bb2-bbff-f098a08e4cd5 nodeName:}" failed. No retries permitted until 2026-05-28 18:55:43.022600415 +0000 UTC m=+527.281822577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-webhook-cert") pod "maas-controller-56c767b8c7-mh9fw" (UID: "46f56c04-c3da-4bb2-bbff-f098a08e4cd5") : secret "maas-controller-webhook-cert" not found
May 28 18:55:42.531592 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.531564    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j788\" (UniqueName: \"kubernetes.io/projected/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-kube-api-access-9j788\") pod \"maas-controller-56c767b8c7-mh9fw\" (UID: \"46f56c04-c3da-4bb2-bbff-f098a08e4cd5\") " pod="opendatahub/maas-controller-56c767b8c7-mh9fw"
May 28 18:55:42.623198 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.623163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlgq4\" (UniqueName: \"kubernetes.io/projected/023630b9-8a4b-4865-a250-e1990721d70b-kube-api-access-jlgq4\") pod \"maas-controller-cb7848758-qxnmd\" (UID: \"023630b9-8a4b-4865-a250-e1990721d70b\") " pod="opendatahub/maas-controller-cb7848758-qxnmd"
May 28 18:55:42.623361 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.623209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/023630b9-8a4b-4865-a250-e1990721d70b-webhook-cert\") pod \"maas-controller-cb7848758-qxnmd\" (UID: \"023630b9-8a4b-4865-a250-e1990721d70b\") " pod="opendatahub/maas-controller-cb7848758-qxnmd"
May 28 18:55:42.724425 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.724383    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jlgq4\" (UniqueName: \"kubernetes.io/projected/023630b9-8a4b-4865-a250-e1990721d70b-kube-api-access-jlgq4\") pod \"maas-controller-cb7848758-qxnmd\" (UID: \"023630b9-8a4b-4865-a250-e1990721d70b\") " pod="opendatahub/maas-controller-cb7848758-qxnmd"
May 28 18:55:42.724425 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.724429    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/023630b9-8a4b-4865-a250-e1990721d70b-webhook-cert\") pod \"maas-controller-cb7848758-qxnmd\" (UID: \"023630b9-8a4b-4865-a250-e1990721d70b\") " pod="opendatahub/maas-controller-cb7848758-qxnmd"
May 28 18:55:42.726784 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.726761    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/023630b9-8a4b-4865-a250-e1990721d70b-webhook-cert\") pod \"maas-controller-cb7848758-qxnmd\" (UID: \"023630b9-8a4b-4865-a250-e1990721d70b\") " pod="opendatahub/maas-controller-cb7848758-qxnmd"
May 28 18:55:42.733722 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.733695    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlgq4\" (UniqueName: \"kubernetes.io/projected/023630b9-8a4b-4865-a250-e1990721d70b-kube-api-access-jlgq4\") pod \"maas-controller-cb7848758-qxnmd\" (UID: \"023630b9-8a4b-4865-a250-e1990721d70b\") " pod="opendatahub/maas-controller-cb7848758-qxnmd"
May 28 18:55:42.741490 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.741471    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-cb7848758-qxnmd"
May 28 18:55:42.868960 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:42.868935    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-cb7848758-qxnmd"]
May 28 18:55:42.871024 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:55:42.870995    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod023630b9_8a4b_4865_a250_e1990721d70b.slice/crio-36327b5588aafc95dfe01ff3f75f313c8b3b6f619088fe38095853eb778e6590 WatchSource:0}: Error finding container 36327b5588aafc95dfe01ff3f75f313c8b3b6f619088fe38095853eb778e6590: Status 404 returned error can't find the container with id 36327b5588aafc95dfe01ff3f75f313c8b3b6f619088fe38095853eb778e6590
May 28 18:55:43.027541 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:43.027497    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-webhook-cert\") pod \"maas-controller-56c767b8c7-mh9fw\" (UID: \"46f56c04-c3da-4bb2-bbff-f098a08e4cd5\") " pod="opendatahub/maas-controller-56c767b8c7-mh9fw"
May 28 18:55:43.029877 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:43.029859    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-webhook-cert\") pod \"maas-controller-56c767b8c7-mh9fw\" (UID: \"46f56c04-c3da-4bb2-bbff-f098a08e4cd5\") " pod="opendatahub/maas-controller-56c767b8c7-mh9fw"
May 28 18:55:43.184064 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:43.184022    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c767b8c7-mh9fw"
May 28 18:55:43.276071 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:43.276022    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-cb7848758-qxnmd" event={"ID":"023630b9-8a4b-4865-a250-e1990721d70b","Type":"ContainerStarted","Data":"36327b5588aafc95dfe01ff3f75f313c8b3b6f619088fe38095853eb778e6590"}
May 28 18:55:43.323343 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:43.323317    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56c767b8c7-mh9fw"]
May 28 18:55:43.325294 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:55:43.325265    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46f56c04_c3da_4bb2_bbff_f098a08e4cd5.slice/crio-e0e76899ec22eb220ddbc2f20d957c1919aff29c8c1ca2653e969b0916a8be0f WatchSource:0}: Error finding container e0e76899ec22eb220ddbc2f20d957c1919aff29c8c1ca2653e969b0916a8be0f: Status 404 returned error can't find the container with id e0e76899ec22eb220ddbc2f20d957c1919aff29c8c1ca2653e969b0916a8be0f
May 28 18:55:44.282734 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:44.282679    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c767b8c7-mh9fw" event={"ID":"46f56c04-c3da-4bb2-bbff-f098a08e4cd5","Type":"ContainerStarted","Data":"e0e76899ec22eb220ddbc2f20d957c1919aff29c8c1ca2653e969b0916a8be0f"}
May 28 18:55:46.292727 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:46.292684    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-cb7848758-qxnmd" event={"ID":"023630b9-8a4b-4865-a250-e1990721d70b","Type":"ContainerStarted","Data":"1fce235844edf174f05e2a5645508ee84093a08f755d066a22a381ee9cb3dac9"}
May 28 18:55:46.293200 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:46.292791    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-cb7848758-qxnmd"
May 28 18:55:46.293913 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:46.293893    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c767b8c7-mh9fw" event={"ID":"46f56c04-c3da-4bb2-bbff-f098a08e4cd5","Type":"ContainerStarted","Data":"62c4f9c6c8ffe138ba59d71a93a07705920e553fc39733bfcdc366fd76cfceab"}
May 28 18:55:46.294037 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:46.294026    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56c767b8c7-mh9fw"
May 28 18:55:46.311063 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:46.311010    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-cb7848758-qxnmd" podStartSLOduration=1.644227634 podStartE2EDuration="4.310994189s" podCreationTimestamp="2026-05-28 18:55:42 +0000 UTC" firstStartedPulling="2026-05-28 18:55:42.872217027 +0000 UTC m=+527.131439172" lastFinishedPulling="2026-05-28 18:55:45.538983574 +0000 UTC m=+529.798205727" observedRunningTime="2026-05-28 18:55:46.308660846 +0000 UTC m=+530.567883014" watchObservedRunningTime="2026-05-28 18:55:46.310994189 +0000 UTC m=+530.570216352"
May 28 18:55:46.325043 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:46.324990    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-56c767b8c7-mh9fw" podStartSLOduration=2.105584586 podStartE2EDuration="4.324975162s" podCreationTimestamp="2026-05-28 18:55:42 +0000 UTC" firstStartedPulling="2026-05-28 18:55:43.326624917 +0000 UTC m=+527.585847061" lastFinishedPulling="2026-05-28 18:55:45.546015489 +0000 UTC m=+529.805237637" observedRunningTime="2026-05-28 18:55:46.323910324 +0000 UTC m=+530.583132501" watchObservedRunningTime="2026-05-28 18:55:46.324975162 +0000 UTC m=+530.584197315"
May 28 18:55:57.303689 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:57.303653    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56c767b8c7-mh9fw"
May 28 18:55:57.304138 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:57.303918    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-cb7848758-qxnmd"
May 28 18:55:57.361977 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:57.361943    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56c767b8c7-mh9fw"]
May 28 18:55:57.362214 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:57.362182    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-56c767b8c7-mh9fw" podUID="46f56c04-c3da-4bb2-bbff-f098a08e4cd5" containerName="manager" containerID="cri-o://62c4f9c6c8ffe138ba59d71a93a07705920e553fc39733bfcdc366fd76cfceab" gracePeriod=30
May 28 18:55:57.611254 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:57.611232    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c767b8c7-mh9fw"
May 28 18:55:57.669454 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:57.669420    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9j788\" (UniqueName: \"kubernetes.io/projected/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-kube-api-access-9j788\") pod \"46f56c04-c3da-4bb2-bbff-f098a08e4cd5\" (UID: \"46f56c04-c3da-4bb2-bbff-f098a08e4cd5\") "
May 28 18:55:57.669742 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:57.669480    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-webhook-cert\") pod \"46f56c04-c3da-4bb2-bbff-f098a08e4cd5\" (UID: \"46f56c04-c3da-4bb2-bbff-f098a08e4cd5\") "
May 28 18:55:57.671655 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:57.671602    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "46f56c04-c3da-4bb2-bbff-f098a08e4cd5" (UID: "46f56c04-c3da-4bb2-bbff-f098a08e4cd5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 18:55:57.671740 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:57.671630    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-kube-api-access-9j788" (OuterVolumeSpecName: "kube-api-access-9j788") pod "46f56c04-c3da-4bb2-bbff-f098a08e4cd5" (UID: "46f56c04-c3da-4bb2-bbff-f098a08e4cd5"). InnerVolumeSpecName "kube-api-access-9j788". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 18:55:57.771036 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:57.770996    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-webhook-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:55:57.771036 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:57.771031    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j788\" (UniqueName: \"kubernetes.io/projected/46f56c04-c3da-4bb2-bbff-f098a08e4cd5-kube-api-access-9j788\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:55:58.340203 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:58.340161    2577 generic.go:358] "Generic (PLEG): container finished" podID="46f56c04-c3da-4bb2-bbff-f098a08e4cd5" containerID="62c4f9c6c8ffe138ba59d71a93a07705920e553fc39733bfcdc366fd76cfceab" exitCode=0
May 28 18:55:58.340605 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:58.340232    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c767b8c7-mh9fw"
May 28 18:55:58.340605 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:58.340229    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c767b8c7-mh9fw" event={"ID":"46f56c04-c3da-4bb2-bbff-f098a08e4cd5","Type":"ContainerDied","Data":"62c4f9c6c8ffe138ba59d71a93a07705920e553fc39733bfcdc366fd76cfceab"}
May 28 18:55:58.340605 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:58.340334    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c767b8c7-mh9fw" event={"ID":"46f56c04-c3da-4bb2-bbff-f098a08e4cd5","Type":"ContainerDied","Data":"e0e76899ec22eb220ddbc2f20d957c1919aff29c8c1ca2653e969b0916a8be0f"}
May 28 18:55:58.340605 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:58.340353    2577 scope.go:117] "RemoveContainer" containerID="62c4f9c6c8ffe138ba59d71a93a07705920e553fc39733bfcdc366fd76cfceab"
May 28 18:55:58.348963 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:58.348943    2577 scope.go:117] "RemoveContainer" containerID="62c4f9c6c8ffe138ba59d71a93a07705920e553fc39733bfcdc366fd76cfceab"
May 28 18:55:58.349212 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:55:58.349191    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62c4f9c6c8ffe138ba59d71a93a07705920e553fc39733bfcdc366fd76cfceab\": container with ID starting with 62c4f9c6c8ffe138ba59d71a93a07705920e553fc39733bfcdc366fd76cfceab not found: ID does not exist" containerID="62c4f9c6c8ffe138ba59d71a93a07705920e553fc39733bfcdc366fd76cfceab"
May 28 18:55:58.349280 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:58.349222    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62c4f9c6c8ffe138ba59d71a93a07705920e553fc39733bfcdc366fd76cfceab"} err="failed to get container status \"62c4f9c6c8ffe138ba59d71a93a07705920e553fc39733bfcdc366fd76cfceab\": rpc error: code = NotFound desc = could not find container \"62c4f9c6c8ffe138ba59d71a93a07705920e553fc39733bfcdc366fd76cfceab\": container with ID starting with 62c4f9c6c8ffe138ba59d71a93a07705920e553fc39733bfcdc366fd76cfceab not found: ID does not exist"
May 28 18:55:58.363375 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:58.363337    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56c767b8c7-mh9fw"]
May 28 18:55:58.364999 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:55:58.364976    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56c767b8c7-mh9fw"]
May 28 18:56:00.358477 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:00.358441    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46f56c04-c3da-4bb2-bbff-f098a08e4cd5" path="/var/lib/kubelet/pods/46f56c04-c3da-4bb2-bbff-f098a08e4cd5/volumes"
May 28 18:56:16.842992 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:16.842954    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-77dc64f78f-cpqz8"]
May 28 18:56:16.843467 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:16.843365    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46f56c04-c3da-4bb2-bbff-f098a08e4cd5" containerName="manager"
May 28 18:56:16.843467 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:16.843382    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="46f56c04-c3da-4bb2-bbff-f098a08e4cd5" containerName="manager"
May 28 18:56:16.843564 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:16.843546    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="46f56c04-c3da-4bb2-bbff-f098a08e4cd5" containerName="manager"
May 28 18:56:16.845704 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:16.845684    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-77dc64f78f-cpqz8"
May 28 18:56:16.848388 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:16.848359    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-ddz7l\""
May 28 18:56:16.848523 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:16.848424    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
May 28 18:56:16.858141 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:16.858117    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-77dc64f78f-cpqz8"]
May 28 18:56:16.951807 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:16.951762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v45xn\" (UniqueName: \"kubernetes.io/projected/3868ef6d-8e98-484f-9a46-0e826a430efc-kube-api-access-v45xn\") pod \"maas-api-77dc64f78f-cpqz8\" (UID: \"3868ef6d-8e98-484f-9a46-0e826a430efc\") " pod="opendatahub/maas-api-77dc64f78f-cpqz8"
May 28 18:56:16.951986 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:16.951860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3868ef6d-8e98-484f-9a46-0e826a430efc-maas-api-tls\") pod \"maas-api-77dc64f78f-cpqz8\" (UID: \"3868ef6d-8e98-484f-9a46-0e826a430efc\") " pod="opendatahub/maas-api-77dc64f78f-cpqz8"
May 28 18:56:17.052834 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.052790    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v45xn\" (UniqueName: \"kubernetes.io/projected/3868ef6d-8e98-484f-9a46-0e826a430efc-kube-api-access-v45xn\") pod \"maas-api-77dc64f78f-cpqz8\" (UID: \"3868ef6d-8e98-484f-9a46-0e826a430efc\") " pod="opendatahub/maas-api-77dc64f78f-cpqz8"
May 28 18:56:17.053020 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.052861    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3868ef6d-8e98-484f-9a46-0e826a430efc-maas-api-tls\") pod \"maas-api-77dc64f78f-cpqz8\" (UID: \"3868ef6d-8e98-484f-9a46-0e826a430efc\") " pod="opendatahub/maas-api-77dc64f78f-cpqz8"
May 28 18:56:17.055473 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.055446    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3868ef6d-8e98-484f-9a46-0e826a430efc-maas-api-tls\") pod \"maas-api-77dc64f78f-cpqz8\" (UID: \"3868ef6d-8e98-484f-9a46-0e826a430efc\") " pod="opendatahub/maas-api-77dc64f78f-cpqz8"
May 28 18:56:17.064929 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.064901    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v45xn\" (UniqueName: \"kubernetes.io/projected/3868ef6d-8e98-484f-9a46-0e826a430efc-kube-api-access-v45xn\") pod \"maas-api-77dc64f78f-cpqz8\" (UID: \"3868ef6d-8e98-484f-9a46-0e826a430efc\") " pod="opendatahub/maas-api-77dc64f78f-cpqz8"
May 28 18:56:17.156993 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.156954    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-77dc64f78f-cpqz8"
May 28 18:56:17.203304 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.203269    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-qtg8b"]
May 28 18:56:17.206509 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.206485    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-qtg8b"
May 28 18:56:17.209165 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.209140    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
May 28 18:56:17.220518 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.220492    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-qtg8b"]
May 28 18:56:17.302717 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.302595    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-77dc64f78f-cpqz8"]
May 28 18:56:17.305080 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:56:17.305040    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3868ef6d_8e98_484f_9a46_0e826a430efc.slice/crio-e2c3c60683371ad26a8e87086f0b301a07cd1b3e3ec422d17a20d62fb8c9f9bc WatchSource:0}: Error finding container e2c3c60683371ad26a8e87086f0b301a07cd1b3e3ec422d17a20d62fb8c9f9bc: Status 404 returned error can't find the container with id e2c3c60683371ad26a8e87086f0b301a07cd1b3e3ec422d17a20d62fb8c9f9bc
May 28 18:56:17.355355 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.355324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc6ks\" (UniqueName: \"kubernetes.io/projected/1a9f69e6-bd32-45e1-9291-79c46af08eb1-kube-api-access-wc6ks\") pod \"payload-processing-58b6c8fdc7-qtg8b\" (UID: \"1a9f69e6-bd32-45e1-9291-79c46af08eb1\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-qtg8b"
May 28 18:56:17.410321 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.410218    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-77dc64f78f-cpqz8" event={"ID":"3868ef6d-8e98-484f-9a46-0e826a430efc","Type":"ContainerStarted","Data":"e2c3c60683371ad26a8e87086f0b301a07cd1b3e3ec422d17a20d62fb8c9f9bc"}
May 28 18:56:17.456697 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.456650    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wc6ks\" (UniqueName: \"kubernetes.io/projected/1a9f69e6-bd32-45e1-9291-79c46af08eb1-kube-api-access-wc6ks\") pod \"payload-processing-58b6c8fdc7-qtg8b\" (UID: \"1a9f69e6-bd32-45e1-9291-79c46af08eb1\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-qtg8b"
May 28 18:56:17.464441 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.464404    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc6ks\" (UniqueName: \"kubernetes.io/projected/1a9f69e6-bd32-45e1-9291-79c46af08eb1-kube-api-access-wc6ks\") pod \"payload-processing-58b6c8fdc7-qtg8b\" (UID: \"1a9f69e6-bd32-45e1-9291-79c46af08eb1\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-qtg8b"
May 28 18:56:17.524519 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.524480    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-qtg8b"
May 28 18:56:17.659248 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:17.659216    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-qtg8b"]
May 28 18:56:17.662317 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:56:17.662252    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a9f69e6_bd32_45e1_9291_79c46af08eb1.slice/crio-e8a53541b21b579402e447ddca40cb29f0268152d396982265cdd383c0fff25e WatchSource:0}: Error finding container e8a53541b21b579402e447ddca40cb29f0268152d396982265cdd383c0fff25e: Status 404 returned error can't find the container with id e8a53541b21b579402e447ddca40cb29f0268152d396982265cdd383c0fff25e
May 28 18:56:18.415559 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:18.415520    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-qtg8b" event={"ID":"1a9f69e6-bd32-45e1-9291-79c46af08eb1","Type":"ContainerStarted","Data":"e8a53541b21b579402e447ddca40cb29f0268152d396982265cdd383c0fff25e"}
May 28 18:56:20.424802 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:20.424765    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-qtg8b" event={"ID":"1a9f69e6-bd32-45e1-9291-79c46af08eb1","Type":"ContainerStarted","Data":"2066d0082b4b676bc21410f2b21089188e37aeb2fbe6bcc16b4b5c2e4c273d16"}
May 28 18:56:20.425274 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:20.424862    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-qtg8b"
May 28 18:56:20.426152 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:20.426125    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-77dc64f78f-cpqz8" event={"ID":"3868ef6d-8e98-484f-9a46-0e826a430efc","Type":"ContainerStarted","Data":"7d582fa6243d1175714fbcceee4abee520b0d4c0c936505a5225f86af694cc55"}
May 28 18:56:20.426285 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:20.426216    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-77dc64f78f-cpqz8"
May 28 18:56:20.441477 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:20.441430    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-qtg8b" podStartSLOduration=1.5433643780000001 podStartE2EDuration="3.441418488s" podCreationTimestamp="2026-05-28 18:56:17 +0000 UTC" firstStartedPulling="2026-05-28 18:56:17.663982022 +0000 UTC m=+561.923204167" lastFinishedPulling="2026-05-28 18:56:19.56203613 +0000 UTC m=+563.821258277" observedRunningTime="2026-05-28 18:56:20.43926315 +0000 UTC m=+564.698485320" watchObservedRunningTime="2026-05-28 18:56:20.441418488 +0000 UTC m=+564.700640653"
May 28 18:56:20.455892 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:20.455845    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-77dc64f78f-cpqz8" podStartSLOduration=2.202361513 podStartE2EDuration="4.45583156s" podCreationTimestamp="2026-05-28 18:56:16 +0000 UTC" firstStartedPulling="2026-05-28 18:56:17.306522153 +0000 UTC m=+561.565744300" lastFinishedPulling="2026-05-28 18:56:19.559992204 +0000 UTC m=+563.819214347" observedRunningTime="2026-05-28 18:56:20.454257983 +0000 UTC m=+564.713480161" watchObservedRunningTime="2026-05-28 18:56:20.45583156 +0000 UTC m=+564.715053725"
May 28 18:56:26.435705 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:26.435670    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-77dc64f78f-cpqz8"
May 28 18:56:31.432006 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:31.431974    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-qtg8b"
May 28 18:56:56.273367 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:56.273338    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 18:56:56.274654 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:56:56.274635    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 18:57:01.865112 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.865071    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"]
May 28 18:57:01.868781 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.868752    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:01.872671 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.872644    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
May 28 18:57:01.872810 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.872644    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-t4krc\""
May 28 18:57:01.872810 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.872644    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
May 28 18:57:01.872810 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.872653    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
May 28 18:57:01.876768 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.876744    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"]
May 28 18:57:01.965938 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.965903    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"]
May 28 18:57:01.969424 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.969390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0919738c-94e5-4656-941f-130c7d88b075-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:01.969515 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.969459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0919738c-94e5-4656-941f-130c7d88b075-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:01.969515 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.969482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0919738c-94e5-4656-941f-130c7d88b075-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:01.969515 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.969500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj8lx\" (UniqueName: \"kubernetes.io/projected/0919738c-94e5-4656-941f-130c7d88b075-kube-api-access-kj8lx\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:01.969626 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.969537    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0919738c-94e5-4656-941f-130c7d88b075-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:01.969667 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.969605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0919738c-94e5-4656-941f-130c7d88b075-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:01.969756 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.969739    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:01.972421 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.972395    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
May 28 18:57:01.978244 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:01.978217    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"]
May 28 18:57:02.070352 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.070316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16384088-d588-4af5-a4f0-bb485d34cce0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.070535 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.070379    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16384088-d588-4af5-a4f0-bb485d34cce0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.070535 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.070404    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0919738c-94e5-4656-941f-130c7d88b075-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:02.070535 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.070432    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16384088-d588-4af5-a4f0-bb485d34cce0-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.070535 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.070457    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrsm2\" (UniqueName: \"kubernetes.io/projected/16384088-d588-4af5-a4f0-bb485d34cce0-kube-api-access-hrsm2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.070535 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.070480    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0919738c-94e5-4656-941f-130c7d88b075-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:02.070535 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.070510    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16384088-d588-4af5-a4f0-bb485d34cce0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.070860 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.070582    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0919738c-94e5-4656-941f-130c7d88b075-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:02.070860 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.070620    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0919738c-94e5-4656-941f-130c7d88b075-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:02.070860 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.070656    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kj8lx\" (UniqueName: \"kubernetes.io/projected/0919738c-94e5-4656-941f-130c7d88b075-kube-api-access-kj8lx\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:02.070860 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.070688    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0919738c-94e5-4656-941f-130c7d88b075-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:02.070860 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.070720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16384088-d588-4af5-a4f0-bb485d34cce0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.071111 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.070947    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0919738c-94e5-4656-941f-130c7d88b075-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:02.071256 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.071231    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0919738c-94e5-4656-941f-130c7d88b075-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:02.071256 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.071253    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0919738c-94e5-4656-941f-130c7d88b075-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:02.072866 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.072844    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0919738c-94e5-4656-941f-130c7d88b075-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:02.073348 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.073325    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0919738c-94e5-4656-941f-130c7d88b075-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:02.080929 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.080895    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj8lx\" (UniqueName: \"kubernetes.io/projected/0919738c-94e5-4656-941f-130c7d88b075-kube-api-access-kj8lx\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-27pz2\" (UID: \"0919738c-94e5-4656-941f-130c7d88b075\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:02.171480 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.171444    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16384088-d588-4af5-a4f0-bb485d34cce0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.171480 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.171487    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16384088-d588-4af5-a4f0-bb485d34cce0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.171785 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.171540    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16384088-d588-4af5-a4f0-bb485d34cce0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.171785 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.171567    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16384088-d588-4af5-a4f0-bb485d34cce0-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.171785 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.171598    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hrsm2\" (UniqueName: \"kubernetes.io/projected/16384088-d588-4af5-a4f0-bb485d34cce0-kube-api-access-hrsm2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.171785 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.171669    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16384088-d588-4af5-a4f0-bb485d34cce0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.172043 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.172021    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16384088-d588-4af5-a4f0-bb485d34cce0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.172100 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.172078    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16384088-d588-4af5-a4f0-bb485d34cce0-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.172136 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.172104    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16384088-d588-4af5-a4f0-bb485d34cce0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.173881 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.173859    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16384088-d588-4af5-a4f0-bb485d34cce0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.174186 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.174162    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16384088-d588-4af5-a4f0-bb485d34cce0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.179204 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.179178    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrsm2\" (UniqueName: \"kubernetes.io/projected/16384088-d588-4af5-a4f0-bb485d34cce0-kube-api-access-hrsm2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq\" (UID: \"16384088-d588-4af5-a4f0-bb485d34cce0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.179334 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.179322    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:02.281029 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.280988    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:02.336564 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.336537    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"]
May 28 18:57:02.340090 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:57:02.339277    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0919738c_94e5_4656_941f_130c7d88b075.slice/crio-2a2bc16de022d5edf1cc27fe5f7b4ccc89ec2006dfa398fea64a2a6aebf7dc38 WatchSource:0}: Error finding container 2a2bc16de022d5edf1cc27fe5f7b4ccc89ec2006dfa398fea64a2a6aebf7dc38: Status 404 returned error can't find the container with id 2a2bc16de022d5edf1cc27fe5f7b4ccc89ec2006dfa398fea64a2a6aebf7dc38
May 28 18:57:02.341092 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.341073    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 18:57:02.431418 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.431390    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"]
May 28 18:57:02.433186 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:57:02.433138    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16384088_d588_4af5_a4f0_bb485d34cce0.slice/crio-db09b8087fc1a226b1ef1dc97f6d7835d2fe9c01038b508558049581b8f3acb0 WatchSource:0}: Error finding container db09b8087fc1a226b1ef1dc97f6d7835d2fe9c01038b508558049581b8f3acb0: Status 404 returned error can't find the container with id db09b8087fc1a226b1ef1dc97f6d7835d2fe9c01038b508558049581b8f3acb0
May 28 18:57:02.589003 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.588964    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq" event={"ID":"16384088-d588-4af5-a4f0-bb485d34cce0","Type":"ContainerStarted","Data":"db09b8087fc1a226b1ef1dc97f6d7835d2fe9c01038b508558049581b8f3acb0"}
May 28 18:57:02.589994 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:02.589969    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2" event={"ID":"0919738c-94e5-4656-941f-130c7d88b075","Type":"ContainerStarted","Data":"2a2bc16de022d5edf1cc27fe5f7b4ccc89ec2006dfa398fea64a2a6aebf7dc38"}
May 28 18:57:09.620520 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:09.620469    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq" event={"ID":"16384088-d588-4af5-a4f0-bb485d34cce0","Type":"ContainerStarted","Data":"125c3ac140255d2d17b2a35392d73df721ec1b7470d19f3657fdf0b0306de0f9"}
May 28 18:57:09.622738 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:09.622702    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2" event={"ID":"0919738c-94e5-4656-941f-130c7d88b075","Type":"ContainerStarted","Data":"cac875035ee69fa9d5a1886e6d131fda47e0c23df832cb58fe4a562583cf0756"}
May 28 18:57:09.861696 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:09.861644    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"]
May 28 18:57:09.865649 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:09.865629    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:09.868256 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:09.868230    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
May 28 18:57:09.874702 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:09.874678    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"]
May 28 18:57:09.954336 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:09.954299    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96ec260-a541-40e7-9175-10d8fca1cb5d-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:09.954502 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:09.954344    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c96ec260-a541-40e7-9175-10d8fca1cb5d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:09.954502 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:09.954385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv5s2\" (UniqueName: \"kubernetes.io/projected/c96ec260-a541-40e7-9175-10d8fca1cb5d-kube-api-access-kv5s2\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:09.954502 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:09.954430    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96ec260-a541-40e7-9175-10d8fca1cb5d-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:09.954502 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:09.954447    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96ec260-a541-40e7-9175-10d8fca1cb5d-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:09.954502 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:09.954462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96ec260-a541-40e7-9175-10d8fca1cb5d-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.055221 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.055175    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96ec260-a541-40e7-9175-10d8fca1cb5d-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.055221 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.055223    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c96ec260-a541-40e7-9175-10d8fca1cb5d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.055510 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.055257    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kv5s2\" (UniqueName: \"kubernetes.io/projected/c96ec260-a541-40e7-9175-10d8fca1cb5d-kube-api-access-kv5s2\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.055510 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.055301    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96ec260-a541-40e7-9175-10d8fca1cb5d-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.055510 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.055318    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96ec260-a541-40e7-9175-10d8fca1cb5d-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.055510 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.055342    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96ec260-a541-40e7-9175-10d8fca1cb5d-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.055791 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.055757    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c96ec260-a541-40e7-9175-10d8fca1cb5d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.055791 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.055771    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96ec260-a541-40e7-9175-10d8fca1cb5d-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.055975 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.055893    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96ec260-a541-40e7-9175-10d8fca1cb5d-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.057758 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.057736    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96ec260-a541-40e7-9175-10d8fca1cb5d-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.058067 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.058030    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96ec260-a541-40e7-9175-10d8fca1cb5d-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.064477 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.064455    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv5s2\" (UniqueName: \"kubernetes.io/projected/c96ec260-a541-40e7-9175-10d8fca1cb5d-kube-api-access-kv5s2\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn\" (UID: \"c96ec260-a541-40e7-9175-10d8fca1cb5d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.179591 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.179467    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:10.371382 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.371212    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"]
May 28 18:57:10.629761 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.629714    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn" event={"ID":"c96ec260-a541-40e7-9175-10d8fca1cb5d","Type":"ContainerStarted","Data":"54cdf859ca015b9f465671da099a64b568fea80048087bcd6f26a4e76f02c7d1"}
May 28 18:57:10.630242 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:10.629771    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn" event={"ID":"c96ec260-a541-40e7-9175-10d8fca1cb5d","Type":"ContainerStarted","Data":"5ece2aaba3ac5be8e40996284bfd2f6dc78cbd29a6edfcfb421c9a520418f6ec"}
May 28 18:57:15.650446 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:15.650414    2577 generic.go:358] "Generic (PLEG): container finished" podID="16384088-d588-4af5-a4f0-bb485d34cce0" containerID="125c3ac140255d2d17b2a35392d73df721ec1b7470d19f3657fdf0b0306de0f9" exitCode=0
May 28 18:57:15.650870 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:15.650489    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq" event={"ID":"16384088-d588-4af5-a4f0-bb485d34cce0","Type":"ContainerDied","Data":"125c3ac140255d2d17b2a35392d73df721ec1b7470d19f3657fdf0b0306de0f9"}
May 28 18:57:15.651854 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:15.651804    2577 generic.go:358] "Generic (PLEG): container finished" podID="0919738c-94e5-4656-941f-130c7d88b075" containerID="cac875035ee69fa9d5a1886e6d131fda47e0c23df832cb58fe4a562583cf0756" exitCode=0
May 28 18:57:15.651914 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:15.651870    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2" event={"ID":"0919738c-94e5-4656-941f-130c7d88b075","Type":"ContainerDied","Data":"cac875035ee69fa9d5a1886e6d131fda47e0c23df832cb58fe4a562583cf0756"}
May 28 18:57:16.660550 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:16.660504    2577 generic.go:358] "Generic (PLEG): container finished" podID="c96ec260-a541-40e7-9175-10d8fca1cb5d" containerID="54cdf859ca015b9f465671da099a64b568fea80048087bcd6f26a4e76f02c7d1" exitCode=0
May 28 18:57:16.661047 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:16.660682    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn" event={"ID":"c96ec260-a541-40e7-9175-10d8fca1cb5d","Type":"ContainerDied","Data":"54cdf859ca015b9f465671da099a64b568fea80048087bcd6f26a4e76f02c7d1"}
May 28 18:57:17.665966 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:17.665932    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2" event={"ID":"0919738c-94e5-4656-941f-130c7d88b075","Type":"ContainerStarted","Data":"9585380fe722d3f9dc28bb122b33f2f197d3ef12aca1498ab0e109f6c37f4178"}
May 28 18:57:17.666420 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:17.666166    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:17.667586 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:17.667562    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn" event={"ID":"c96ec260-a541-40e7-9175-10d8fca1cb5d","Type":"ContainerStarted","Data":"f57f6db7bc905e8b32786a21ab7750933b961d5246d898a4c0b97ef0a7473509"}
May 28 18:57:17.667789 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:17.667771    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:17.669051 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:17.669032    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq" event={"ID":"16384088-d588-4af5-a4f0-bb485d34cce0","Type":"ContainerStarted","Data":"0187e1e3d15e4b38260e75b54be10912abd9e9c5a17811cbab3f46267a386f5c"}
May 28 18:57:17.669218 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:17.669203    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:57:17.685269 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:17.685224    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2" podStartSLOduration=1.883002071 podStartE2EDuration="16.685211903s" podCreationTimestamp="2026-05-28 18:57:01 +0000 UTC" firstStartedPulling="2026-05-28 18:57:02.341204516 +0000 UTC m=+606.600426664" lastFinishedPulling="2026-05-28 18:57:17.143414349 +0000 UTC m=+621.402636496" observedRunningTime="2026-05-28 18:57:17.68342052 +0000 UTC m=+621.942642698" watchObservedRunningTime="2026-05-28 18:57:17.685211903 +0000 UTC m=+621.944434068"
May 28 18:57:17.701903 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:17.701844    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq" podStartSLOduration=1.9882405859999999 podStartE2EDuration="16.701830474s" podCreationTimestamp="2026-05-28 18:57:01 +0000 UTC" firstStartedPulling="2026-05-28 18:57:02.435205315 +0000 UTC m=+606.694427462" lastFinishedPulling="2026-05-28 18:57:17.148795204 +0000 UTC m=+621.408017350" observedRunningTime="2026-05-28 18:57:17.699887811 +0000 UTC m=+621.959109977" watchObservedRunningTime="2026-05-28 18:57:17.701830474 +0000 UTC m=+621.961052640"
May 28 18:57:17.718304 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:17.718237    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn" podStartSLOduration=8.236343391 podStartE2EDuration="8.718218249s" podCreationTimestamp="2026-05-28 18:57:09 +0000 UTC" firstStartedPulling="2026-05-28 18:57:16.661540535 +0000 UTC m=+620.920762682" lastFinishedPulling="2026-05-28 18:57:17.143415382 +0000 UTC m=+621.402637540" observedRunningTime="2026-05-28 18:57:17.716195482 +0000 UTC m=+621.975417645" watchObservedRunningTime="2026-05-28 18:57:17.718218249 +0000 UTC m=+621.977440416"
May 28 18:57:28.685627 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:28.685586    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-27pz2"
May 28 18:57:28.691157 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:28.691129    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn"
May 28 18:57:28.691897 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:57:28.691875    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq"
May 28 18:58:01.057510 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.057460    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"]
May 28 18:58:01.061737 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.061714    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.064346 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.064324    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
May 28 18:58:01.069310 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.069152    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"]
May 28 18:58:01.135995 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.135963    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.136172 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.136058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snwt6\" (UniqueName: \"kubernetes.io/projected/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-kube-api-access-snwt6\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.136172 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.136129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.136273 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.136185    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.136273 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.136228    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.136273 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.136264    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.237569 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.237532    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-snwt6\" (UniqueName: \"kubernetes.io/projected/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-kube-api-access-snwt6\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.237774 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.237653    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.237774 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.237722    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.237896 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.237775    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.237896 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.237802    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.237896 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.237876    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.238170 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.238130    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.238305 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.238230    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.238364 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.238317    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.240392 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.240288    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.240392 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.240324    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.247421 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.247391    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snwt6\" (UniqueName: \"kubernetes.io/projected/a1790c5b-3b4e-4b34-b1ee-9782ab637dfc-kube-api-access-snwt6\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x\" (UID: \"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.373695 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.373569    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:01.523649 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.523492    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"]
May 28 18:58:01.526638 ip-10-0-137-226 kubenswrapper[2577]: W0528 18:58:01.526593    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1790c5b_3b4e_4b34_b1ee_9782ab637dfc.slice/crio-738bb089c9023f87c4947e3c0f927a5904f3e36aa832b9128b98878628acbc20 WatchSource:0}: Error finding container 738bb089c9023f87c4947e3c0f927a5904f3e36aa832b9128b98878628acbc20: Status 404 returned error can't find the container with id 738bb089c9023f87c4947e3c0f927a5904f3e36aa832b9128b98878628acbc20
May 28 18:58:01.848298 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.848258    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x" event={"ID":"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc","Type":"ContainerStarted","Data":"b74fe5d8b3a4a9338adbe912348d2a2e94c63f1991cd2766901fed5fa3854216"}
May 28 18:58:01.848298 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:01.848301    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x" event={"ID":"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc","Type":"ContainerStarted","Data":"738bb089c9023f87c4947e3c0f927a5904f3e36aa832b9128b98878628acbc20"}
May 28 18:58:07.873054 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:07.873018    2577 generic.go:358] "Generic (PLEG): container finished" podID="a1790c5b-3b4e-4b34-b1ee-9782ab637dfc" containerID="b74fe5d8b3a4a9338adbe912348d2a2e94c63f1991cd2766901fed5fa3854216" exitCode=0
May 28 18:58:07.873432 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:07.873078    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x" event={"ID":"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc","Type":"ContainerDied","Data":"b74fe5d8b3a4a9338adbe912348d2a2e94c63f1991cd2766901fed5fa3854216"}
May 28 18:58:08.878160 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:08.878117    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x" event={"ID":"a1790c5b-3b4e-4b34-b1ee-9782ab637dfc","Type":"ContainerStarted","Data":"4158e6f67d2ce316b7b15c26d4b8dbd73eca9ed98bd58c7d718a9d8640dd508c"}
May 28 18:58:08.878551 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:08.878331    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:58:08.897204 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:08.897161    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x" podStartSLOduration=7.578580567 podStartE2EDuration="7.897148119s" podCreationTimestamp="2026-05-28 18:58:01 +0000 UTC" firstStartedPulling="2026-05-28 18:58:07.873759309 +0000 UTC m=+672.132981453" lastFinishedPulling="2026-05-28 18:58:08.192326861 +0000 UTC m=+672.451549005" observedRunningTime="2026-05-28 18:58:08.894580406 +0000 UTC m=+673.153802573" watchObservedRunningTime="2026-05-28 18:58:08.897148119 +0000 UTC m=+673.156370323"
May 28 18:58:19.894884 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:58:19.894847    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x"
May 28 18:59:46.556580 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:46.556541    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-cb7848758-qxnmd"]
May 28 18:59:46.557122 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:46.556826    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-cb7848758-qxnmd" podUID="023630b9-8a4b-4865-a250-e1990721d70b" containerName="manager" containerID="cri-o://1fce235844edf174f05e2a5645508ee84093a08f755d066a22a381ee9cb3dac9" gracePeriod=30
May 28 18:59:46.806371 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:46.806347    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-cb7848758-qxnmd"
May 28 18:59:46.848384 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:46.848304    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/023630b9-8a4b-4865-a250-e1990721d70b-webhook-cert\") pod \"023630b9-8a4b-4865-a250-e1990721d70b\" (UID: \"023630b9-8a4b-4865-a250-e1990721d70b\") "
May 28 18:59:46.848527 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:46.848398    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlgq4\" (UniqueName: \"kubernetes.io/projected/023630b9-8a4b-4865-a250-e1990721d70b-kube-api-access-jlgq4\") pod \"023630b9-8a4b-4865-a250-e1990721d70b\" (UID: \"023630b9-8a4b-4865-a250-e1990721d70b\") "
May 28 18:59:46.850562 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:46.850535    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/023630b9-8a4b-4865-a250-e1990721d70b-kube-api-access-jlgq4" (OuterVolumeSpecName: "kube-api-access-jlgq4") pod "023630b9-8a4b-4865-a250-e1990721d70b" (UID: "023630b9-8a4b-4865-a250-e1990721d70b"). InnerVolumeSpecName "kube-api-access-jlgq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 18:59:46.850711 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:46.850635    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/023630b9-8a4b-4865-a250-e1990721d70b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "023630b9-8a4b-4865-a250-e1990721d70b" (UID: "023630b9-8a4b-4865-a250-e1990721d70b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 18:59:46.949448 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:46.949409    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/023630b9-8a4b-4865-a250-e1990721d70b-webhook-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:59:46.949448 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:46.949441    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlgq4\" (UniqueName: \"kubernetes.io/projected/023630b9-8a4b-4865-a250-e1990721d70b-kube-api-access-jlgq4\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
May 28 18:59:47.259183 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:47.259143    2577 generic.go:358] "Generic (PLEG): container finished" podID="023630b9-8a4b-4865-a250-e1990721d70b" containerID="1fce235844edf174f05e2a5645508ee84093a08f755d066a22a381ee9cb3dac9" exitCode=0
May 28 18:59:47.259348 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:47.259233    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-cb7848758-qxnmd" event={"ID":"023630b9-8a4b-4865-a250-e1990721d70b","Type":"ContainerDied","Data":"1fce235844edf174f05e2a5645508ee84093a08f755d066a22a381ee9cb3dac9"}
May 28 18:59:47.259348 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:47.259264    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-cb7848758-qxnmd" event={"ID":"023630b9-8a4b-4865-a250-e1990721d70b","Type":"ContainerDied","Data":"36327b5588aafc95dfe01ff3f75f313c8b3b6f619088fe38095853eb778e6590"}
May 28 18:59:47.259348 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:47.259284    2577 scope.go:117] "RemoveContainer" containerID="1fce235844edf174f05e2a5645508ee84093a08f755d066a22a381ee9cb3dac9"
May 28 18:59:47.259511 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:47.259444    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-cb7848758-qxnmd"
May 28 18:59:47.268755 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:47.268732    2577 scope.go:117] "RemoveContainer" containerID="1fce235844edf174f05e2a5645508ee84093a08f755d066a22a381ee9cb3dac9"
May 28 18:59:47.269018 ip-10-0-137-226 kubenswrapper[2577]: E0528 18:59:47.268999    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fce235844edf174f05e2a5645508ee84093a08f755d066a22a381ee9cb3dac9\": container with ID starting with 1fce235844edf174f05e2a5645508ee84093a08f755d066a22a381ee9cb3dac9 not found: ID does not exist" containerID="1fce235844edf174f05e2a5645508ee84093a08f755d066a22a381ee9cb3dac9"
May 28 18:59:47.269081 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:47.269027    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fce235844edf174f05e2a5645508ee84093a08f755d066a22a381ee9cb3dac9"} err="failed to get container status \"1fce235844edf174f05e2a5645508ee84093a08f755d066a22a381ee9cb3dac9\": rpc error: code = NotFound desc = could not find container \"1fce235844edf174f05e2a5645508ee84093a08f755d066a22a381ee9cb3dac9\": container with ID starting with 1fce235844edf174f05e2a5645508ee84093a08f755d066a22a381ee9cb3dac9 not found: ID does not exist"
May 28 18:59:47.283636 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:47.283596    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-cb7848758-qxnmd"]
May 28 18:59:47.288873 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:47.288849    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-cb7848758-qxnmd"]
May 28 18:59:48.360258 ip-10-0-137-226 kubenswrapper[2577]: I0528 18:59:48.360222    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="023630b9-8a4b-4865-a250-e1990721d70b" path="/var/lib/kubelet/pods/023630b9-8a4b-4865-a250-e1990721d70b/volumes"
May 28 19:01:56.303380 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:01:56.303299    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 19:01:56.307485 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:01:56.307458    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 19:06:56.331322 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:06:56.331295    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 19:06:56.336702 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:06:56.336671    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 19:11:56.358851 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:11:56.358822    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 19:11:56.365547 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:11:56.365520    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 19:16:56.385710 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:16:56.385677    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 19:16:56.394068 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:16:56.394045    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 19:21:14.261289 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:14.261222    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-gxxq7_ee73c29d-b2f3-4876-be80-8345b5bf2e71/manager/0.log"
May 28 19:21:14.376327 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:14.376280    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-77dc64f78f-cpqz8_3868ef6d-8e98-484f-9a46-0e826a430efc/maas-api/0.log"
May 28 19:21:14.868747 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:14.868715    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-jtf76_b033be53-342a-4f0d-a7da-76327e02eef5/manager/2.log"
May 28 19:21:15.098092 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:15.098061    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75cc899db8-kpb7q_62ae530e-ee79-4380-98a0-4ef991090cef/manager/0.log"
May 28 19:21:15.314949 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:15.314919    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-8bd5w_937a17ac-4b76-486e-b6f6-f86c98175b28/postgres/0.log"
May 28 19:21:16.772227 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:16.772201    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-lbm6s_8eb1be56-a272-40d1-b758-84a238337f53/manager/0.log"
May 28 19:21:17.349347 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:17.349312    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-bx8ll_3731f783-5532-41bc-9341-043b56fcb1d7/manager/0.log"
May 28 19:21:17.809345 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:17.809312    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-bg9lx_1c5bd532-1000-400d-a898-859a9488ab5f/discovery/0.log"
May 28 19:21:18.026354 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:18.026323    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cf74c98fc-vmhgz_bca077fc-d6dc-4412-aa92-c606a758230a/kube-auth-proxy/0.log"
May 28 19:21:18.266257 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:18.266228    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-qtg8b_1a9f69e6-bd32-45e1-9291-79c46af08eb1/payload-processing/0.log"
May 28 19:21:18.710839 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:18.710809    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x_a1790c5b-3b4e-4b34-b1ee-9782ab637dfc/storage-initializer/0.log"
May 28 19:21:18.718370 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:18.718340    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-d6g2x_a1790c5b-3b4e-4b34-b1ee-9782ab637dfc/main/0.log"
May 28 19:21:18.830004 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:18.829971    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn_c96ec260-a541-40e7-9175-10d8fca1cb5d/storage-initializer/0.log"
May 28 19:21:18.837760 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:18.837737    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-2r9fn_c96ec260-a541-40e7-9175-10d8fca1cb5d/main/0.log"
May 28 19:21:18.942153 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:18.942125    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-27pz2_0919738c-94e5-4656-941f-130c7d88b075/storage-initializer/0.log"
May 28 19:21:18.949378 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:18.949349    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-27pz2_0919738c-94e5-4656-941f-130c7d88b075/main/0.log"
May 28 19:21:19.070039 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:19.069941    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq_16384088-d588-4af5-a4f0-bb485d34cce0/storage-initializer/0.log"
May 28 19:21:19.083507 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:19.083478    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccn2jbq_16384088-d588-4af5-a4f0-bb485d34cce0/main/0.log"
May 28 19:21:25.965565 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:25.965534    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bd8c2_5474ed71-aeca-4921-9680-b338954bf5fd/global-pull-secret-syncer/0.log"
May 28 19:21:26.203895 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:26.203850    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pm7cd_532d9740-8a10-452a-88d7-64ecd63e0f66/konnectivity-agent/0.log"
May 28 19:21:26.306883 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:26.306806    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-226.ec2.internal_ec0af40b48c17550ff6150b136d1afe4/haproxy/0.log"
May 28 19:21:30.544406 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:30.544371    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-lbm6s_8eb1be56-a272-40d1-b758-84a238337f53/manager/0.log"
May 28 19:21:30.756898 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:30.756864    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-bx8ll_3731f783-5532-41bc-9341-043b56fcb1d7/manager/0.log"
May 28 19:21:32.337189 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.337085    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6c75959455-bk4zh_932039c7-8302-443b-9af2-30c993764ffa/cluster-monitoring-operator/0.log"
May 28 19:21:32.467139 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.467097    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-577976b98c-s56wz_8cbd2029-9fd3-45cc-a807-98245e5b0d75/metrics-server/0.log"
May 28 19:21:32.490128 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.490076    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-665c9c8584-w2mxt_89ab8170-7fd7-4238-9213-ba216b213b30/monitoring-plugin/0.log"
May 28 19:21:32.641085 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.641038    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r4d59_f8c10cda-fc2b-487c-8b95-44064737610d/node-exporter/0.log"
May 28 19:21:32.662273 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.662248    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r4d59_f8c10cda-fc2b-487c-8b95-44064737610d/kube-rbac-proxy/0.log"
May 28 19:21:32.678294 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.678271    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r4d59_f8c10cda-fc2b-487c-8b95-44064737610d/init-textfile/0.log"
May 28 19:21:32.772769 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.772727    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc/prometheus/0.log"
May 28 19:21:32.789413 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.789384    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc/config-reloader/0.log"
May 28 19:21:32.809009 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.808978    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc/thanos-sidecar/0.log"
May 28 19:21:32.822989 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.822964    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc/kube-rbac-proxy-web/0.log"
May 28 19:21:32.841856 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.841833    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc/kube-rbac-proxy/0.log"
May 28 19:21:32.861887 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.861867    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc/kube-rbac-proxy-thanos/0.log"
May 28 19:21:32.881874 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.881849    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_57ee20ae-0bc5-4acb-bf1a-43c0d0d4f3cc/init-config-reloader/0.log"
May 28 19:21:32.953448 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.953375    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-4qpd4_d80cb94e-6a08-46a3-8865-57b8b7029b3c/prometheus-operator-admission-webhook/0.log"
May 28 19:21:32.979580 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.979554    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77b4bfff95-6xj9h_5537b1f2-9cad-4fd5-8953-213489037a0d/telemeter-client/0.log"
May 28 19:21:32.996834 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:32.996811    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77b4bfff95-6xj9h_5537b1f2-9cad-4fd5-8953-213489037a0d/reload/0.log"
May 28 19:21:33.014803 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:33.014782    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77b4bfff95-6xj9h_5537b1f2-9cad-4fd5-8953-213489037a0d/kube-rbac-proxy/0.log"
May 28 19:21:34.717499 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.717469    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"]
May 28 19:21:34.717911 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.717896    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="023630b9-8a4b-4865-a250-e1990721d70b" containerName="manager"
May 28 19:21:34.717959 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.717913    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="023630b9-8a4b-4865-a250-e1990721d70b" containerName="manager"
May 28 19:21:34.717994 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.717989    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="023630b9-8a4b-4865-a250-e1990721d70b" containerName="manager"
May 28 19:21:34.720245 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.720228    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.722987 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.722966    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-r6z8d\"/\"openshift-service-ca.crt\""
May 28 19:21:34.723123 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.723098    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-r6z8d\"/\"default-dockercfg-hbxd5\""
May 28 19:21:34.724209 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.724194    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-r6z8d\"/\"kube-root-ca.crt\""
May 28 19:21:34.728313 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.728292    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"]
May 28 19:21:34.799059 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.799026    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/1.log"
May 28 19:21:34.803158 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.803136    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-kshzv_3f6a673c-afbc-441f-a108-f66950a3558b/console-operator/2.log"
May 28 19:21:34.859942 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.859899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/906c5a01-c740-425f-86f3-b4a6f43c88ad-lib-modules\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.859942 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.859944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/906c5a01-c740-425f-86f3-b4a6f43c88ad-proc\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.860171 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.860038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc97j\" (UniqueName: \"kubernetes.io/projected/906c5a01-c740-425f-86f3-b4a6f43c88ad-kube-api-access-kc97j\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.860171 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.860081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/906c5a01-c740-425f-86f3-b4a6f43c88ad-podres\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.860171 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.860108    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/906c5a01-c740-425f-86f3-b4a6f43c88ad-sys\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.961039 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.961000    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/906c5a01-c740-425f-86f3-b4a6f43c88ad-lib-modules\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.961237 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.961046    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/906c5a01-c740-425f-86f3-b4a6f43c88ad-proc\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.961237 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.961137    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kc97j\" (UniqueName: \"kubernetes.io/projected/906c5a01-c740-425f-86f3-b4a6f43c88ad-kube-api-access-kc97j\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.961237 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.961182    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/906c5a01-c740-425f-86f3-b4a6f43c88ad-lib-modules\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.961237 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.961192    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/906c5a01-c740-425f-86f3-b4a6f43c88ad-proc\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.961237 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.961192    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/906c5a01-c740-425f-86f3-b4a6f43c88ad-podres\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.961436 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.961254    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/906c5a01-c740-425f-86f3-b4a6f43c88ad-sys\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.961436 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.961290    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/906c5a01-c740-425f-86f3-b4a6f43c88ad-podres\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.961436 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.961321    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/906c5a01-c740-425f-86f3-b4a6f43c88ad-sys\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:34.973188 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:34.973120    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc97j\" (UniqueName: \"kubernetes.io/projected/906c5a01-c740-425f-86f3-b4a6f43c88ad-kube-api-access-kc97j\") pod \"perf-node-gather-daemonset-vpk64\" (UID: \"906c5a01-c740-425f-86f3-b4a6f43c88ad\") " pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:35.031220 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:35.031183    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:35.173640 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:35.173586    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"]
May 28 19:21:35.175944 ip-10-0-137-226 kubenswrapper[2577]: W0528 19:21:35.175913    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod906c5a01_c740_425f_86f3_b4a6f43c88ad.slice/crio-298f821681f18bf4d219eb3b776336f9aed41daa851668c82e1655f929635c3b WatchSource:0}: Error finding container 298f821681f18bf4d219eb3b776336f9aed41daa851668c82e1655f929635c3b: Status 404 returned error can't find the container with id 298f821681f18bf4d219eb3b776336f9aed41daa851668c82e1655f929635c3b
May 28 19:21:35.177602 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:35.177586    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 19:21:35.333945 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:35.333913    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-t4dk9_addb0542-e63c-4184-bbe8-1e21d45a4f68/download-server/0.log"
May 28 19:21:36.138974 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:36.138942    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64" event={"ID":"906c5a01-c740-425f-86f3-b4a6f43c88ad","Type":"ContainerStarted","Data":"c1bdab1166808aa9bc77270a88e84ee4e542585e6edcbdc5851725c82a2ad727"}
May 28 19:21:36.139383 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:36.138980    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64" event={"ID":"906c5a01-c740-425f-86f3-b4a6f43c88ad","Type":"ContainerStarted","Data":"298f821681f18bf4d219eb3b776336f9aed41daa851668c82e1655f929635c3b"}
May 28 19:21:36.139383 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:36.139087    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:36.158412 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:36.158359    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64" podStartSLOduration=2.158344506 podStartE2EDuration="2.158344506s" podCreationTimestamp="2026-05-28 19:21:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 19:21:36.155950837 +0000 UTC m=+2080.415173005" watchObservedRunningTime="2026-05-28 19:21:36.158344506 +0000 UTC m=+2080.417566671"
May 28 19:21:36.733804 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:36.733780    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4n69j_0c79a026-b52c-4e1e-9720-1bbe00a9ed38/dns/0.log"
May 28 19:21:36.753348 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:36.753323    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4n69j_0c79a026-b52c-4e1e-9720-1bbe00a9ed38/kube-rbac-proxy/0.log"
May 28 19:21:36.831810 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:36.831783    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c4fhx_a83d99d8-7c19-4e57-aca6-26f8a033d681/dns-node-resolver/0.log"
May 28 19:21:37.314779 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:37.314736    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-676cdccb6-g86th_4cea4760-a4e5-43f5-9a87-f1f2094eaddd/registry/0.log"
May 28 19:21:37.351505 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:37.351477    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qbwcr_4b1230db-d35f-489e-a592-e9d91bbb6d74/node-ca/0.log"
May 28 19:21:38.210634 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:38.210581    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-bg9lx_1c5bd532-1000-400d-a898-859a9488ab5f/discovery/0.log"
May 28 19:21:38.253277 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:38.253248    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cf74c98fc-vmhgz_bca077fc-d6dc-4412-aa92-c606a758230a/kube-auth-proxy/0.log"
May 28 19:21:38.375374 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:38.370713    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-qtg8b_1a9f69e6-bd32-45e1-9291-79c46af08eb1/payload-processing/0.log"
May 28 19:21:38.950737 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:38.950711    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hmmqn_74ae6b85-71a4-4b53-a23e-4975b6464338/serve-healthcheck-canary/0.log"
May 28 19:21:39.409361 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:39.409330    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-jx2bq_3a0962b8-484f-46c3-8f88-4eaf3130145e/insights-operator/0.log"
May 28 19:21:39.410566 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:39.410545    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-jx2bq_3a0962b8-484f-46c3-8f88-4eaf3130145e/insights-operator/1.log"
May 28 19:21:39.464228 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:39.464199    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6f9td_eff66dae-0031-4fe2-907d-85f57640b231/kube-rbac-proxy/0.log"
May 28 19:21:39.498469 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:39.498443    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6f9td_eff66dae-0031-4fe2-907d-85f57640b231/exporter/0.log"
May 28 19:21:39.523473 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:39.523444    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6f9td_eff66dae-0031-4fe2-907d-85f57640b231/extractor/0.log"
May 28 19:21:41.499930 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:41.499894    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-gxxq7_ee73c29d-b2f3-4876-be80-8345b5bf2e71/manager/0.log"
May 28 19:21:41.550211 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:41.550173    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-77dc64f78f-cpqz8_3868ef6d-8e98-484f-9a46-0e826a430efc/maas-api/0.log"
May 28 19:21:41.799888 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:41.799799    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-jtf76_b033be53-342a-4f0d-a7da-76327e02eef5/manager/1.log"
May 28 19:21:41.810344 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:41.810320    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-jtf76_b033be53-342a-4f0d-a7da-76327e02eef5/manager/2.log"
May 28 19:21:41.899644 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:41.899593    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75cc899db8-kpb7q_62ae530e-ee79-4380-98a0-4ef991090cef/manager/0.log"
May 28 19:21:41.945830 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:41.945802    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-8bd5w_937a17ac-4b76-486e-b6f6-f86c98175b28/postgres/0.log"
May 28 19:21:42.153025 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:42.153000    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-r6z8d/perf-node-gather-daemonset-vpk64"
May 28 19:21:47.412413 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:47.412381    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-qqsdr_df63be57-56fd-4e9a-b2a2-f16efc2ccf4d/migrator/0.log"
May 28 19:21:47.429376 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:47.429351    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-qqsdr_df63be57-56fd-4e9a-b2a2-f16efc2ccf4d/graceful-termination/0.log"
May 28 19:21:47.825881 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:47.825805    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-h85g9_4f285834-d577-4e96-b077-8d242366862e/kube-storage-version-migrator-operator/1.log"
May 28 19:21:47.826713 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:47.826695    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-h85g9_4f285834-d577-4e96-b077-8d242366862e/kube-storage-version-migrator-operator/0.log"
May 28 19:21:48.883498 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:48.883466    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-98z5n_90a5b717-a5bb-4b1e-b516-b3dec5e1601e/kube-multus/0.log"
May 28 19:21:49.138355 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:49.138287    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cg96j_01a5120d-33a6-42dd-b48c-7490c470f11f/kube-multus-additional-cni-plugins/0.log"
May 28 19:21:49.156722 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:49.156690    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cg96j_01a5120d-33a6-42dd-b48c-7490c470f11f/egress-router-binary-copy/0.log"
May 28 19:21:49.177055 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:49.177029    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cg96j_01a5120d-33a6-42dd-b48c-7490c470f11f/cni-plugins/0.log"
May 28 19:21:49.203366 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:49.203342    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cg96j_01a5120d-33a6-42dd-b48c-7490c470f11f/bond-cni-plugin/0.log"
May 28 19:21:49.227753 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:49.227729    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cg96j_01a5120d-33a6-42dd-b48c-7490c470f11f/routeoverride-cni/0.log"
May 28 19:21:49.249305 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:49.249285    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cg96j_01a5120d-33a6-42dd-b48c-7490c470f11f/whereabouts-cni-bincopy/0.log"
May 28 19:21:49.270172 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:49.270149    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cg96j_01a5120d-33a6-42dd-b48c-7490c470f11f/whereabouts-cni/0.log"
May 28 19:21:49.505302 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:49.505220    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9xhpt_1625d5d8-1437-4fb6-b18e-320acb890173/network-metrics-daemon/0.log"
May 28 19:21:49.526421 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:49.526397    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9xhpt_1625d5d8-1437-4fb6-b18e-320acb890173/kube-rbac-proxy/0.log"
May 28 19:21:50.424795 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:50.424764    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9h2tl_1573bbfa-cdc9-48b4-a667-ab2be639d3a8/ovn-controller/0.log"
May 28 19:21:50.460737 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:50.460704    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9h2tl_1573bbfa-cdc9-48b4-a667-ab2be639d3a8/ovn-acl-logging/0.log"
May 28 19:21:50.478631 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:50.478586    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9h2tl_1573bbfa-cdc9-48b4-a667-ab2be639d3a8/kube-rbac-proxy-node/0.log"
May 28 19:21:50.498402 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:50.498375    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9h2tl_1573bbfa-cdc9-48b4-a667-ab2be639d3a8/kube-rbac-proxy-ovn-metrics/0.log"
May 28 19:21:50.518317 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:50.518260    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9h2tl_1573bbfa-cdc9-48b4-a667-ab2be639d3a8/northd/0.log"
May 28 19:21:50.537630 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:50.537538    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9h2tl_1573bbfa-cdc9-48b4-a667-ab2be639d3a8/nbdb/0.log"
May 28 19:21:50.557803 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:50.557784    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9h2tl_1573bbfa-cdc9-48b4-a667-ab2be639d3a8/sbdb/0.log"
May 28 19:21:50.656155 ip-10-0-137-226 kubenswrapper[2577]: I0528 19:21:50.656130    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9h2tl_1573bbfa-cdc9-48b4-a667-ab2be639d3a8/ovnkube-controller/0.log"