May 21 16:29:19.115195 ip-10-0-136-21 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 21 16:29:19.115208 ip-10-0-136-21 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 21 16:29:19.115217 ip-10-0-136-21 systemd[1]: kubelet.service: Failed with result 'resources'.
May 21 16:29:19.115581 ip-10-0-136-21 systemd[1]: Failed to start Kubernetes Kubelet.
May 21 16:29:29.299234 ip-10-0-136-21 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
May 21 16:29:29.299250 ip-10-0-136-21 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7a0b5111593f458d8907cbfbad330eee --
May 21 16:31:49.406347 ip-10-0-136-21 systemd[1]: Starting Kubernetes Kubelet...
May 21 16:31:49.867596 ip-10-0-136-21 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 21 16:31:49.867596 ip-10-0-136-21 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 21 16:31:49.867596 ip-10-0-136-21 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 21 16:31:49.867596 ip-10-0-136-21 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 21 16:31:49.867596 ip-10-0-136-21 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 21 16:31:49.870213 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.870122    2562 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 21 16:31:49.874758 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874740    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 21 16:31:49.874758 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874759    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874763    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874766    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874769    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874786    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874790    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874793    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874796    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874799    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874801    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874804    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874812    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874815    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874818    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874820    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874823    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874825    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874829    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874831    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874834    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 21 16:31:49.874839 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874836    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874839    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874842    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874844    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874847    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874850    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874853    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874856    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874858    2562 feature_gate.go:328] unrecognized feature gate: Example
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874861    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874863    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874866    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874868    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874871    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874888    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874891    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874894    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874898    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874901    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874904    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 21 16:31:49.875369 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874906    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874909    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874911    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874914    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874916    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874918    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874921    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874923    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874925    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874928    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874930    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874933    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874935    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874937    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874941    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874944    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874946    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874949    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874952    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874954    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 21 16:31:49.875935 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874956    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874959    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874962    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874965    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874971    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874983    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874986    2562 feature_gate.go:328] unrecognized feature gate: Example2
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874989    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874992    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874994    2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874996    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.874999    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875002    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875005    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875007    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875010    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875013    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875015    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875018    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 21 16:31:49.876470 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875020    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875023    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875025    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875028    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875030    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875032    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875510    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875517    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875521    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875524    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875526    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875529    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875531    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875534    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875537    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875540    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875542    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875545    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875553    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875556    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 21 16:31:49.876955 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875558    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875560    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875563    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875565    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875568    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875570    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875573    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875575    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875578    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875580    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875582    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875585    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875587    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875589    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875591    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875594    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875596    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875599    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875601    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875605    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
May 21 16:31:49.877509 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875607    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875610    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875612    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875615    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875617    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875620    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875622    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875626    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875629    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875632    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875634    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875642    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875645    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875647    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875650    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875653    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875656    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875658    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875660    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 21 16:31:49.878067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875663    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875665    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875668    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875670    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875672    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875675    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875677    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875681    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875684    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875687    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875690    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875693    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875697    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875700    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875702    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875705    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875707    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875710    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875712    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875715    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 21 16:31:49.878568 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875717    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875719    2562 feature_gate.go:328] unrecognized feature gate: Example2
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875722    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875724    2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875727    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875734    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875737    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875740    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875742    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875745    2562 feature_gate.go:328] unrecognized feature gate: Example
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875747    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875749    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.875752    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875840    2562 flags.go:64] FLAG: --address="0.0.0.0"
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875854    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875867    2562 flags.go:64] FLAG: --anonymous-auth="true"
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875872    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875876    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875879    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875884    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875888    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 21 16:31:49.879116 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875891    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875894    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875897    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875901    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875904    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875915    2562 flags.go:64] FLAG: --cgroup-root=""
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875918    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875921    2562 flags.go:64] FLAG: --client-ca-file=""
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875924    2562 flags.go:64] FLAG: --cloud-config=""
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875926    2562 flags.go:64] FLAG: --cloud-provider="external"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875929    2562 flags.go:64] FLAG: --cluster-dns="[]"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875936    2562 flags.go:64] FLAG: --cluster-domain=""
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875939    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875942    2562 flags.go:64] FLAG: --config-dir=""
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875945    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875949    2562 flags.go:64] FLAG: --container-log-max-files="5"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875953    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875961    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875964    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875968    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875971    2562 flags.go:64] FLAG: --contention-profiling="false"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875974    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875976    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875980    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875983    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 21 16:31:49.879672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875987    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875990    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875993    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875995    2562 flags.go:64] FLAG: --enable-load-reader="false"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.875998    2562 flags.go:64] FLAG: --enable-server="true"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876001    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876007    2562 flags.go:64] FLAG: --event-burst="100"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876010    2562 flags.go:64] FLAG: --event-qps="50"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876013    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876016    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876020    2562 flags.go:64] FLAG: --eviction-hard=""
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876024    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876027    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876030    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876033    2562 flags.go:64] FLAG: --eviction-soft=""
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876036    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876039    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876042    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876045    2562 flags.go:64] FLAG: --experimental-mounter-path=""
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876064    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876067    2562 flags.go:64] FLAG: --fail-swap-on="true"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876070    2562 flags.go:64] FLAG: --feature-gates=""
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876074    2562 flags.go:64] FLAG: --file-check-frequency="20s"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876077    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876080    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 21 16:31:49.880382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876089    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876092    2562 flags.go:64] FLAG: --healthz-port="10248"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876096    2562 flags.go:64] FLAG: --help="false"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876099    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-136-21.ec2.internal"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876102    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876105    2562 flags.go:64] FLAG: --http-check-frequency="20s"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876108    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876111    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876114    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876117    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876120    2562 flags.go:64] FLAG: --image-service-endpoint=""
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876122    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876125    2562 flags.go:64] FLAG: --kube-api-burst="100"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876128    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876131    2562 flags.go:64] FLAG: --kube-api-qps="50"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876134    2562 flags.go:64] FLAG: --kube-reserved=""
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876137    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876140    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876143    2562 flags.go:64] FLAG: --kubelet-cgroups=""
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876146    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876148    2562 flags.go:64] FLAG: --lock-file=""
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876151    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876154    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876157    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 21 16:31:49.881026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876162    2562 flags.go:64] FLAG: --log-json-split-stream="false"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876165    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876167    2562 flags.go:64] FLAG: --log-text-split-stream="false"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876170    2562 flags.go:64] FLAG: --logging-format="text"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876173    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876176    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876179    2562 flags.go:64] FLAG: --manifest-url=""
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876181    2562 flags.go:64] FLAG: --manifest-url-header=""
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876185    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876194    2562 flags.go:64] FLAG: --max-open-files="1000000"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876198    2562 flags.go:64] FLAG: --max-pods="110"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876201    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876204    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876207    2562 flags.go:64] FLAG: --memory-manager-policy="None"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876211    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876214    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876217    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876219    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876228    2562 flags.go:64] FLAG: --node-status-max-images="50"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876231    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876234    2562 flags.go:64] FLAG: --oom-score-adj="-999"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876237    2562 flags.go:64] FLAG: --pod-cidr=""
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876239    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 21 16:31:49.881710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876246    2562 flags.go:64] FLAG: --pod-manifest-path=""
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876248    2562 flags.go:64] FLAG: --pod-max-pids="-1"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876251    2562 flags.go:64] FLAG: --pods-per-core="0"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876255    2562 flags.go:64] FLAG: --port="10250"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876258    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876260    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0408dca4cc3276ce1"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876263    2562 flags.go:64] FLAG: --qos-reserved=""
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876266    2562 flags.go:64] FLAG: --read-only-port="10255"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876269    2562 flags.go:64] FLAG: --register-node="true"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876271    2562 flags.go:64] FLAG: --register-schedulable="true"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876274    2562 flags.go:64] FLAG: --register-with-taints=""
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876278    2562 flags.go:64] FLAG: --registry-burst="10"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876280    2562 flags.go:64] FLAG: --registry-qps="5"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876283    2562 flags.go:64] FLAG: --reserved-cpus=""
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876286    2562 flags.go:64] FLAG: --reserved-memory=""
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876290    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876292    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876295    2562 flags.go:64] FLAG: --rotate-certificates="false"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876298    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876307    2562 flags.go:64] FLAG: --runonce="false"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876310    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876312    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876315    2562 flags.go:64] FLAG: --seccomp-default="false"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876318    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876320    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876323    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 21 16:31:49.882370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876326    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876329    2562 flags.go:64] FLAG: --storage-driver-password="root"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876332    2562 flags.go:64] FLAG: --storage-driver-secure="false"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876334    2562 flags.go:64] FLAG: --storage-driver-table="stats"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876337    2562 flags.go:64] FLAG: --storage-driver-user="root"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876340    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876343    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876345    2562 flags.go:64] FLAG: --system-cgroups=""
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876348    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876357    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876360    2562 flags.go:64] FLAG: --tls-cert-file=""
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876363    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876370    2562 flags.go:64] FLAG: --tls-min-version=""
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876373    2562 flags.go:64] FLAG: --tls-private-key-file=""
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876376    2562 flags.go:64] FLAG: --topology-manager-policy="none"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876379    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876384    2562 flags.go:64] FLAG: --topology-manager-scope="container"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876387    2562 flags.go:64] FLAG: --v="2"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876392    2562 flags.go:64] FLAG: --version="false"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876396    2562 flags.go:64] FLAG: --vmodule=""
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876400    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.876403    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876527    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876531    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 21 16:31:49.883119 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876534    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876537    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876540    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876543    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876546    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876549    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876551    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876554    2562 feature_gate.go:328] unrecognized feature gate: Example
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876556    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876558    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876561    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876564    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876566    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876569    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876571    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876573    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876576    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876579    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876581    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876584    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 21 16:31:49.883761 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876586    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876589    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876592    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876594    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876598    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876600    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876603    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876605    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876608    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876610    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876612    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876615    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876619    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876622    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876625    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876629    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876632    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876634    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876637    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876639    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 21 16:31:49.884407 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876641    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876645    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876648    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876651    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876653    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876655    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876658    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876660    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876663    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876666    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876668    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876671    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876673    2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876675    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876678    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876680    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876684    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876686    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876689    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 21 16:31:49.884942 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876691    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876693    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876696    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876698    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876701    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876703    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876706    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876708    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876711    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876713    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876716    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876718    2562 feature_gate.go:328] unrecognized feature gate: Example2
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876721    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876723    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876725    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876728    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876730    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876733    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876735    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876738    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 21 16:31:49.885497 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876740    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876742    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876747    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876749    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.876752    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.877695    2562 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.884692    2562 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.884714    2562 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884776    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884782    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884785    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884788    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884791    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884794    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884799    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 21 16:31:49.886040 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884803    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884806    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884810    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884813    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884815    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884818    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884820    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884823    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884826    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884828    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884830    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884833    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884835    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884838    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884840    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884843    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884846    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884848    2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884850    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884853    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 21 16:31:49.886508 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884856    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884859    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884861    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884864    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884873    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884876    2562 feature_gate.go:328] unrecognized feature gate: Example2
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884878    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884881    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884883    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884886    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884888    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884891    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884893    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884896    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884898    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884900    2562 feature_gate.go:328] unrecognized feature gate: Example
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884903    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884905    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884908    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884910    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 21 16:31:49.887067 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884912    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884915    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884917    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884920    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884922    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884924    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884927    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884929    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884932    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884934    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884937    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884939    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884942    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884945    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884947    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884950    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884953    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884959    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884962    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884965    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 21 16:31:49.887598 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884968    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884970    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884973    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884975    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884978    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884980    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884982    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884985    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884987    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884990    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884992    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884994    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884997    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.884999    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885002    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885004    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885007    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885009    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 21 16:31:49.888144 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885011    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.885016    2562 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885186    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885193    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885196    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885199    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885202    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885205    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885207    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885210    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885213    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885215    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885218    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885221    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885224    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 21 16:31:49.888626 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885228    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885231    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885234    2562 feature_gate.go:328] unrecognized feature gate: Example
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885237    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885240    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885243    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885246    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885248    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885251    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885254    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885256    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885259    2562 feature_gate.go:328] unrecognized feature gate: Example2
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885261    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885263    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885266    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885268    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885272    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885274    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885276    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885279    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 21 16:31:49.889096 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885281    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885284    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885286    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885289    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885291    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885293    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885296    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885300    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885304    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885307    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885311    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885313    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885316    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885318    2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885321    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885323    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885325    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885328    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885330    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
May 21 16:31:49.889634 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885333    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885335    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885337    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885340    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885342    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885344    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885347    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885349    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885351    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885354    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885356    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885358    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885361    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885363    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885366    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885368    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885370    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885373    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885375    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885377    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 21 16:31:49.890284 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885380    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885383    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885385    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885387    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885390    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885393    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885395    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885398    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885400    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885402    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885405    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885407    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885410    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:49.885412    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.885417    2562 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 21 16:31:49.890860 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.886425    2562 server.go:962] "Client rotation is on, will bootstrap in background"
May 21 16:31:49.891297 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.888544    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 21 16:31:49.891297 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.889675    2562 server.go:1019] "Starting client certificate rotation"
May 21 16:31:49.891297 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.889785    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 21 16:31:49.891403 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.891387    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 21 16:31:49.918520 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.918490    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 21 16:31:49.921945 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.921562    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 21 16:31:49.933843 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.933826    2562 log.go:25] "Validated CRI v1 runtime API"
May 21 16:31:49.939174 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.939154    2562 log.go:25] "Validated CRI v1 image API"
May 21 16:31:49.945744 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.945723    2562 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 21 16:31:49.949172 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.949149    2562 fs.go:135] Filesystem UUIDs: map[410543f3-e92d-4821-8e48-50af8dcfcc0b:/dev/nvme0n1p4 42d19efd-c352-439c-8851-c187f5d7ed0c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
May 21 16:31:49.949251 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.949170    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 21 16:31:49.954152 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.954024    2562 manager.go:217] Machine: {Timestamp:2026-05-21 16:31:49.953461745 +0000 UTC m=+0.424582908 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099395 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec230e1773aa0ba26b248993af98a964 SystemUUID:ec230e17-73aa-0ba2-6b24-8993af98a964 BootID:7a0b5111-593f-458d-8907-cbfbad330eee Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4f:ba:46:c8:01 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4f:ba:46:c8:01 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:75:b6:73:6a:42 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 21 16:31:49.955244 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.955231    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 21 16:31:49.955398 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.955345    2562 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 21 16:31:49.955463 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.955395    2562 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 21 16:31:49.959372 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.959342    2562 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 21 16:31:49.959525 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.959375    2562 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-21.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 21 16:31:49.959578 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.959536    2562 topology_manager.go:138] "Creating topology manager with none policy"
May 21 16:31:49.959578 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.959545    2562 container_manager_linux.go:306] "Creating device plugin manager"
May 21 16:31:49.959578 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.959556    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 21 16:31:49.960172 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.960162    2562 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 21 16:31:49.961790 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.961778    2562 state_mem.go:36] "Initialized new in-memory state store"
May 21 16:31:49.961909 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.961900    2562 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 21 16:31:49.964811 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.964646    2562 kubelet.go:491] "Attempting to sync node with API server"
May 21 16:31:49.964811 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.964657    2562 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 21 16:31:49.964811 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.964692    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 21 16:31:49.964811 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.964701    2562 kubelet.go:397] "Adding apiserver pod source"
May 21 16:31:49.964811 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.964710    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 21 16:31:49.966064 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.966036    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 21 16:31:49.966172 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.966161    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 21 16:31:49.970243 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.970224    2562 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 21 16:31:49.973266 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.973241    2562 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 21 16:31:49.975483 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.975464    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 21 16:31:49.975565 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.975489    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 21 16:31:49.975565 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.975498    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 21 16:31:49.975565 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.975507    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 21 16:31:49.975565 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.975519    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 21 16:31:49.975565 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.975527    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 21 16:31:49.975565 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.975535    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 21 16:31:49.975565 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.975543    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 21 16:31:49.975565 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.975555    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 21 16:31:49.975565 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.975563    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 21 16:31:49.975835 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.975577    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 21 16:31:49.975835 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.975586    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 21 16:31:49.976735 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.976721    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 21 16:31:49.976796 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.976738    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 21 16:31:49.977347 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:49.977323    2562 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-21.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 21 16:31:49.977445 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:49.977363    2562 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 21 16:31:49.980014 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.979998    2562 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 21 16:31:49.980117 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.980042    2562 server.go:1295] "Started kubelet"
May 21 16:31:49.980163 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.980119    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 21 16:31:49.980235 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.980174    2562 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 21 16:31:49.980282 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.980252    2562 server_v1.go:47] "podresources" method="list" useActivePods=true
May 21 16:31:49.980775 ip-10-0-136-21 systemd[1]: Started Kubernetes Kubelet.
May 21 16:31:49.981419 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.981356    2562 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 21 16:31:49.982190 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.982175    2562 server.go:317] "Adding debug handlers to kubelet server"
May 21 16:31:49.987658 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.987637    2562 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-21.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 21 16:31:49.987918 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.987894    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 21 16:31:49.988442 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.988428    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 21 16:31:49.988581 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.988560    2562 volume_manager.go:295] "The desired_state_of_world populator starts"
May 21 16:31:49.988581 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.988573    2562 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 21 16:31:49.988718 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.988693    2562 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 21 16:31:49.988718 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:49.987681    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-21.ec2.internal.18b1a1b124661ad5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-21.ec2.internal,UID:ip-10-0-136-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-21.ec2.internal,},FirstTimestamp:2026-05-21 16:31:49.980011221 +0000 UTC m=+0.451132192,LastTimestamp:2026-05-21 16:31:49.980011221 +0000 UTC m=+0.451132192,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-21.ec2.internal,}"
May 21 16:31:49.988833 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.988745    2562 reconstruct.go:97] "Volume reconstruction finished"
May 21 16:31:49.988833 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.988750    2562 reconciler.go:26] "Reconciler: start to sync state"
May 21 16:31:49.989298 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.989280    2562 factory.go:55] Registering systemd factory
May 21 16:31:49.989298 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.989300    2562 factory.go:223] Registration of the systemd container factory successfully
May 21 16:31:49.989564 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:49.989480    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found"
May 21 16:31:49.989730 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.989668    2562 factory.go:153] Registering CRI-O factory
May 21 16:31:49.989730 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.989690    2562 factory.go:223] Registration of the crio container factory successfully
May 21 16:31:49.989877 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.989743    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 21 16:31:49.989877 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.989764    2562 factory.go:103] Registering Raw factory
May 21 16:31:49.989877 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.989778    2562 manager.go:1196] Started watching for new ooms in manager
May 21 16:31:49.990210 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:49.990196    2562 manager.go:319] Starting recovery of all containers
May 21 16:31:49.992305 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:49.992134    2562 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 21 16:31:49.997014 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:49.996975    2562 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 21 16:31:49.997167 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:49.997035    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-21.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 21 16:31:50.000365 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.000344    2562 manager.go:324] Recovery completed
May 21 16:31:50.002309 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.002198    2562 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
May 21 16:31:50.005500 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.005484    2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 21 16:31:50.008385 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.008363    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientMemory"
May 21 16:31:50.008459 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.008399    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasNoDiskPressure"
May 21 16:31:50.008459 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.008410    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientPID"
May 21 16:31:50.008889 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.008875    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 21 16:31:50.008952 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.008889    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 21 16:31:50.008952 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.008905    2562 state_mem.go:36] "Initialized new in-memory state store"
May 21 16:31:50.010283 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.010211    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-21.ec2.internal.18b1a1b126171108  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-21.ec2.internal,UID:ip-10-0-136-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-21.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-21.ec2.internal,},FirstTimestamp:2026-05-21 16:31:50.0083858 +0000 UTC m=+0.479506770,LastTimestamp:2026-05-21 16:31:50.0083858 +0000 UTC m=+0.479506770,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-21.ec2.internal,}"
May 21 16:31:50.010783 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.010768    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lh44c"
May 21 16:31:50.012244 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.012232    2562 policy_none.go:49] "None policy: Start"
May 21 16:31:50.012280 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.012255    2562 memory_manager.go:186] "Starting memorymanager" policy="None"
May 21 16:31:50.012280 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.012267    2562 state_mem.go:35] "Initializing new in-memory state store"
May 21 16:31:50.017932 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.017908    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lh44c"
May 21 16:31:50.018374 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.018280    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-21.ec2.internal.18b1a1b12617593f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-21.ec2.internal,UID:ip-10-0-136-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-21.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-21.ec2.internal,},FirstTimestamp:2026-05-21 16:31:50.008404287 +0000 UTC m=+0.479525257,LastTimestamp:2026-05-21 16:31:50.008404287 +0000 UTC m=+0.479525257,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-21.ec2.internal,}"
May 21 16:31:50.051721 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.051693    2562 manager.go:341] "Starting Device Plugin manager"
May 21 16:31:50.051848 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.051728    2562 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 21 16:31:50.051848 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.051738    2562 server.go:85] "Starting device plugin registration server"
May 21 16:31:50.052036 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.052022    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
May 21 16:31:50.052157 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.052040    2562 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 21 16:31:50.052281 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.052218    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 21 16:31:50.052357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.052312    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 21 16:31:50.052357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.052321    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 21 16:31:50.054701 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.054682    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 21 16:31:50.054798 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.054722    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-21.ec2.internal\" not found"
May 21 16:31:50.102106 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.102071    2562 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 21 16:31:50.104401 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.103333    2562 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 21 16:31:50.104401 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.103354    2562 status_manager.go:230] "Starting to sync pod status with apiserver"
May 21 16:31:50.104401 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.103377    2562 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 21 16:31:50.104401 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.103384    2562 kubelet.go:2451] "Starting kubelet main sync loop"
May 21 16:31:50.104401 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.103422    2562 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 21 16:31:50.107524 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.107501    2562 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 21 16:31:50.153338 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.153262    2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 21 16:31:50.155024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.155005    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientMemory"
May 21 16:31:50.155129 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.155037    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasNoDiskPressure"
May 21 16:31:50.155129 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.155068    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientPID"
May 21 16:31:50.155129 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.155102    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-21.ec2.internal"
May 21 16:31:50.163738 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.163720    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-21.ec2.internal"
May 21 16:31:50.163795 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.163746    2562 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-21.ec2.internal\": node \"ip-10-0-136-21.ec2.internal\" not found"
May 21 16:31:50.181304 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.181278    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found"
May 21 16:31:50.203582 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.203542    2562 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal"]
May 21 16:31:50.203645 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.203629    2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 21 16:31:50.205618 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.205602    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientMemory"
May 21 16:31:50.205702 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.205635    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasNoDiskPressure"
May 21 16:31:50.205702 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.205645    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientPID"
May 21 16:31:50.207923 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.207909    2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 21 16:31:50.208113 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.208100    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.208148 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.208131    2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 21 16:31:50.208711 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.208698    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientMemory"
May 21 16:31:50.208711 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.208706    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientMemory"
May 21 16:31:50.208799 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.208727    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasNoDiskPressure"
May 21 16:31:50.208799 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.208731    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasNoDiskPressure"
May 21 16:31:50.208799 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.208743    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientPID"
May 21 16:31:50.208799 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.208749    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientPID"
May 21 16:31:50.210874 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.210858    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.210920 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.210893    2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 21 16:31:50.211651 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.211636    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientMemory"
May 21 16:31:50.211743 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.211667    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasNoDiskPressure"
May 21 16:31:50.211743 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.211681    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientPID"
May 21 16:31:50.230809 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.230785    2562 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-21.ec2.internal\" not found" node="ip-10-0-136-21.ec2.internal"
May 21 16:31:50.235319 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.235301    2562 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-21.ec2.internal\" not found" node="ip-10-0-136-21.ec2.internal"
May 21 16:31:50.281600 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.281572    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found"
May 21 16:31:50.289572 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.289550    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f30f9f8d58e543e15d2503e352ff11f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal\" (UID: \"1f30f9f8d58e543e15d2503e352ff11f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.289634 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.289579    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f30f9f8d58e543e15d2503e352ff11f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal\" (UID: \"1f30f9f8d58e543e15d2503e352ff11f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.289634 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.289597    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02780303cf6bfe2b07d5904f0becc31e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-21.ec2.internal\" (UID: \"02780303cf6bfe2b07d5904f0becc31e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.381977 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.381947    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found"
May 21 16:31:50.390300 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.390277    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f30f9f8d58e543e15d2503e352ff11f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal\" (UID: \"1f30f9f8d58e543e15d2503e352ff11f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.390350 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.390307    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02780303cf6bfe2b07d5904f0becc31e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-21.ec2.internal\" (UID: \"02780303cf6bfe2b07d5904f0becc31e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.390350 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.390325    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f30f9f8d58e543e15d2503e352ff11f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal\" (UID: \"1f30f9f8d58e543e15d2503e352ff11f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.390410 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.390375    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02780303cf6bfe2b07d5904f0becc31e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-21.ec2.internal\" (UID: \"02780303cf6bfe2b07d5904f0becc31e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.390410 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.390381    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f30f9f8d58e543e15d2503e352ff11f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal\" (UID: \"1f30f9f8d58e543e15d2503e352ff11f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.390474 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.390438    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f30f9f8d58e543e15d2503e352ff11f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal\" (UID: \"1f30f9f8d58e543e15d2503e352ff11f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.482789 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.482714    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found"
May 21 16:31:50.534222 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.534192    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.538134 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.537892    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.583765 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.583729    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found"
May 21 16:31:50.684249 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.684206    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found"
May 21 16:31:50.784780 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.784697    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found"
May 21 16:31:50.833645 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.833618    2562 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 21 16:31:50.888632 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.888607    2562 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.889699 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.889686    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 21 16:31:50.889825 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.889809    2562 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection force closed via ClientConn.Close\") has prevented the request from succeeding"
May 21 16:31:50.889867 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.889815    2562 kubelet.go:3342] "Failed creating a mirror pod" err="Post \"https://aa717aab008844f988a4214fb7059392-bb82a67bb276d00e.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": write tcp 10.0.136.21:48714->18.214.59.191:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.889867 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.889838    2562 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal"
May 21 16:31:50.889942 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.889807    2562 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection force closed via ClientConn.Close\") has prevented the request from succeeding"
May 21 16:31:50.904951 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.904923    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 21 16:31:50.965610 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.965581    2562 apiserver.go:52] "Watching apiserver"
May 21 16:31:50.974033 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.974009    2562 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 21 16:31:50.976137 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.976115    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9","openshift-dns/node-resolver-pmgp7","openshift-image-registry/node-ca-jslnj","openshift-multus/multus-additional-cni-plugins-28xqm","openshift-multus/network-metrics-daemon-hrrn5","openshift-network-diagnostics/network-check-target-mr4zm","openshift-network-operator/iptables-alerter-mkjnr","kube-system/global-pull-secret-syncer-9tf8m","kube-system/konnectivity-agent-ffwpm","openshift-cluster-node-tuning-operator/tuned-q4cqj","openshift-multus/multus-6pj29","openshift-ovn-kubernetes/ovnkube-node-9th77"]
May 21 16:31:50.980688 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.980668    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:50.982792 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.982760    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pmgp7"
May 21 16:31:50.985154 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.985127    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jslnj"
May 21 16:31:50.985261 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.985243    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:50.986827 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.986810    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 21 16:31:50.987548 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.987526    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:50.987637 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.987614    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:31:50.988627 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.988612    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 21 16:31:50.989702 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.989687    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:31:50.989751 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.989737    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:31:50.991823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.991809    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mkjnr"
May 21 16:31:50.993661 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993643    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/efbc6e4e-c86f-40cd-bab6-db67d363b367-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:50.993737 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993669    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:50.993737 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993686    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 21 16:31:50.993737 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993698    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6654p\" (UniqueName: \"kubernetes.io/projected/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-kube-api-access-6654p\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:50.993850 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993764    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:50.993850 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993801    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-socket-dir\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:50.993850 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993825    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-device-dir\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:50.993987 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993859    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54zkf\" (UniqueName: \"kubernetes.io/projected/644ab036-af41-44aa-848b-dc52cc599ca7-kube-api-access-54zkf\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:50.993987 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993898    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/118ef813-488d-4e5f-a9a5-9e6579c03825-hosts-file\") pod \"node-resolver-pmgp7\" (UID: \"118ef813-488d-4e5f-a9a5-9e6579c03825\") " pod="openshift-dns/node-resolver-pmgp7"
May 21 16:31:50.993987 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993909    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 21 16:31:50.993987 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993921    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/24b5a0be-656b-4d0f-8b47-7f43220f88a8-serviceca\") pod \"node-ca-jslnj\" (UID: \"24b5a0be-656b-4d0f-8b47-7f43220f88a8\") " pod="openshift-image-registry/node-ca-jslnj"
May 21 16:31:50.993987 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993911    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v9qk8\""
May 21 16:31:50.993987 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993934    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 21 16:31:50.993987 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993953    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbw9q\" (UniqueName: \"kubernetes.io/projected/118ef813-488d-4e5f-a9a5-9e6579c03825-kube-api-access-qbw9q\") pod \"node-resolver-pmgp7\" (UID: \"118ef813-488d-4e5f-a9a5-9e6579c03825\") " pod="openshift-dns/node-resolver-pmgp7"
May 21 16:31:50.993987 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993973    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 21 16:31:50.993987 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993979    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/efbc6e4e-c86f-40cd-bab6-db67d363b367-os-release\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:50.993987 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.993983    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994006    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/efbc6e4e-c86f-40cd-bab6-db67d363b367-cni-binary-copy\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994028    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/efbc6e4e-c86f-40cd-bab6-db67d363b367-tuning-conf-dir\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994070    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994072    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qvj4\" (UniqueName: \"kubernetes.io/projected/efbc6e4e-c86f-40cd-bab6-db67d363b367-kube-api-access-2qvj4\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994098    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-etc-selinux\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/118ef813-488d-4e5f-a9a5-9e6579c03825-tmp-dir\") pod \"node-resolver-pmgp7\" (UID: \"118ef813-488d-4e5f-a9a5-9e6579c03825\") " pod="openshift-dns/node-resolver-pmgp7"
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994162    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994162    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24b5a0be-656b-4d0f-8b47-7f43220f88a8-host\") pod \"node-ca-jslnj\" (UID: \"24b5a0be-656b-4d0f-8b47-7f43220f88a8\") " pod="openshift-image-registry/node-ca-jslnj"
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994173    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/efbc6e4e-c86f-40cd-bab6-db67d363b367-system-cni-dir\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994255    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/efbc6e4e-c86f-40cd-bab6-db67d363b367-cnibin\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994273    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/efbc6e4e-c86f-40cd-bab6-db67d363b367-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994293    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgmwg\" (UniqueName: \"kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg\") pod \"network-check-target-mr4zm\" (UID: \"dc89b885-b24b-427a-8f58-a6e184eb40e3\") " pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994316    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-registration-dir\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994333    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-sys-fs\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:50.994415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.994364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2nwx\" (UniqueName: \"kubernetes.io/projected/24b5a0be-656b-4d0f-8b47-7f43220f88a8-kube-api-access-v2nwx\") pod \"node-ca-jslnj\" (UID: \"24b5a0be-656b-4d0f-8b47-7f43220f88a8\") " pod="openshift-image-registry/node-ca-jslnj"
May 21 16:31:50.996248 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.996230    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:50.996337 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:50.996296    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:31:50.996337 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.996315    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ffwpm"
May 21 16:31:50.998439 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.998409    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:50.998681 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.998662    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 21 16:31:51.000070 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.999746    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4j2ht\""
May 21 16:31:51.000070 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:50.999747    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2bj5g\""
May 21 16:31:51.000556 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.000536    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 21 16:31:51.000648 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.000602    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6pj29"
May 21 16:31:51.001680 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.001660    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 21 16:31:51.001773 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.001730    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 21 16:31:51.001832 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.001777    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 21 16:31:51.001832 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.001829    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jmzqx\""
May 21 16:31:51.003428 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.003409    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z7482\""
May 21 16:31:51.003609 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.003592    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.017382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.017354    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 21 16:31:51.017781 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.017763    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 21 16:31:51.018257 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.018232    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 21 16:31:51.018351 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.018307    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 21 16:31:51.018402 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.018354    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gbm4r\""
May 21 16:31:51.018402 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.018356    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 21 16:31:51.018494 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.018478    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 21 16:31:51.018905 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.018887    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 21 16:31:51.018905 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.018906    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 21 16:31:51.019079 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.018922    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-92flp\""
May 21 16:31:51.019079 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.018933    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 21 16:31:51.019079 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.018979    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 21 16:31:51.019663 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.019632    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-20 16:26:50 +0000 UTC" deadline="2028-01-21 15:40:22.91375685 +0000 UTC"
May 21 16:31:51.019663 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.019663    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14639h8m31.894097591s"
May 21 16:31:51.021452 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.021436    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rlx6f\""
May 21 16:31:51.022356 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.022338    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-89jxh\""
May 21 16:31:51.022562 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.022545    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 21 16:31:51.024170 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.024155    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 21 16:31:51.026624 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.026602    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 21 16:31:51.041183 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.041112    2562 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 21 16:31:51.059568 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.059539    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2ffb7"
May 21 16:31:51.069035 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.069013    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2ffb7"
May 21 16:31:51.089519 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.089500    2562 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 21 16:31:51.094918 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.094892    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2pz4\" (UniqueName: \"kubernetes.io/projected/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-kube-api-access-z2pz4\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.095027 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.094921    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-var-lib-kubelet\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.095027 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.094945    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/efbc6e4e-c86f-40cd-bab6-db67d363b367-cni-binary-copy\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.095027 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.094973    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:51.095027 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095016    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-sys\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.095203 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095067    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-cnibin\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.095203 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-kubelet\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.095203 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095101    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-systemd-units\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.095203 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095119    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk8z7\" (UniqueName: \"kubernetes.io/projected/8279711d-7202-4a89-8ec3-cb7cdc2024ba-kube-api-access-gk8z7\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.095203 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095142    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24b5a0be-656b-4d0f-8b47-7f43220f88a8-host\") pod \"node-ca-jslnj\" (UID: \"24b5a0be-656b-4d0f-8b47-7f43220f88a8\") " pod="openshift-image-registry/node-ca-jslnj"
May 21 16:31:51.095203 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095167    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/efbc6e4e-c86f-40cd-bab6-db67d363b367-cnibin\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.095421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095205    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24b5a0be-656b-4d0f-8b47-7f43220f88a8-host\") pod \"node-ca-jslnj\" (UID: \"24b5a0be-656b-4d0f-8b47-7f43220f88a8\") " pod="openshift-image-registry/node-ca-jslnj"
May 21 16:31:51.095421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095232    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/983e0a71-f9f9-4edb-9303-3d2f6efd9a7f-iptables-alerter-script\") pod \"iptables-alerter-mkjnr\" (UID: \"983e0a71-f9f9-4edb-9303-3d2f6efd9a7f\") " pod="openshift-network-operator/iptables-alerter-mkjnr"
May 21 16:31:51.095421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095241    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/efbc6e4e-c86f-40cd-bab6-db67d363b367-cnibin\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.095421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095257    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-cni-binary-copy\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.095421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-cni-bin\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.095421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8279711d-7202-4a89-8ec3-cb7cdc2024ba-env-overrides\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.095421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095313    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8279711d-7202-4a89-8ec3-cb7cdc2024ba-ovn-node-metrics-cert\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.095421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095328    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-kubernetes\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.095421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095342    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-var-lib-kubelet\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.095421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095356    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-hostroot\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.095421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8279711d-7202-4a89-8ec3-cb7cdc2024ba-ovnkube-config\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.095421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095409    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6654p\" (UniqueName: \"kubernetes.io/projected/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-kube-api-access-6654p\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:51.095421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095425    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095442    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-54zkf\" (UniqueName: \"kubernetes.io/projected/644ab036-af41-44aa-848b-dc52cc599ca7-kube-api-access-54zkf\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095463    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/118ef813-488d-4e5f-a9a5-9e6579c03825-hosts-file\") pod \"node-resolver-pmgp7\" (UID: \"118ef813-488d-4e5f-a9a5-9e6579c03825\") " pod="openshift-dns/node-resolver-pmgp7"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095485    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-multus-daemon-config\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095510    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-var-lib-openvswitch\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095531    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095538    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095558    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/118ef813-488d-4e5f-a9a5-9e6579c03825-hosts-file\") pod \"node-resolver-pmgp7\" (UID: \"118ef813-488d-4e5f-a9a5-9e6579c03825\") " pod="openshift-dns/node-resolver-pmgp7"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095588    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/efbc6e4e-c86f-40cd-bab6-db67d363b367-tuning-conf-dir\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095594    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/efbc6e4e-c86f-40cd-bab6-db67d363b367-cni-binary-copy\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095611    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-multus-socket-dir-parent\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095633    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/efbc6e4e-c86f-40cd-bab6-db67d363b367-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095661    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-tmp\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-multus-cni-dir\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-run-netns\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095811    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v2nwx\" (UniqueName: \"kubernetes.io/projected/24b5a0be-656b-4d0f-8b47-7f43220f88a8-kube-api-access-v2nwx\") pod \"node-ca-jslnj\" (UID: \"24b5a0be-656b-4d0f-8b47-7f43220f88a8\") " pod="openshift-image-registry/node-ca-jslnj"
May 21 16:31:51.095960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095840    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/efbc6e4e-c86f-40cd-bab6-db67d363b367-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095859    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-run-k8s-cni-cncf-io\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095885    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-var-lib-cni-multus\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095913    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-multus-conf-dir\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095949    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-slash\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.095973    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-lib-modules\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096001    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-host\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096027    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt6mj\" (UniqueName: \"kubernetes.io/projected/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-kube-api-access-vt6mj\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096079    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-run-openvswitch\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096126    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-node-log\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096154    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qbw9q\" (UniqueName: \"kubernetes.io/projected/118ef813-488d-4e5f-a9a5-9e6579c03825-kube-api-access-qbw9q\") pod \"node-resolver-pmgp7\" (UID: \"118ef813-488d-4e5f-a9a5-9e6579c03825\") " pod="openshift-dns/node-resolver-pmgp7"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096173    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/efbc6e4e-c86f-40cd-bab6-db67d363b367-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096181    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-sysctl-conf\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096234    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-var-lib-cni-bin\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096252    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-etc-kubernetes\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096268    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8279711d-7202-4a89-8ec3-cb7cdc2024ba-ovnkube-script-lib\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096294    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/efbc6e4e-c86f-40cd-bab6-db67d363b367-system-cni-dir\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.096604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096361    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/efbc6e4e-c86f-40cd-bab6-db67d363b367-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096378    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/efbc6e4e-c86f-40cd-bab6-db67d363b367-system-cni-dir\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096402    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-run-ovn-kubernetes\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096452    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-sys-fs\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096449    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/efbc6e4e-c86f-40cd-bab6-db67d363b367-tuning-conf-dir\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096477    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6bsd\" (UniqueName: \"kubernetes.io/projected/983e0a71-f9f9-4edb-9303-3d2f6efd9a7f-kube-api-access-r6bsd\") pod \"iptables-alerter-mkjnr\" (UID: \"983e0a71-f9f9-4edb-9303-3d2f6efd9a7f\") " pod="openshift-network-operator/iptables-alerter-mkjnr"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096494    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-sysctl-d\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096510    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-run-ovn\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096536    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096564    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-device-dir\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096570    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-sys-fs\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096611    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-device-dir\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.096619    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.096697    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs podName:bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f nodeName:}" failed. No retries permitted until 2026-05-21 16:31:51.596652095 +0000 UTC m=+2.067773262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs") pod "network-metrics-daemon-hrrn5" (UID: "bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096714    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/24b5a0be-656b-4d0f-8b47-7f43220f88a8-serviceca\") pod \"node-ca-jslnj\" (UID: \"24b5a0be-656b-4d0f-8b47-7f43220f88a8\") " pod="openshift-image-registry/node-ca-jslnj"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096740    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/336f84d9-849e-405a-8c9d-a8ae4c452a9d-kubelet-config\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:51.097227 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096785    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/336f84d9-849e-405a-8c9d-a8ae4c452a9d-dbus\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096808    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d277af9-bea0-4ffb-adbe-e5c7930e51f4-agent-certs\") pod \"konnectivity-agent-ffwpm\" (UID: \"5d277af9-bea0-4ffb-adbe-e5c7930e51f4\") " pod="kube-system/konnectivity-agent-ffwpm"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096832    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-etc-openvswitch\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096899    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/efbc6e4e-c86f-40cd-bab6-db67d363b367-os-release\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096941    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2qvj4\" (UniqueName: \"kubernetes.io/projected/efbc6e4e-c86f-40cd-bab6-db67d363b367-kube-api-access-2qvj4\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.096996    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/efbc6e4e-c86f-40cd-bab6-db67d363b367-os-release\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-sysconfig\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097033    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-run\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097079    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-os-release\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097083    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/24b5a0be-656b-4d0f-8b47-7f43220f88a8-serviceca\") pod \"node-ca-jslnj\" (UID: \"24b5a0be-656b-4d0f-8b47-7f43220f88a8\") " pod="openshift-image-registry/node-ca-jslnj"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097102    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-run-multus-certs\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097118    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-run-systemd\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097190    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-etc-selinux\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097250    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/118ef813-488d-4e5f-a9a5-9e6579c03825-tmp-dir\") pod \"node-resolver-pmgp7\" (UID: \"118ef813-488d-4e5f-a9a5-9e6579c03825\") " pod="openshift-dns/node-resolver-pmgp7"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097273    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-etc-selinux\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097313    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-modprobe-d\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097355    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-log-socket\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.097969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097403    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kgmwg\" (UniqueName: \"kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg\") pod \"network-check-target-mr4zm\" (UID: \"dc89b885-b24b-427a-8f58-a6e184eb40e3\") " pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:31:51.098823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097446    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-registration-dir\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.098823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097535    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/118ef813-488d-4e5f-a9a5-9e6579c03825-tmp-dir\") pod \"node-resolver-pmgp7\" (UID: \"118ef813-488d-4e5f-a9a5-9e6579c03825\") " pod="openshift-dns/node-resolver-pmgp7"
May 21 16:31:51.098823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097560    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d277af9-bea0-4ffb-adbe-e5c7930e51f4-konnectivity-ca\") pod \"konnectivity-agent-ffwpm\" (UID: \"5d277af9-bea0-4ffb-adbe-e5c7930e51f4\") " pod="kube-system/konnectivity-agent-ffwpm"
May 21 16:31:51.098823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097575    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-registration-dir\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.098823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097586    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/983e0a71-f9f9-4edb-9303-3d2f6efd9a7f-host-slash\") pod \"iptables-alerter-mkjnr\" (UID: \"983e0a71-f9f9-4edb-9303-3d2f6efd9a7f\") " pod="openshift-network-operator/iptables-alerter-mkjnr"
May 21 16:31:51.098823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097618    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-tuned\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.098823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097632    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-system-cni-dir\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.098823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-run-netns\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.098823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097682    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-cni-netd\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.098823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097701    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-socket-dir\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.098823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097735    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-systemd\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.098823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.097787    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/644ab036-af41-44aa-848b-dc52cc599ca7-socket-dir\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.105064 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.105018    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 21 16:31:51.105923 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.105662    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 16:31:51.105923 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.105682    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 16:31:51.105923 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.105693    2562 projected.go:194] Error preparing data for projected volume kube-api-access-kgmwg for pod openshift-network-diagnostics/network-check-target-mr4zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:31:51.105923 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.105760    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg podName:dc89b885-b24b-427a-8f58-a6e184eb40e3 nodeName:}" failed. No retries permitted until 2026-05-21 16:31:51.605742539 +0000 UTC m=+2.076863496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kgmwg" (UniqueName: "kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg") pod "network-check-target-mr4zm" (UID: "dc89b885-b24b-427a-8f58-a6e184eb40e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:31:51.109078 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.108581    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2nwx\" (UniqueName: \"kubernetes.io/projected/24b5a0be-656b-4d0f-8b47-7f43220f88a8-kube-api-access-v2nwx\") pod \"node-ca-jslnj\" (UID: \"24b5a0be-656b-4d0f-8b47-7f43220f88a8\") " pod="openshift-image-registry/node-ca-jslnj"
May 21 16:31:51.109078 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.108599    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6654p\" (UniqueName: \"kubernetes.io/projected/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-kube-api-access-6654p\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:51.109078 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.108659    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54zkf\" (UniqueName: \"kubernetes.io/projected/644ab036-af41-44aa-848b-dc52cc599ca7-kube-api-access-54zkf\") pod \"aws-ebs-csi-driver-node-t6ck9\" (UID: \"644ab036-af41-44aa-848b-dc52cc599ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.109078 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.108674    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbw9q\" (UniqueName: \"kubernetes.io/projected/118ef813-488d-4e5f-a9a5-9e6579c03825-kube-api-access-qbw9q\") pod \"node-resolver-pmgp7\" (UID: \"118ef813-488d-4e5f-a9a5-9e6579c03825\") " pod="openshift-dns/node-resolver-pmgp7"
May 21 16:31:51.109078 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.108937    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qvj4\" (UniqueName: \"kubernetes.io/projected/efbc6e4e-c86f-40cd-bab6-db67d363b367-kube-api-access-2qvj4\") pod \"multus-additional-cni-plugins-28xqm\" (UID: \"efbc6e4e-c86f-40cd-bab6-db67d363b367\") " pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.198316 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198287    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-sysctl-conf\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.198316 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198320    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-var-lib-cni-bin\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.198538 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198337    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-etc-kubernetes\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.198538 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198352    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8279711d-7202-4a89-8ec3-cb7cdc2024ba-ovnkube-script-lib\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.198538 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198370    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-run-ovn-kubernetes\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.198538 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198396    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r6bsd\" (UniqueName: \"kubernetes.io/projected/983e0a71-f9f9-4edb-9303-3d2f6efd9a7f-kube-api-access-r6bsd\") pod \"iptables-alerter-mkjnr\" (UID: \"983e0a71-f9f9-4edb-9303-3d2f6efd9a7f\") " pod="openshift-network-operator/iptables-alerter-mkjnr"
May 21 16:31:51.198538 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198411    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-sysctl-d\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.198538 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198426    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-run-ovn\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.198538 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198430    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-var-lib-cni-bin\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.198538 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198456    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/336f84d9-849e-405a-8c9d-a8ae4c452a9d-kubelet-config\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:51.198538 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198436    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-etc-kubernetes\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.198538 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198506    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/336f84d9-849e-405a-8c9d-a8ae4c452a9d-kubelet-config\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:51.198538 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198504    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-sysctl-conf\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198568    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-run-ovn\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198605    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-sysctl-d\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198611    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-run-ovn-kubernetes\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198655    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/336f84d9-849e-405a-8c9d-a8ae4c452a9d-dbus\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198672    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d277af9-bea0-4ffb-adbe-e5c7930e51f4-agent-certs\") pod \"konnectivity-agent-ffwpm\" (UID: \"5d277af9-bea0-4ffb-adbe-e5c7930e51f4\") " pod="kube-system/konnectivity-agent-ffwpm"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198691    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-etc-openvswitch\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198722    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-sysconfig\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198736    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-run\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198761    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-os-release\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198785    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-run-multus-certs\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198791    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/336f84d9-849e-405a-8c9d-a8ae4c452a9d-dbus\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198811    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-run-systemd\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198844    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-modprobe-d\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198841    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-etc-openvswitch\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198866    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-log-socket\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198872    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-os-release\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198900    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d277af9-bea0-4ffb-adbe-e5c7930e51f4-konnectivity-ca\") pod \"konnectivity-agent-ffwpm\" (UID: \"5d277af9-bea0-4ffb-adbe-e5c7930e51f4\") " pod="kube-system/konnectivity-agent-ffwpm"
May 21 16:31:51.199024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198920    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/983e0a71-f9f9-4edb-9303-3d2f6efd9a7f-host-slash\") pod \"iptables-alerter-mkjnr\" (UID: \"983e0a71-f9f9-4edb-9303-3d2f6efd9a7f\") " pod="openshift-network-operator/iptables-alerter-mkjnr"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198942    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-tuned\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198956    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-run-multus-certs\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198964    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-system-cni-dir\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199133    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-run-netns\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198964    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-run\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.198920    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-run-systemd\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199005    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-system-cni-dir\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199192    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-run-netns\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199071    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-log-socket\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199072    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-modprobe-d\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199045    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8279711d-7202-4a89-8ec3-cb7cdc2024ba-ovnkube-script-lib\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199015    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/983e0a71-f9f9-4edb-9303-3d2f6efd9a7f-host-slash\") pod \"iptables-alerter-mkjnr\" (UID: \"983e0a71-f9f9-4edb-9303-3d2f6efd9a7f\") " pod="openshift-network-operator/iptables-alerter-mkjnr"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199235    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-cni-netd\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199161    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-cni-netd\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199269    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-systemd\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199286    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z2pz4\" (UniqueName: \"kubernetes.io/projected/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-kube-api-access-z2pz4\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199289    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-sysconfig\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.200114 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199310    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-var-lib-kubelet\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199337    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199360    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-sys\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199363    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-systemd\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199397    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-cnibin\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199406    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-sys\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199458    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-kubelet\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199483    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-systemd-units\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199495    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d277af9-bea0-4ffb-adbe-e5c7930e51f4-konnectivity-ca\") pod \"konnectivity-agent-ffwpm\" (UID: \"5d277af9-bea0-4ffb-adbe-e5c7930e51f4\") " pod="kube-system/konnectivity-agent-ffwpm"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199507    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gk8z7\" (UniqueName: \"kubernetes.io/projected/8279711d-7202-4a89-8ec3-cb7cdc2024ba-kube-api-access-gk8z7\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199522    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-cnibin\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.199484    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199544    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-systemd-units\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199557    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-kubelet\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199560    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-var-lib-kubelet\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.199583    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret podName:336f84d9-849e-405a-8c9d-a8ae4c452a9d nodeName:}" failed. No retries permitted until 2026-05-21 16:31:51.699565622 +0000 UTC m=+2.170686595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret") pod "global-pull-secret-syncer-9tf8m" (UID: "336f84d9-849e-405a-8c9d-a8ae4c452a9d") : object "kube-system"/"original-pull-secret" not registered
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199604    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/983e0a71-f9f9-4edb-9303-3d2f6efd9a7f-iptables-alerter-script\") pod \"iptables-alerter-mkjnr\" (UID: \"983e0a71-f9f9-4edb-9303-3d2f6efd9a7f\") " pod="openshift-network-operator/iptables-alerter-mkjnr"
May 21 16:31:51.200892 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199646    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-cni-binary-copy\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199671    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-cni-bin\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199696    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8279711d-7202-4a89-8ec3-cb7cdc2024ba-env-overrides\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199724    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8279711d-7202-4a89-8ec3-cb7cdc2024ba-ovn-node-metrics-cert\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199748    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-kubernetes\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199772    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-var-lib-kubelet\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199795    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-hostroot\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199820    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8279711d-7202-4a89-8ec3-cb7cdc2024ba-ovnkube-config\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199853    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-multus-daemon-config\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199882    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-var-lib-openvswitch\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.199938    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200018    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-multus-socket-dir-parent\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200045    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-tmp\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200091    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-multus-cni-dir\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200117    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-run-netns\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200117    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/983e0a71-f9f9-4edb-9303-3d2f6efd9a7f-iptables-alerter-script\") pod \"iptables-alerter-mkjnr\" (UID: \"983e0a71-f9f9-4edb-9303-3d2f6efd9a7f\") " pod="openshift-network-operator/iptables-alerter-mkjnr"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200163    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-run-k8s-cni-cncf-io\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.201754 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200175    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8279711d-7202-4a89-8ec3-cb7cdc2024ba-env-overrides\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200187    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-cni-binary-copy\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200193    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-var-lib-cni-multus\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200166    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-run-netns\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200214    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-kubernetes\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200217    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-multus-conf-dir\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200237    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-cni-bin\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200239    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-run-k8s-cni-cncf-io\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200252    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-multus-conf-dir\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200252    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-slash\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200271    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-var-lib-openvswitch\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200289    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-slash\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200291    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-host-var-lib-cni-multus\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200290    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-lib-modules\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200315    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-var-lib-kubelet\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200327    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-host\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200351    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6mj\" (UniqueName: \"kubernetes.io/projected/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-kube-api-access-vt6mj\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200365    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-lib-modules\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.202604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200377    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-run-openvswitch\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200403    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-node-log\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200408    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-host\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200445    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200353    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-hostroot\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200655    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-run-openvswitch\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200672    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-multus-socket-dir-parent\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200724    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8279711d-7202-4a89-8ec3-cb7cdc2024ba-node-log\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200779    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-multus-cni-dir\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200809    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-multus-daemon-config\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.200921    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8279711d-7202-4a89-8ec3-cb7cdc2024ba-ovnkube-config\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.201723    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-etc-tuned\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.202821    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8279711d-7202-4a89-8ec3-cb7cdc2024ba-ovn-node-metrics-cert\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.202880    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-tmp\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.203685 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.203103    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d277af9-bea0-4ffb-adbe-e5c7930e51f4-agent-certs\") pod \"konnectivity-agent-ffwpm\" (UID: \"5d277af9-bea0-4ffb-adbe-e5c7930e51f4\") " pod="kube-system/konnectivity-agent-ffwpm"
May 21 16:31:51.209364 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.209333    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6bsd\" (UniqueName: \"kubernetes.io/projected/983e0a71-f9f9-4edb-9303-3d2f6efd9a7f-kube-api-access-r6bsd\") pod \"iptables-alerter-mkjnr\" (UID: \"983e0a71-f9f9-4edb-9303-3d2f6efd9a7f\") " pod="openshift-network-operator/iptables-alerter-mkjnr"
May 21 16:31:51.209647 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.209629    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2pz4\" (UniqueName: \"kubernetes.io/projected/c4ed2ec0-f891-490b-b3f1-285d5cc61b29-kube-api-access-z2pz4\") pod \"tuned-q4cqj\" (UID: \"c4ed2ec0-f891-490b-b3f1-285d5cc61b29\") " pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.209861 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.209839    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk8z7\" (UniqueName: \"kubernetes.io/projected/8279711d-7202-4a89-8ec3-cb7cdc2024ba-kube-api-access-gk8z7\") pod \"ovnkube-node-9th77\" (UID: \"8279711d-7202-4a89-8ec3-cb7cdc2024ba\") " pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.209904 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.209837    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt6mj\" (UniqueName: \"kubernetes.io/projected/b8141c2f-6c4c-4246-b69f-2e04049cd7e6-kube-api-access-vt6mj\") pod \"multus-6pj29\" (UID: \"b8141c2f-6c4c-4246-b69f-2e04049cd7e6\") " pod="openshift-multus/multus-6pj29"
May 21 16:31:51.214076 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:51.214023    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f30f9f8d58e543e15d2503e352ff11f.slice/crio-2ddff67352e28876fc1982b3bc1c2293e7bf00177705cedc3dcfb6b95e5a0473 WatchSource:0}: Error finding container 2ddff67352e28876fc1982b3bc1c2293e7bf00177705cedc3dcfb6b95e5a0473: Status 404 returned error can't find the container with id 2ddff67352e28876fc1982b3bc1c2293e7bf00177705cedc3dcfb6b95e5a0473
May 21 16:31:51.214285 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:51.214271    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02780303cf6bfe2b07d5904f0becc31e.slice/crio-063f7b68885157261c7eee06834a999db7024a79c79f6a1b10be0559b760e9de WatchSource:0}: Error finding container 063f7b68885157261c7eee06834a999db7024a79c79f6a1b10be0559b760e9de: Status 404 returned error can't find the container with id 063f7b68885157261c7eee06834a999db7024a79c79f6a1b10be0559b760e9de
May 21 16:31:51.217268 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.217253    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 21 16:31:51.305922 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.305842    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9"
May 21 16:31:51.312303 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:51.312276    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod644ab036_af41_44aa_848b_dc52cc599ca7.slice/crio-3f912acbcc30ed6a8fd7a4b80349fb7b64799691f9d022d52fa849672afbbee5 WatchSource:0}: Error finding container 3f912acbcc30ed6a8fd7a4b80349fb7b64799691f9d022d52fa849672afbbee5: Status 404 returned error can't find the container with id 3f912acbcc30ed6a8fd7a4b80349fb7b64799691f9d022d52fa849672afbbee5
May 21 16:31:51.319547 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.319526    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pmgp7"
May 21 16:31:51.325673 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:51.325643    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod118ef813_488d_4e5f_a9a5_9e6579c03825.slice/crio-c45b3b492ffae21fd59a44c8882e18b847e9ac3c761a8d0ea87d5c8f0297e96e WatchSource:0}: Error finding container c45b3b492ffae21fd59a44c8882e18b847e9ac3c761a8d0ea87d5c8f0297e96e: Status 404 returned error can't find the container with id c45b3b492ffae21fd59a44c8882e18b847e9ac3c761a8d0ea87d5c8f0297e96e
May 21 16:31:51.326821 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.326804    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jslnj"
May 21 16:31:51.332923 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:51.332892    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b5a0be_656b_4d0f_8b47_7f43220f88a8.slice/crio-0ff10dd2085afc58ea90e0c6061d66d8c55ebf07aac5e630715f88e84cfa9d27 WatchSource:0}: Error finding container 0ff10dd2085afc58ea90e0c6061d66d8c55ebf07aac5e630715f88e84cfa9d27: Status 404 returned error can't find the container with id 0ff10dd2085afc58ea90e0c6061d66d8c55ebf07aac5e630715f88e84cfa9d27
May 21 16:31:51.333936 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.333915    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-28xqm"
May 21 16:31:51.341017 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:51.340996    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefbc6e4e_c86f_40cd_bab6_db67d363b367.slice/crio-56710a166148be0bf5739bd579325b41854b9378437fa639e37c082393a51cbf WatchSource:0}: Error finding container 56710a166148be0bf5739bd579325b41854b9378437fa639e37c082393a51cbf: Status 404 returned error can't find the container with id 56710a166148be0bf5739bd579325b41854b9378437fa639e37c082393a51cbf
May 21 16:31:51.359440 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.359410    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mkjnr"
May 21 16:31:51.365204 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.365180    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ffwpm"
May 21 16:31:51.365537 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:51.365511    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983e0a71_f9f9_4edb_9303_3d2f6efd9a7f.slice/crio-989156444d8e01a399a0c07a29971fde9fe2e603c41aae6503bd0aa8e5189739 WatchSource:0}: Error finding container 989156444d8e01a399a0c07a29971fde9fe2e603c41aae6503bd0aa8e5189739: Status 404 returned error can't find the container with id 989156444d8e01a399a0c07a29971fde9fe2e603c41aae6503bd0aa8e5189739
May 21 16:31:51.371510 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:51.371483    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d277af9_bea0_4ffb_adbe_e5c7930e51f4.slice/crio-72e6901889d7868dc83ec09ccc35ba24656a66328f15ffd9099af93fccef83f9 WatchSource:0}: Error finding container 72e6901889d7868dc83ec09ccc35ba24656a66328f15ffd9099af93fccef83f9: Status 404 returned error can't find the container with id 72e6901889d7868dc83ec09ccc35ba24656a66328f15ffd9099af93fccef83f9
May 21 16:31:51.380174 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.380157    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q4cqj"
May 21 16:31:51.386624 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:51.386593    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4ed2ec0_f891_490b_b3f1_285d5cc61b29.slice/crio-d496fae584b94b3779a694d4d6021180780c1c64ad50af6d531825ffa34fc368 WatchSource:0}: Error finding container d496fae584b94b3779a694d4d6021180780c1c64ad50af6d531825ffa34fc368: Status 404 returned error can't find the container with id d496fae584b94b3779a694d4d6021180780c1c64ad50af6d531825ffa34fc368
May 21 16:31:51.387408 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.387394    2562 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 21 16:31:51.401421 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.401398    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6pj29"
May 21 16:31:51.407370 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.407343    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:31:51.407590 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:51.407569    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8141c2f_6c4c_4246_b69f_2e04049cd7e6.slice/crio-a2c230e25bcde2e1ff52566a1d336a1f0a9de6ffeab36793c33b92d357e7a6ba WatchSource:0}: Error finding container a2c230e25bcde2e1ff52566a1d336a1f0a9de6ffeab36793c33b92d357e7a6ba: Status 404 returned error can't find the container with id a2c230e25bcde2e1ff52566a1d336a1f0a9de6ffeab36793c33b92d357e7a6ba
May 21 16:31:51.414130 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:31:51.414104    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8279711d_7202_4a89_8ec3_cb7cdc2024ba.slice/crio-b642074a44821e7d7a18bf3c5f8066021cf66811260588ce3e81b4f3b128ad30 WatchSource:0}: Error finding container b642074a44821e7d7a18bf3c5f8066021cf66811260588ce3e81b4f3b128ad30: Status 404 returned error can't find the container with id b642074a44821e7d7a18bf3c5f8066021cf66811260588ce3e81b4f3b128ad30
May 21 16:31:51.415830 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.415805    2562 kuberuntime_manager.go:1358] "Unhandled Error" err=<
May 21 16:31:51.415830 ip-10-0-136-21 kubenswrapper[2562]:         container &Container{Name:ovn-controller,Image:quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:820c703d821e1195aeca93e06a8dc65eb8ceb4147c1fbf92c4c56269bc726cb0,Command:[/bin/bash -c set -e
May 21 16:31:51.415830 ip-10-0-136-21 kubenswrapper[2562]:         . /ovnkube-lib/ovnkube-lib.sh || exit 1
May 21 16:31:51.415830 ip-10-0-136-21 kubenswrapper[2562]:         start-ovn-controller ${OVN_LOG_LEVEL}
May 21 16:31:51.415830 ip-10-0-136-21 kubenswrapper[2562]:         ],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:OVN_LOG_LEVEL,Value:info,ValueFrom:nil,},EnvVar{Name:K8S_NODE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:spec.nodeName,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{10 -3} {<nil>} 10m DecimalSI},memory: {{314572800 0} {<nil>} 300Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:ovnkube-script-lib,ReadOnly:false,MountPath:/ovnkube-lib,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:run-openvswitch,ReadOnly:false,MountPath:/run/openvswitch,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:run-ovn,ReadOnly:false,MountPath:/run/ovn/,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:etc-openvswitch,ReadOnly:false,MountPath:/etc/openvswitch,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:etc-openvswitch,ReadOnly:false,MountPath:/etc/ovn/,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:var-lib-openvswitch,ReadOnly:false,MountPath:/var/lib/openvswitch,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:env-overrides,ReadOnly:false,MountPath:/env,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:node-log,ReadOnly:false,MountPath:/var/log/ovn/,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:log-socket,ReadOnly:false,MountPath:/dev/log,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-gk8z7,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:*true,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:FallbackToLogsOnError,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod ovnkube-node-9th77_openshift-ovn-kubernetes(8279711d-7202-4a89-8ec3-cb7cdc2024ba): ErrImagePull: pull QPS exceeded
May 21 16:31:51.415830 ip-10-0-136-21 kubenswrapper[2562]:  > logger="UnhandledError"
May 21 16:31:51.490554 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.490456    2562 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 21 16:31:51.603759 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.603184    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:51.603759 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.603315    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:31:51.603759 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.603375    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs podName:bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f nodeName:}" failed. No retries permitted until 2026-05-21 16:31:52.603357325 +0000 UTC m=+3.074478302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs") pod "network-metrics-daemon-hrrn5" (UID: "bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:31:51.704409 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.704344    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kgmwg\" (UniqueName: \"kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg\") pod \"network-check-target-mr4zm\" (UID: \"dc89b885-b24b-427a-8f58-a6e184eb40e3\") " pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:31:51.704409 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.704411    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:51.704647 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.704560    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 21 16:31:51.704647 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.704625    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret podName:336f84d9-849e-405a-8c9d-a8ae4c452a9d nodeName:}" failed. No retries permitted until 2026-05-21 16:31:52.704605817 +0000 UTC m=+3.175726790 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret") pod "global-pull-secret-syncer-9tf8m" (UID: "336f84d9-849e-405a-8c9d-a8ae4c452a9d") : object "kube-system"/"original-pull-secret" not registered
May 21 16:31:51.704994 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.704975    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 16:31:51.705090 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.704999    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 16:31:51.705090 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.705012    2562 projected.go:194] Error preparing data for projected volume kube-api-access-kgmwg for pod openshift-network-diagnostics/network-check-target-mr4zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:31:51.705090 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:51.705078    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg podName:dc89b885-b24b-427a-8f58-a6e184eb40e3 nodeName:}" failed. No retries permitted until 2026-05-21 16:31:52.705045357 +0000 UTC m=+3.176166330 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kgmwg" (UniqueName: "kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg") pod "network-check-target-mr4zm" (UID: "dc89b885-b24b-427a-8f58-a6e184eb40e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:31:51.910536 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:51.910448    2562 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 21 16:31:52.083081 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.069833    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-20 16:26:51 +0000 UTC" deadline="2027-10-25 22:28:30.554383091 +0000 UTC"
May 21 16:31:52.083081 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.069874    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12533h56m38.48451268s"
May 21 16:31:52.128509 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.128431    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" event={"ID":"1f30f9f8d58e543e15d2503e352ff11f","Type":"ContainerStarted","Data":"2ddff67352e28876fc1982b3bc1c2293e7bf00177705cedc3dcfb6b95e5a0473"}
May 21 16:31:52.133418 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.133376    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q4cqj" event={"ID":"c4ed2ec0-f891-490b-b3f1-285d5cc61b29","Type":"ContainerStarted","Data":"d496fae584b94b3779a694d4d6021180780c1c64ad50af6d531825ffa34fc368"}
May 21 16:31:52.138066 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.138010    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pmgp7" event={"ID":"118ef813-488d-4e5f-a9a5-9e6579c03825","Type":"ContainerStarted","Data":"c45b3b492ffae21fd59a44c8882e18b847e9ac3c761a8d0ea87d5c8f0297e96e"}
May 21 16:31:52.139898 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.139835    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal" event={"ID":"02780303cf6bfe2b07d5904f0becc31e","Type":"ContainerStarted","Data":"063f7b68885157261c7eee06834a999db7024a79c79f6a1b10be0559b760e9de"}
May 21 16:31:52.145121 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.145092    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9th77" event={"ID":"8279711d-7202-4a89-8ec3-cb7cdc2024ba","Type":"ContainerStarted","Data":"b642074a44821e7d7a18bf3c5f8066021cf66811260588ce3e81b4f3b128ad30"}
May 21 16:31:52.149480 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.149385    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6pj29" event={"ID":"b8141c2f-6c4c-4246-b69f-2e04049cd7e6","Type":"ContainerStarted","Data":"a2c230e25bcde2e1ff52566a1d336a1f0a9de6ffeab36793c33b92d357e7a6ba"}
May 21 16:31:52.156274 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.156205    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ffwpm" event={"ID":"5d277af9-bea0-4ffb-adbe-e5c7930e51f4","Type":"ContainerStarted","Data":"72e6901889d7868dc83ec09ccc35ba24656a66328f15ffd9099af93fccef83f9"}
May 21 16:31:52.159703 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.159641    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mkjnr" event={"ID":"983e0a71-f9f9-4edb-9303-3d2f6efd9a7f","Type":"ContainerStarted","Data":"989156444d8e01a399a0c07a29971fde9fe2e603c41aae6503bd0aa8e5189739"}
May 21 16:31:52.163869 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.163770    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xqm" event={"ID":"efbc6e4e-c86f-40cd-bab6-db67d363b367","Type":"ContainerStarted","Data":"56710a166148be0bf5739bd579325b41854b9378437fa639e37c082393a51cbf"}
May 21 16:31:52.167770 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.167694    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jslnj" event={"ID":"24b5a0be-656b-4d0f-8b47-7f43220f88a8","Type":"ContainerStarted","Data":"0ff10dd2085afc58ea90e0c6061d66d8c55ebf07aac5e630715f88e84cfa9d27"}
May 21 16:31:52.170500 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.170473    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9" event={"ID":"644ab036-af41-44aa-848b-dc52cc599ca7","Type":"ContainerStarted","Data":"3f912acbcc30ed6a8fd7a4b80349fb7b64799691f9d022d52fa849672afbbee5"}
May 21 16:31:52.612603 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.612222    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:52.612603 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:52.612563    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:31:52.612802 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:52.612646    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs podName:bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f nodeName:}" failed. No retries permitted until 2026-05-21 16:31:54.612624294 +0000 UTC m=+5.083745255 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs") pod "network-metrics-daemon-hrrn5" (UID: "bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:31:52.713182 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.713145    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kgmwg\" (UniqueName: \"kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg\") pod \"network-check-target-mr4zm\" (UID: \"dc89b885-b24b-427a-8f58-a6e184eb40e3\") " pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:31:52.713382 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:52.713204    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:52.713382 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:52.713369    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 21 16:31:52.713503 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:52.713405    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 16:31:52.713503 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:52.713428    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 16:31:52.713503 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:52.713444    2562 projected.go:194] Error preparing data for projected volume kube-api-access-kgmwg for pod openshift-network-diagnostics/network-check-target-mr4zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:31:52.713650 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:52.713431    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret podName:336f84d9-849e-405a-8c9d-a8ae4c452a9d nodeName:}" failed. No retries permitted until 2026-05-21 16:31:54.713412486 +0000 UTC m=+5.184533446 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret") pod "global-pull-secret-syncer-9tf8m" (UID: "336f84d9-849e-405a-8c9d-a8ae4c452a9d") : object "kube-system"/"original-pull-secret" not registered
May 21 16:31:52.713650 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:52.713554    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg podName:dc89b885-b24b-427a-8f58-a6e184eb40e3 nodeName:}" failed. No retries permitted until 2026-05-21 16:31:54.713536916 +0000 UTC m=+5.184657883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kgmwg" (UniqueName: "kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg") pod "network-check-target-mr4zm" (UID: "dc89b885-b24b-427a-8f58-a6e184eb40e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:31:53.070774 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:53.070680    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-20 16:26:51 +0000 UTC" deadline="2027-10-29 08:30:08.233906114 +0000 UTC"
May 21 16:31:53.070774 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:53.070727    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12615h58m15.163184176s"
May 21 16:31:53.104089 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:53.104028    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:31:53.104276 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:53.104181    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:53.104276 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:53.104193    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:31:53.104388 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:53.104303    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:31:53.104388 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:53.104355    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:53.104491 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:53.104434    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:31:53.190079 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:53.189791    2562 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 21 16:31:54.630565 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:54.630526    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:54.631025 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:54.630689    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:31:54.631025 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:54.630767    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs podName:bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f nodeName:}" failed. No retries permitted until 2026-05-21 16:31:58.630745841 +0000 UTC m=+9.101866798 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs") pod "network-metrics-daemon-hrrn5" (UID: "bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:31:54.731880 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:54.731830    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kgmwg\" (UniqueName: \"kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg\") pod \"network-check-target-mr4zm\" (UID: \"dc89b885-b24b-427a-8f58-a6e184eb40e3\") " pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:31:54.732086 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:54.731903    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:54.732086 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:54.732044    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 21 16:31:54.732208 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:54.732112    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret podName:336f84d9-849e-405a-8c9d-a8ae4c452a9d nodeName:}" failed. No retries permitted until 2026-05-21 16:31:58.732099368 +0000 UTC m=+9.203220325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret") pod "global-pull-secret-syncer-9tf8m" (UID: "336f84d9-849e-405a-8c9d-a8ae4c452a9d") : object "kube-system"/"original-pull-secret" not registered
May 21 16:31:54.732606 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:54.732485    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 16:31:54.732606 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:54.732510    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 16:31:54.732606 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:54.732522    2562 projected.go:194] Error preparing data for projected volume kube-api-access-kgmwg for pod openshift-network-diagnostics/network-check-target-mr4zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:31:54.732606 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:54.732566    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg podName:dc89b885-b24b-427a-8f58-a6e184eb40e3 nodeName:}" failed. No retries permitted until 2026-05-21 16:31:58.732551874 +0000 UTC m=+9.203672836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kgmwg" (UniqueName: "kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg") pod "network-check-target-mr4zm" (UID: "dc89b885-b24b-427a-8f58-a6e184eb40e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:31:55.103929 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:55.103807    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:31:55.103929 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:55.103837    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:55.103929 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:55.103807    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:55.104268 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:55.103937    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:31:55.104268 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:55.104006    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:31:55.104268 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:55.104135    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:31:57.104639 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:57.104599    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:57.105118 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:57.104788    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:31:57.105118 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:57.104950    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:57.105118 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:57.105086    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:31:57.105263 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:57.105142    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:31:57.105263 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:57.105228    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:31:58.661372 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:58.660719    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:58.661372 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:58.660940    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:31:58.661372 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:58.661005    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs podName:bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f nodeName:}" failed. No retries permitted until 2026-05-21 16:32:06.660987052 +0000 UTC m=+17.132108010 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs") pod "network-metrics-daemon-hrrn5" (UID: "bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:31:58.761790 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:58.761749    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kgmwg\" (UniqueName: \"kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg\") pod \"network-check-target-mr4zm\" (UID: \"dc89b885-b24b-427a-8f58-a6e184eb40e3\") " pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:31:58.761985 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:58.761809    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:58.761985 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:58.761934    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 21 16:31:58.761985 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:58.761935    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 16:31:58.761985 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:58.761961    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 16:31:58.761985 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:58.761970    2562 projected.go:194] Error preparing data for projected volume kube-api-access-kgmwg for pod openshift-network-diagnostics/network-check-target-mr4zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:31:58.762239 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:58.762002    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret podName:336f84d9-849e-405a-8c9d-a8ae4c452a9d nodeName:}" failed. No retries permitted until 2026-05-21 16:32:06.761984307 +0000 UTC m=+17.233105284 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret") pod "global-pull-secret-syncer-9tf8m" (UID: "336f84d9-849e-405a-8c9d-a8ae4c452a9d") : object "kube-system"/"original-pull-secret" not registered
May 21 16:31:58.762239 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:58.762023    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg podName:dc89b885-b24b-427a-8f58-a6e184eb40e3 nodeName:}" failed. No retries permitted until 2026-05-21 16:32:06.762013664 +0000 UTC m=+17.233134632 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kgmwg" (UniqueName: "kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg") pod "network-check-target-mr4zm" (UID: "dc89b885-b24b-427a-8f58-a6e184eb40e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:31:59.104416 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:59.104326    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:31:59.104590 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:59.104326    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:31:59.104590 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:59.104461    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:31:59.104590 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:59.104546    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:31:59.104590 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:31:59.104326    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:31:59.104775 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:31:59.104655    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:01.104553 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:01.104511    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:01.104553 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:01.104511    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:01.104996 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:01.104642    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:01.104996 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:01.104750    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:01.104996 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:01.104829    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:01.104996 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:01.104931    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:03.103949 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:03.103909    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:03.104442 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:03.103961    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:03.104442 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:03.104076    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:03.104442 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:03.104123    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:03.104442 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:03.104175    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:03.104442 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:03.104253    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:04.195383 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:04.195340    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9th77" event={"ID":"8279711d-7202-4a89-8ec3-cb7cdc2024ba","Type":"ContainerStarted","Data":"9ee54f24f0487c268ffa7f9a30b1d349467ca7792dd9b89a5101ef9214782542"}
May 21 16:32:04.196735 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:04.196712    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6pj29" event={"ID":"b8141c2f-6c4c-4246-b69f-2e04049cd7e6","Type":"ContainerStarted","Data":"55c8f20e877b88088b484f1786a838c3d5f61be7028b9da08b277f37320f1988"}
May 21 16:32:04.200693 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:04.200666    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q4cqj" event={"ID":"c4ed2ec0-f891-490b-b3f1-285d5cc61b29","Type":"ContainerStarted","Data":"9e13f914467094dbe7da5b0369af8670c5939378c20bd065c1b427b1c7126c45"}
May 21 16:32:04.203372 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:04.203351    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal" event={"ID":"02780303cf6bfe2b07d5904f0becc31e","Type":"ContainerStarted","Data":"0150b0748f99c430329e17bdc25c35928ac46c308105f90a276f42fe2dc56da9"}
May 21 16:32:04.217664 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:04.217617    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q4cqj" podStartSLOduration=1.773262623 podStartE2EDuration="14.21760407s" podCreationTimestamp="2026-05-21 16:31:50 +0000 UTC" firstStartedPulling="2026-05-21 16:31:51.388406781 +0000 UTC m=+1.859527741" lastFinishedPulling="2026-05-21 16:32:03.83274823 +0000 UTC m=+14.303869188" observedRunningTime="2026-05-21 16:32:04.217348827 +0000 UTC m=+14.688469804" watchObservedRunningTime="2026-05-21 16:32:04.21760407 +0000 UTC m=+14.688725082"
May 21 16:32:04.233887 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:04.233835    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal" podStartSLOduration=14.233816544 podStartE2EDuration="14.233816544s" podCreationTimestamp="2026-05-21 16:31:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 16:32:04.233690322 +0000 UTC m=+14.704811301" watchObservedRunningTime="2026-05-21 16:32:04.233816544 +0000 UTC m=+14.704937522"
May 21 16:32:05.104109 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:05.103905    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:05.104298 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:05.103941    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:05.104298 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:05.104207    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:05.104298 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:05.103951    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:05.104476 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:05.104301    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:05.104476 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:05.104373    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:05.213245 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:05.213189    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9th77" event={"ID":"8279711d-7202-4a89-8ec3-cb7cdc2024ba","Type":"ContainerStarted","Data":"5882a7a5ffa0131df260cea494391283d6c2d5567c03145178b1a9ff182bf173"}
May 21 16:32:05.230138 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:05.230076    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-6pj29" podStartSLOduration=2.56995336 podStartE2EDuration="15.230028226s" podCreationTimestamp="2026-05-21 16:31:50 +0000 UTC" firstStartedPulling="2026-05-21 16:31:51.410926226 +0000 UTC m=+1.882047198" lastFinishedPulling="2026-05-21 16:32:04.071001093 +0000 UTC m=+14.542122064" observedRunningTime="2026-05-21 16:32:05.229855942 +0000 UTC m=+15.700976921" watchObservedRunningTime="2026-05-21 16:32:05.230028226 +0000 UTC m=+15.701149206"
May 21 16:32:06.216952 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.216904    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pmgp7" event={"ID":"118ef813-488d-4e5f-a9a5-9e6579c03825","Type":"ContainerStarted","Data":"2e38ee8b8a35fd58eabcf5819d19ec188071903c15625a5e1597b990d62e645f"}
May 21 16:32:06.219159 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.219120    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ffwpm" event={"ID":"5d277af9-bea0-4ffb-adbe-e5c7930e51f4","Type":"ContainerStarted","Data":"b88090e813b8b22343c9fb0230cd1c4b8e9c40b17caa0799eedbaa1bf95b2116"}
May 21 16:32:06.220772 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.220744    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mkjnr" event={"ID":"983e0a71-f9f9-4edb-9303-3d2f6efd9a7f","Type":"ContainerStarted","Data":"f01d1867ebf41faa95c7c99c026e6b511e52cee7861cfc272130e036ba929b95"}
May 21 16:32:06.222659 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.222633    2562 generic.go:358] "Generic (PLEG): container finished" podID="efbc6e4e-c86f-40cd-bab6-db67d363b367" containerID="b644bfe169700974fd62c7290c10a573f595722d901d3728f38b5db747029a76" exitCode=0
May 21 16:32:06.222785 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.222708    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xqm" event={"ID":"efbc6e4e-c86f-40cd-bab6-db67d363b367","Type":"ContainerDied","Data":"b644bfe169700974fd62c7290c10a573f595722d901d3728f38b5db747029a76"}
May 21 16:32:06.224346 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.224324    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jslnj" event={"ID":"24b5a0be-656b-4d0f-8b47-7f43220f88a8","Type":"ContainerStarted","Data":"cd85d425d039a1a34969f937a4e3cc1f5438b86301d76cbdf0ba72558b17e112"}
May 21 16:32:06.226270 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.226235    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9" event={"ID":"644ab036-af41-44aa-848b-dc52cc599ca7","Type":"ContainerStarted","Data":"e1ca30580724b743f56ea617806c46798f12517490f0f46af7a78491a8180485"}
May 21 16:32:06.228139 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.227616    2562 generic.go:358] "Generic (PLEG): container finished" podID="1f30f9f8d58e543e15d2503e352ff11f" containerID="6155bab4923e02a6b1677ffd1af7ec4cdbf20149bd641a90c954f6fb5cd800cf" exitCode=0
May 21 16:32:06.228139 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.227652    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" event={"ID":"1f30f9f8d58e543e15d2503e352ff11f","Type":"ContainerDied","Data":"6155bab4923e02a6b1677ffd1af7ec4cdbf20149bd641a90c954f6fb5cd800cf"}
May 21 16:32:06.228139 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.227868    2562 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal"
May 21 16:32:06.238776 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.238755    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 21 16:32:06.239109 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.239035    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-pmgp7" podStartSLOduration=3.747664029 podStartE2EDuration="16.239021637s" podCreationTimestamp="2026-05-21 16:31:50 +0000 UTC" firstStartedPulling="2026-05-21 16:31:51.327547047 +0000 UTC m=+1.798668018" lastFinishedPulling="2026-05-21 16:32:03.81890465 +0000 UTC m=+14.290025626" observedRunningTime="2026-05-21 16:32:06.233782099 +0000 UTC m=+16.704903079" watchObservedRunningTime="2026-05-21 16:32:06.239021637 +0000 UTC m=+16.710142617"
May 21 16:32:06.239336 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.239322    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal"]
May 21 16:32:06.250925 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.250879    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mkjnr" podStartSLOduration=3.801123874 podStartE2EDuration="16.250868709s" podCreationTimestamp="2026-05-21 16:31:50 +0000 UTC" firstStartedPulling="2026-05-21 16:31:51.367891128 +0000 UTC m=+1.839012085" lastFinishedPulling="2026-05-21 16:32:03.817635959 +0000 UTC m=+14.288756920" observedRunningTime="2026-05-21 16:32:06.250429816 +0000 UTC m=+16.721550798" watchObservedRunningTime="2026-05-21 16:32:06.250868709 +0000 UTC m=+16.721989688"
May 21 16:32:06.279257 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.279029    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jslnj" podStartSLOduration=7.35436808 podStartE2EDuration="16.279011547s" podCreationTimestamp="2026-05-21 16:31:50 +0000 UTC" firstStartedPulling="2026-05-21 16:31:51.334974927 +0000 UTC m=+1.806095888" lastFinishedPulling="2026-05-21 16:32:00.259618383 +0000 UTC m=+10.730739355" observedRunningTime="2026-05-21 16:32:06.278668883 +0000 UTC m=+16.749789863" watchObservedRunningTime="2026-05-21 16:32:06.279011547 +0000 UTC m=+16.750132580"
May 21 16:32:06.279456 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.279422    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ffwpm" podStartSLOduration=3.835290794 podStartE2EDuration="16.279413843s" podCreationTimestamp="2026-05-21 16:31:50 +0000 UTC" firstStartedPulling="2026-05-21 16:31:51.373105923 +0000 UTC m=+1.844226882" lastFinishedPulling="2026-05-21 16:32:03.817228958 +0000 UTC m=+14.288349931" observedRunningTime="2026-05-21 16:32:06.265001241 +0000 UTC m=+16.736122222" watchObservedRunningTime="2026-05-21 16:32:06.279413843 +0000 UTC m=+16.750534823"
May 21 16:32:06.720037 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.720000    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:06.720265 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:06.720199    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:32:06.720331 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:06.720278    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs podName:bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f nodeName:}" failed. No retries permitted until 2026-05-21 16:32:22.720257257 +0000 UTC m=+33.191378214 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs") pod "network-metrics-daemon-hrrn5" (UID: "bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:32:06.821206 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.821159    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kgmwg\" (UniqueName: \"kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg\") pod \"network-check-target-mr4zm\" (UID: \"dc89b885-b24b-427a-8f58-a6e184eb40e3\") " pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:06.821404 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:06.821232    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:06.821469 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:06.821423    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 21 16:32:06.821524 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:06.821489    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret podName:336f84d9-849e-405a-8c9d-a8ae4c452a9d nodeName:}" failed. No retries permitted until 2026-05-21 16:32:22.821471067 +0000 UTC m=+33.292592040 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret") pod "global-pull-secret-syncer-9tf8m" (UID: "336f84d9-849e-405a-8c9d-a8ae4c452a9d") : object "kube-system"/"original-pull-secret" not registered
May 21 16:32:06.821953 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:06.821921    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 16:32:06.821953 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:06.821945    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 16:32:06.821953 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:06.821959    2562 projected.go:194] Error preparing data for projected volume kube-api-access-kgmwg for pod openshift-network-diagnostics/network-check-target-mr4zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:32:06.822202 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:06.822003    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg podName:dc89b885-b24b-427a-8f58-a6e184eb40e3 nodeName:}" failed. No retries permitted until 2026-05-21 16:32:22.821989797 +0000 UTC m=+33.293110769 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kgmwg" (UniqueName: "kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg") pod "network-check-target-mr4zm" (UID: "dc89b885-b24b-427a-8f58-a6e184eb40e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:32:07.104676 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:07.104433    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:07.104824 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:07.104795    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:07.104824 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:07.104537    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:07.104953 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:07.104882    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:07.104953 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:07.104491    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:07.105069 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:07.104951    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:07.218985 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:07.218953    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 21 16:32:07.231676 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:07.231632    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9" event={"ID":"644ab036-af41-44aa-848b-dc52cc599ca7","Type":"ContainerStarted","Data":"2e1fa00da1e99c86ff48fd20c405071b024dcd2ef6bec1435312ee3a7143fb5d"}
May 21 16:32:07.236916 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:07.236605    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" event={"ID":"1f30f9f8d58e543e15d2503e352ff11f","Type":"ContainerStarted","Data":"8872a7ba8041ae6a911b93e82ad62379ee2d04e8f81fa107ed5e834857de7deb"}
May 21 16:32:07.250936 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:07.250857    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" podStartSLOduration=1.250835715 podStartE2EDuration="1.250835715s" podCreationTimestamp="2026-05-21 16:32:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 16:32:07.250446975 +0000 UTC m=+17.721567955" watchObservedRunningTime="2026-05-21 16:32:07.250835715 +0000 UTC m=+17.721956698"
May 21 16:32:08.063069 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:08.062938    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-21T16:32:07.218971639Z","UUID":"6f4fafae-b809-439b-b20e-f2779e90f6e3","Handler":null,"Name":"","Endpoint":""}
May 21 16:32:08.065570 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:08.065543    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 21 16:32:08.065715 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:08.065579    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 21 16:32:08.243036 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:08.243004    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9" event={"ID":"644ab036-af41-44aa-848b-dc52cc599ca7","Type":"ContainerStarted","Data":"c4a24dd0f6f221b65ae9b6173f6b5bf739f746e7881398be6935496a5d6e09a9"}
May 21 16:32:09.103875 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:09.103841    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:09.104179 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:09.103956    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:09.104248 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:09.104191    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:09.104248 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:09.104215    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:09.104359 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:09.104296    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:09.104442 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:09.104424    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:09.264581 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:09.264518    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6ck9" podStartSLOduration=2.43655442 podStartE2EDuration="19.264497102s" podCreationTimestamp="2026-05-21 16:31:50 +0000 UTC" firstStartedPulling="2026-05-21 16:31:51.314123166 +0000 UTC m=+1.785244125" lastFinishedPulling="2026-05-21 16:32:08.142065847 +0000 UTC m=+18.613186807" observedRunningTime="2026-05-21 16:32:09.263431751 +0000 UTC m=+19.734552732" watchObservedRunningTime="2026-05-21 16:32:09.264497102 +0000 UTC m=+19.735618082"
May 21 16:32:09.429259 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:09.428704    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ffwpm"
May 21 16:32:09.429915 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:09.429888    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ffwpm"
May 21 16:32:09.600466 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:09.600411    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ffwpm"
May 21 16:32:09.600970 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:09.600946    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ffwpm"
May 21 16:32:11.104579 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:11.104542    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:11.105106 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:11.104541    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:11.105106 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:11.104672    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:11.105106 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:11.104768    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:11.105106 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:11.104542    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:11.105106 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:11.104895    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:13.104061 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:13.104022    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:13.104582 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:13.104066    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:13.104582 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:13.104167    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:13.104582 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:13.104197    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:13.104582 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:13.104292    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:13.104582 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:13.104374    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:15.103927 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:15.103833    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:15.103927 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:15.103859    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:15.104429 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:15.103833    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:15.104429 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:15.103981    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:15.104429 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:15.104139    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:15.104429 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:15.104236    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:16.262912 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:16.262873    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xqm" event={"ID":"efbc6e4e-c86f-40cd-bab6-db67d363b367","Type":"ContainerStarted","Data":"86f145f4e5759d0945e02d7caa43bde061b6cf9ad617897ec07ec5ea5f63e8fc"}
May 21 16:32:17.104069 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:17.103869    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:17.104249 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:17.103882    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:17.104249 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:17.104147    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:17.104249 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:17.103882    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:17.104371 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:17.104296    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:17.104371 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:17.104223    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:17.266138 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:17.266103    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9th77" event={"ID":"8279711d-7202-4a89-8ec3-cb7cdc2024ba","Type":"ContainerStarted","Data":"430e8e0933e981262183eea1fd5578238080ceaebca45c557cd0a1de3924133a"}
May 21 16:32:17.266138 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:17.266143    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9th77" event={"ID":"8279711d-7202-4a89-8ec3-cb7cdc2024ba","Type":"ContainerStarted","Data":"8fe532d09004482c8ced2ebdcbae57284686de5b8aef5b3bb8dcc99811c2c181"}
May 21 16:32:17.267589 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:17.267561    2562 generic.go:358] "Generic (PLEG): container finished" podID="efbc6e4e-c86f-40cd-bab6-db67d363b367" containerID="86f145f4e5759d0945e02d7caa43bde061b6cf9ad617897ec07ec5ea5f63e8fc" exitCode=0
May 21 16:32:17.267713 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:17.267599    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xqm" event={"ID":"efbc6e4e-c86f-40cd-bab6-db67d363b367","Type":"ContainerDied","Data":"86f145f4e5759d0945e02d7caa43bde061b6cf9ad617897ec07ec5ea5f63e8fc"}
May 21 16:32:18.271493 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:18.271458    2562 generic.go:358] "Generic (PLEG): container finished" podID="efbc6e4e-c86f-40cd-bab6-db67d363b367" containerID="52d869427231fcba75841e398c3eb68635409306f2e45f9453805e98870898a8" exitCode=0
May 21 16:32:18.271854 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:18.271507    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xqm" event={"ID":"efbc6e4e-c86f-40cd-bab6-db67d363b367","Type":"ContainerDied","Data":"52d869427231fcba75841e398c3eb68635409306f2e45f9453805e98870898a8"}
May 21 16:32:19.103910 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:19.103809    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:19.104069 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:19.103814    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:19.104069 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:19.103928    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:19.104069 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:19.104014    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:19.104069 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:19.103814    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:19.104211 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:19.104110    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:19.275691 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:19.275654    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9th77" event={"ID":"8279711d-7202-4a89-8ec3-cb7cdc2024ba","Type":"ContainerStarted","Data":"83ecf5b155daa852d01c034aa680894363b5d67b7a9cd6e516b27e1040a7f005"}
May 21 16:32:19.277766 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:19.277734    2562 generic.go:358] "Generic (PLEG): container finished" podID="efbc6e4e-c86f-40cd-bab6-db67d363b367" containerID="1743d1fc75332ff6ab8924f4f95b85cc54f4d179ab5055b87a271c209ee5834a" exitCode=0
May 21 16:32:19.277891 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:19.277781    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xqm" event={"ID":"efbc6e4e-c86f-40cd-bab6-db67d363b367","Type":"ContainerDied","Data":"1743d1fc75332ff6ab8924f4f95b85cc54f4d179ab5055b87a271c209ee5834a"}
May 21 16:32:20.612627 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:20.612590    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:820c703d821e1195aeca93e06a8dc65eb8ceb4147c1fbf92c4c56269bc726cb0\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-9th77" podUID="8279711d-7202-4a89-8ec3-cb7cdc2024ba"
May 21 16:32:21.104354 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:21.104316    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:21.104547 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:21.104316    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:21.104547 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:21.104456    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:21.104547 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:21.104507    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:21.104547 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:21.104316    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:21.104739 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:21.104625    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:21.284416 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:21.284384    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9th77" event={"ID":"8279711d-7202-4a89-8ec3-cb7cdc2024ba","Type":"ContainerStarted","Data":"e550c867917197acb0d0e56abc945367e5d66d54365b30bce013bca944375664"}
May 21 16:32:21.284756 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:21.284735    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:32:21.284850 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:21.284771    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:32:21.305763 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:21.305558    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:32:22.290596 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.290551    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9th77" event={"ID":"8279711d-7202-4a89-8ec3-cb7cdc2024ba","Type":"ContainerStarted","Data":"5ce91fb2d10595f5334208efb14525506ddc81bb6616b0c36987b0a937af6543"}
May 21 16:32:22.290596 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.290601    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9th77" event={"ID":"8279711d-7202-4a89-8ec3-cb7cdc2024ba","Type":"ContainerStarted","Data":"d835066588d8466f8cf5ee85613c85a47bffed122cf43535ad2f944790a71c53"}
May 21 16:32:22.291207 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.291022    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:32:22.309579 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.309544    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:32:22.318033 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.317969    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9th77" podStartSLOduration=7.574703714 podStartE2EDuration="32.31794974s" podCreationTimestamp="2026-05-21 16:31:50 +0000 UTC" firstStartedPulling="2026-05-21 16:31:51.415654654 +0000 UTC m=+1.886775614" lastFinishedPulling="2026-05-21 16:32:16.158900678 +0000 UTC m=+26.630021640" observedRunningTime="2026-05-21 16:32:22.315920406 +0000 UTC m=+32.787041385" watchObservedRunningTime="2026-05-21 16:32:22.31794974 +0000 UTC m=+32.789070723"
May 21 16:32:22.744374 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.744341    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:22.744543 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:22.744496    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:32:22.744589 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:22.744579    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs podName:bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f nodeName:}" failed. No retries permitted until 2026-05-21 16:32:54.744557121 +0000 UTC m=+65.215678093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs") pod "network-metrics-daemon-hrrn5" (UID: "bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 16:32:22.804681 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.804651    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hrrn5"]
May 21 16:32:22.804836 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.804792    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:22.805288 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:22.804919    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:22.808198 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.807587    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9tf8m"]
May 21 16:32:22.808198 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.807705    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:22.808198 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:22.807797    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:22.808964 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.808918    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mr4zm"]
May 21 16:32:22.809094 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.809018    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:22.809191 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:22.809125    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:22.845181 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.844873    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:22.845181 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:22.844993    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kgmwg\" (UniqueName: \"kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg\") pod \"network-check-target-mr4zm\" (UID: \"dc89b885-b24b-427a-8f58-a6e184eb40e3\") " pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:22.845181 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:22.845014    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 21 16:32:22.845181 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:22.845110    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret podName:336f84d9-849e-405a-8c9d-a8ae4c452a9d nodeName:}" failed. No retries permitted until 2026-05-21 16:32:54.845094683 +0000 UTC m=+65.316215643 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret") pod "global-pull-secret-syncer-9tf8m" (UID: "336f84d9-849e-405a-8c9d-a8ae4c452a9d") : object "kube-system"/"original-pull-secret" not registered
May 21 16:32:22.845181 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:22.845133    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 16:32:22.845181 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:22.845149    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 16:32:22.845181 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:22.845161    2562 projected.go:194] Error preparing data for projected volume kube-api-access-kgmwg for pod openshift-network-diagnostics/network-check-target-mr4zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:32:22.845707 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:22.845207    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg podName:dc89b885-b24b-427a-8f58-a6e184eb40e3 nodeName:}" failed. No retries permitted until 2026-05-21 16:32:54.845193438 +0000 UTC m=+65.316314399 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kgmwg" (UniqueName: "kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg") pod "network-check-target-mr4zm" (UID: "dc89b885-b24b-427a-8f58-a6e184eb40e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 16:32:24.104321 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:24.104288    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:24.104888 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:24.104436    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:25.103995 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:25.103950    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:25.104189 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:25.104117    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:25.104189 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:25.104174    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:25.104317 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:25.104289    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:26.103662 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:26.103622    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:26.104172 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:26.103736    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:27.104660 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:27.104444    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:27.105214 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:27.104445    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:27.105214 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:27.104758    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tf8m" podUID="336f84d9-849e-405a-8c9d-a8ae4c452a9d"
May 21 16:32:27.105214 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:27.104834    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr4zm" podUID="dc89b885-b24b-427a-8f58-a6e184eb40e3"
May 21 16:32:27.305407 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:27.305374    2562 generic.go:358] "Generic (PLEG): container finished" podID="efbc6e4e-c86f-40cd-bab6-db67d363b367" containerID="42f3297a30ada47f6bb465478b2521c8b9e6c9837f0124e0af33b13bc564d946" exitCode=0
May 21 16:32:27.305567 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:27.305415    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xqm" event={"ID":"efbc6e4e-c86f-40cd-bab6-db67d363b367","Type":"ContainerDied","Data":"42f3297a30ada47f6bb465478b2521c8b9e6c9837f0124e0af33b13bc564d946"}
May 21 16:32:28.104070 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.103962    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:28.104222 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:28.104100    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:32:28.310097 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.310063    2562 generic.go:358] "Generic (PLEG): container finished" podID="efbc6e4e-c86f-40cd-bab6-db67d363b367" containerID="0b441525b6fd12a3f5cab40df8a6b77a831f058d5bf9cdb28cddfb0d1382168b" exitCode=0
May 21 16:32:28.310543 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.310117    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xqm" event={"ID":"efbc6e4e-c86f-40cd-bab6-db67d363b367","Type":"ContainerDied","Data":"0b441525b6fd12a3f5cab40df8a6b77a831f058d5bf9cdb28cddfb0d1382168b"}
May 21 16:32:28.320586 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.320565    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeReady"
May 21 16:32:28.320688 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.320674    2562 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 21 16:32:28.365671 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.365601    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dblrj"]
May 21 16:32:28.384515 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.384490    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7kpdj"]
May 21 16:32:28.384686 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.384667    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dblrj"
May 21 16:32:28.387331 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.387303    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 21 16:32:28.387433 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.387334    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 21 16:32:28.387433 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.387376    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tnm5s\""
May 21 16:32:28.398487 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.398464    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dblrj"]
May 21 16:32:28.398487 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.398489    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7kpdj"]
May 21 16:32:28.398646 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.398584    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:32:28.400875 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.400854    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 21 16:32:28.400981 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.400919    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 21 16:32:28.401139 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.401123    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c67tk\""
May 21 16:32:28.401199 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.401182    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 21 16:32:28.487416 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.487383    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrsml\" (UniqueName: \"kubernetes.io/projected/96db6d8f-de2e-4b26-8601-d4994f37e209-kube-api-access-vrsml\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:32:28.487602 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.487421    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:28.487602 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.487451    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b56561-b24c-4152-ba4d-c89944871373-tmp-dir\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:28.487602 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.487501    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52b56561-b24c-4152-ba4d-c89944871373-config-volume\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:28.487602 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.487526    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw78t\" (UniqueName: \"kubernetes.io/projected/52b56561-b24c-4152-ba4d-c89944871373-kube-api-access-bw78t\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:28.487602 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.487546    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:32:28.588327 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.588294    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:32:28.588528 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.588366    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vrsml\" (UniqueName: \"kubernetes.io/projected/96db6d8f-de2e-4b26-8601-d4994f37e209-kube-api-access-vrsml\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:32:28.588528 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.588389    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:28.588528 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.588410    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b56561-b24c-4152-ba4d-c89944871373-tmp-dir\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:28.588528 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.588429    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52b56561-b24c-4152-ba4d-c89944871373-config-volume\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:28.588528 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:28.588437    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 16:32:28.588528 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.588447    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bw78t\" (UniqueName: \"kubernetes.io/projected/52b56561-b24c-4152-ba4d-c89944871373-kube-api-access-bw78t\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:28.588528 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:28.588508    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert podName:96db6d8f-de2e-4b26-8601-d4994f37e209 nodeName:}" failed. No retries permitted until 2026-05-21 16:32:29.088487217 +0000 UTC m=+39.559608189 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert") pod "ingress-canary-7kpdj" (UID: "96db6d8f-de2e-4b26-8601-d4994f37e209") : secret "canary-serving-cert" not found
May 21 16:32:28.588528 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:28.588521    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 16:32:28.588824 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:28.588576    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls podName:52b56561-b24c-4152-ba4d-c89944871373 nodeName:}" failed. No retries permitted until 2026-05-21 16:32:29.088558932 +0000 UTC m=+39.559679889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls") pod "dns-default-dblrj" (UID: "52b56561-b24c-4152-ba4d-c89944871373") : secret "dns-default-metrics-tls" not found
May 21 16:32:28.588824 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.588813    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b56561-b24c-4152-ba4d-c89944871373-tmp-dir\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:28.589076 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.589033    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52b56561-b24c-4152-ba4d-c89944871373-config-volume\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:28.598627 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.598594    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw78t\" (UniqueName: \"kubernetes.io/projected/52b56561-b24c-4152-ba4d-c89944871373-kube-api-access-bw78t\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:28.598742 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:28.598685    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrsml\" (UniqueName: \"kubernetes.io/projected/96db6d8f-de2e-4b26-8601-d4994f37e209-kube-api-access-vrsml\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:32:29.092528 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:29.092491    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:32:29.092700 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:29.092564    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:29.092700 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:29.092648    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 16:32:29.092700 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:29.092653    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 16:32:29.092796 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:29.092704    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls podName:52b56561-b24c-4152-ba4d-c89944871373 nodeName:}" failed. No retries permitted until 2026-05-21 16:32:30.092689922 +0000 UTC m=+40.563810878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls") pod "dns-default-dblrj" (UID: "52b56561-b24c-4152-ba4d-c89944871373") : secret "dns-default-metrics-tls" not found
May 21 16:32:29.092796 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:29.092724    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert podName:96db6d8f-de2e-4b26-8601-d4994f37e209 nodeName:}" failed. No retries permitted until 2026-05-21 16:32:30.092709762 +0000 UTC m=+40.563830723 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert") pod "ingress-canary-7kpdj" (UID: "96db6d8f-de2e-4b26-8601-d4994f37e209") : secret "canary-serving-cert" not found
May 21 16:32:29.104447 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:29.104420    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:29.104521 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:29.104455    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:29.106975 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:29.106954    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 21 16:32:29.107949 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:29.107929    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w759t\""
May 21 16:32:29.108026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:29.107933    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 21 16:32:29.108026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:29.107934    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 21 16:32:29.315003 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:29.314967    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xqm" event={"ID":"efbc6e4e-c86f-40cd-bab6-db67d363b367","Type":"ContainerStarted","Data":"9acf84cebdf8b3cd2c09fb51a1df2167adc651638d67d6c2e53db247e314554e"}
May 21 16:32:30.099407 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:30.099369    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:30.099589 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:30.099417    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:32:30.099589 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:30.099517    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 16:32:30.099589 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:30.099524    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 16:32:30.099589 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:30.099577    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert podName:96db6d8f-de2e-4b26-8601-d4994f37e209 nodeName:}" failed. No retries permitted until 2026-05-21 16:32:32.099564837 +0000 UTC m=+42.570685794 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert") pod "ingress-canary-7kpdj" (UID: "96db6d8f-de2e-4b26-8601-d4994f37e209") : secret "canary-serving-cert" not found
May 21 16:32:30.099589 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:30.099588    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls podName:52b56561-b24c-4152-ba4d-c89944871373 nodeName:}" failed. No retries permitted until 2026-05-21 16:32:32.099582831 +0000 UTC m=+42.570703787 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls") pod "dns-default-dblrj" (UID: "52b56561-b24c-4152-ba4d-c89944871373") : secret "dns-default-metrics-tls" not found
May 21 16:32:30.104973 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:30.104949    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:30.107921 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:30.107895    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tq9vg\""
May 21 16:32:30.108428 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:30.108410    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 21 16:32:32.113350 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:32.113316    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:32:32.113799 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:32.113392    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:32.113799 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:32.113476    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 16:32:32.113799 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:32.113487    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 16:32:32.113799 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:32.113527    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls podName:52b56561-b24c-4152-ba4d-c89944871373 nodeName:}" failed. No retries permitted until 2026-05-21 16:32:36.113513965 +0000 UTC m=+46.584634922 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls") pod "dns-default-dblrj" (UID: "52b56561-b24c-4152-ba4d-c89944871373") : secret "dns-default-metrics-tls" not found
May 21 16:32:32.113799 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:32.113550    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert podName:96db6d8f-de2e-4b26-8601-d4994f37e209 nodeName:}" failed. No retries permitted until 2026-05-21 16:32:36.113533114 +0000 UTC m=+46.584654090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert") pod "ingress-canary-7kpdj" (UID: "96db6d8f-de2e-4b26-8601-d4994f37e209") : secret "canary-serving-cert" not found
May 21 16:32:36.141453 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:36.141239    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:32:36.141919 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:36.141504    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:36.141919 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:36.141389    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 16:32:36.141919 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:36.141590    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 16:32:36.141919 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:36.141596    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert podName:96db6d8f-de2e-4b26-8601-d4994f37e209 nodeName:}" failed. No retries permitted until 2026-05-21 16:32:44.141573852 +0000 UTC m=+54.612694809 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert") pod "ingress-canary-7kpdj" (UID: "96db6d8f-de2e-4b26-8601-d4994f37e209") : secret "canary-serving-cert" not found
May 21 16:32:36.141919 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:36.141642    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls podName:52b56561-b24c-4152-ba4d-c89944871373 nodeName:}" failed. No retries permitted until 2026-05-21 16:32:44.141626533 +0000 UTC m=+54.612747491 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls") pod "dns-default-dblrj" (UID: "52b56561-b24c-4152-ba4d-c89944871373") : secret "dns-default-metrics-tls" not found
May 21 16:32:44.202562 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:44.202519    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:32:44.202982 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:44.202603    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:32:44.202982 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:44.202674    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 16:32:44.202982 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:44.202685    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 16:32:44.202982 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:44.202746    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls podName:52b56561-b24c-4152-ba4d-c89944871373 nodeName:}" failed. No retries permitted until 2026-05-21 16:33:00.20272963 +0000 UTC m=+70.673850597 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls") pod "dns-default-dblrj" (UID: "52b56561-b24c-4152-ba4d-c89944871373") : secret "dns-default-metrics-tls" not found
May 21 16:32:44.202982 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:44.202760    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert podName:96db6d8f-de2e-4b26-8601-d4994f37e209 nodeName:}" failed. No retries permitted until 2026-05-21 16:33:00.202753773 +0000 UTC m=+70.673874734 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert") pod "ingress-canary-7kpdj" (UID: "96db6d8f-de2e-4b26-8601-d4994f37e209") : secret "canary-serving-cert" not found
May 21 16:32:54.305226 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.305197    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9th77"
May 21 16:32:54.333143 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.333083    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-28xqm" podStartSLOduration=29.368715377 podStartE2EDuration="1m4.333038555s" podCreationTimestamp="2026-05-21 16:31:50 +0000 UTC" firstStartedPulling="2026-05-21 16:31:51.342676115 +0000 UTC m=+1.813797071" lastFinishedPulling="2026-05-21 16:32:26.306999289 +0000 UTC m=+36.778120249" observedRunningTime="2026-05-21 16:32:29.342117982 +0000 UTC m=+39.813238962" watchObservedRunningTime="2026-05-21 16:32:54.333038555 +0000 UTC m=+64.804159534"
May 21 16:32:54.774511 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.774476    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:32:54.777417 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.777397    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 21 16:32:54.785641 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:54.785617    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 21 16:32:54.785705 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:32:54.785695    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs podName:bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f nodeName:}" failed. No retries permitted until 2026-05-21 16:33:58.785678486 +0000 UTC m=+129.256799443 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs") pod "network-metrics-daemon-hrrn5" (UID: "bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f") : secret "metrics-daemon-secret" not found
May 21 16:32:54.875637 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.875589    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kgmwg\" (UniqueName: \"kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg\") pod \"network-check-target-mr4zm\" (UID: \"dc89b885-b24b-427a-8f58-a6e184eb40e3\") " pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:54.875637 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.875645    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:54.878623 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.878595    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 21 16:32:54.878732 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.878620    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 21 16:32:54.888756 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.888729    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 21 16:32:54.889552 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.889535    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/336f84d9-849e-405a-8c9d-a8ae4c452a9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tf8m\" (UID: \"336f84d9-849e-405a-8c9d-a8ae4c452a9d\") " pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:54.898951 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.898927    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgmwg\" (UniqueName: \"kubernetes.io/projected/dc89b885-b24b-427a-8f58-a6e184eb40e3-kube-api-access-kgmwg\") pod \"network-check-target-mr4zm\" (UID: \"dc89b885-b24b-427a-8f58-a6e184eb40e3\") " pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:54.916799 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.916774    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w759t\""
May 21 16:32:54.918874 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.918857    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tf8m"
May 21 16:32:54.924554 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:54.924538    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:32:55.065208 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:55.065135    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9tf8m"]
May 21 16:32:55.069013 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:32:55.068988    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336f84d9_849e_405a_8c9d_a8ae4c452a9d.slice/crio-89bc05dcf0b2606bc92b8f9a6acc55fd4ac5b8d177ba351cdea4927a8335ceb8 WatchSource:0}: Error finding container 89bc05dcf0b2606bc92b8f9a6acc55fd4ac5b8d177ba351cdea4927a8335ceb8: Status 404 returned error can't find the container with id 89bc05dcf0b2606bc92b8f9a6acc55fd4ac5b8d177ba351cdea4927a8335ceb8
May 21 16:32:55.076236 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:55.076210    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mr4zm"]
May 21 16:32:55.079399 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:32:55.079374    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc89b885_b24b_427a_8f58_a6e184eb40e3.slice/crio-ec0851d4156d6b183b69e6005fb252f587173f262fbee8c2d312cd1f2f33914c WatchSource:0}: Error finding container ec0851d4156d6b183b69e6005fb252f587173f262fbee8c2d312cd1f2f33914c: Status 404 returned error can't find the container with id ec0851d4156d6b183b69e6005fb252f587173f262fbee8c2d312cd1f2f33914c
May 21 16:32:55.364225 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:55.364141    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mr4zm" event={"ID":"dc89b885-b24b-427a-8f58-a6e184eb40e3","Type":"ContainerStarted","Data":"ec0851d4156d6b183b69e6005fb252f587173f262fbee8c2d312cd1f2f33914c"}
May 21 16:32:55.365222 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:32:55.365200    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9tf8m" event={"ID":"336f84d9-849e-405a-8c9d-a8ae4c452a9d","Type":"ContainerStarted","Data":"89bc05dcf0b2606bc92b8f9a6acc55fd4ac5b8d177ba351cdea4927a8335ceb8"}
May 21 16:33:00.218546 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:00.218496    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:33:00.219100 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:00.218565    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:33:00.219100 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:33:00.218659    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 16:33:00.219100 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:33:00.218699    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 16:33:00.219100 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:33:00.218729    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls podName:52b56561-b24c-4152-ba4d-c89944871373 nodeName:}" failed. No retries permitted until 2026-05-21 16:33:32.218711045 +0000 UTC m=+102.689832018 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls") pod "dns-default-dblrj" (UID: "52b56561-b24c-4152-ba4d-c89944871373") : secret "dns-default-metrics-tls" not found
May 21 16:33:00.219100 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:33:00.218754    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert podName:96db6d8f-de2e-4b26-8601-d4994f37e209 nodeName:}" failed. No retries permitted until 2026-05-21 16:33:32.218737225 +0000 UTC m=+102.689858194 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert") pod "ingress-canary-7kpdj" (UID: "96db6d8f-de2e-4b26-8601-d4994f37e209") : secret "canary-serving-cert" not found
May 21 16:33:00.376828 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:00.376791    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mr4zm" event={"ID":"dc89b885-b24b-427a-8f58-a6e184eb40e3","Type":"ContainerStarted","Data":"74850a09ed0e0e4e41bf7b5c2f09bfb52378c47c75bdb50ed0b9a93e84f53584"}
May 21 16:33:00.377004 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:00.376959    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:33:00.378376 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:00.378345    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9tf8m" event={"ID":"336f84d9-849e-405a-8c9d-a8ae4c452a9d","Type":"ContainerStarted","Data":"2b21e613f4fb3564907b19c8154719513b9ab4e0e64e18713a7832b50818fd59"}
May 21 16:33:00.392289 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:00.392244    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mr4zm" podStartSLOduration=65.996208133 podStartE2EDuration="1m10.392230539s" podCreationTimestamp="2026-05-21 16:31:50 +0000 UTC" firstStartedPulling="2026-05-21 16:32:55.081359245 +0000 UTC m=+65.552480202" lastFinishedPulling="2026-05-21 16:32:59.477381651 +0000 UTC m=+69.948502608" observedRunningTime="2026-05-21 16:33:00.39147715 +0000 UTC m=+70.862598130" watchObservedRunningTime="2026-05-21 16:33:00.392230539 +0000 UTC m=+70.863351511"
May 21 16:33:00.404002 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:00.403948    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9tf8m" podStartSLOduration=65.994108023 podStartE2EDuration="1m10.40393332s" podCreationTimestamp="2026-05-21 16:31:50 +0000 UTC" firstStartedPulling="2026-05-21 16:32:55.070545719 +0000 UTC m=+65.541666678" lastFinishedPulling="2026-05-21 16:32:59.48037101 +0000 UTC m=+69.951491975" observedRunningTime="2026-05-21 16:33:00.403652986 +0000 UTC m=+70.874773965" watchObservedRunningTime="2026-05-21 16:33:00.40393332 +0000 UTC m=+70.875054292"
May 21 16:33:03.836967 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:03.836927    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"]
May 21 16:33:03.838930 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:03.838908    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"
May 21 16:33:03.841588 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:03.841569    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 21 16:33:03.841689 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:03.841589    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 21 16:33:03.841689 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:03.841589    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
May 21 16:33:03.842581 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:03.842562    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 21 16:33:03.847251 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:03.847217    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"]
May 21 16:33:03.948752 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:03.948704    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1f39dd1-2f46-4a58-95ff-80de5992a3f2-tmp\") pod \"klusterlet-addon-workmgr-97d9d477-2bt78\" (UID: \"a1f39dd1-2f46-4a58-95ff-80de5992a3f2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"
May 21 16:33:03.948922 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:03.948769    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a1f39dd1-2f46-4a58-95ff-80de5992a3f2-klusterlet-config\") pod \"klusterlet-addon-workmgr-97d9d477-2bt78\" (UID: \"a1f39dd1-2f46-4a58-95ff-80de5992a3f2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"
May 21 16:33:03.948922 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:03.948817    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9p7t\" (UniqueName: \"kubernetes.io/projected/a1f39dd1-2f46-4a58-95ff-80de5992a3f2-kube-api-access-t9p7t\") pod \"klusterlet-addon-workmgr-97d9d477-2bt78\" (UID: \"a1f39dd1-2f46-4a58-95ff-80de5992a3f2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"
May 21 16:33:04.049489 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:04.049456    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1f39dd1-2f46-4a58-95ff-80de5992a3f2-tmp\") pod \"klusterlet-addon-workmgr-97d9d477-2bt78\" (UID: \"a1f39dd1-2f46-4a58-95ff-80de5992a3f2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"
May 21 16:33:04.049562 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:04.049512    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a1f39dd1-2f46-4a58-95ff-80de5992a3f2-klusterlet-config\") pod \"klusterlet-addon-workmgr-97d9d477-2bt78\" (UID: \"a1f39dd1-2f46-4a58-95ff-80de5992a3f2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"
May 21 16:33:04.049562 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:04.049551    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t9p7t\" (UniqueName: \"kubernetes.io/projected/a1f39dd1-2f46-4a58-95ff-80de5992a3f2-kube-api-access-t9p7t\") pod \"klusterlet-addon-workmgr-97d9d477-2bt78\" (UID: \"a1f39dd1-2f46-4a58-95ff-80de5992a3f2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"
May 21 16:33:04.049837 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:04.049816    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1f39dd1-2f46-4a58-95ff-80de5992a3f2-tmp\") pod \"klusterlet-addon-workmgr-97d9d477-2bt78\" (UID: \"a1f39dd1-2f46-4a58-95ff-80de5992a3f2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"
May 21 16:33:04.052209 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:04.052190    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a1f39dd1-2f46-4a58-95ff-80de5992a3f2-klusterlet-config\") pod \"klusterlet-addon-workmgr-97d9d477-2bt78\" (UID: \"a1f39dd1-2f46-4a58-95ff-80de5992a3f2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"
May 21 16:33:04.057498 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:04.057468    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9p7t\" (UniqueName: \"kubernetes.io/projected/a1f39dd1-2f46-4a58-95ff-80de5992a3f2-kube-api-access-t9p7t\") pod \"klusterlet-addon-workmgr-97d9d477-2bt78\" (UID: \"a1f39dd1-2f46-4a58-95ff-80de5992a3f2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"
May 21 16:33:04.148314 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:04.148226    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"
May 21 16:33:04.289113 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:04.289079    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"]
May 21 16:33:04.295888 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:33:04.295860    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1f39dd1_2f46_4a58_95ff_80de5992a3f2.slice/crio-9215f55143fc97f88d436b2f7c57bc55c91432cbf9dd6569f696a53c6fc32c80 WatchSource:0}: Error finding container 9215f55143fc97f88d436b2f7c57bc55c91432cbf9dd6569f696a53c6fc32c80: Status 404 returned error can't find the container with id 9215f55143fc97f88d436b2f7c57bc55c91432cbf9dd6569f696a53c6fc32c80
May 21 16:33:04.387914 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:04.387867    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78" event={"ID":"a1f39dd1-2f46-4a58-95ff-80de5992a3f2","Type":"ContainerStarted","Data":"9215f55143fc97f88d436b2f7c57bc55c91432cbf9dd6569f696a53c6fc32c80"}
May 21 16:33:09.399640 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:09.399598    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78" event={"ID":"a1f39dd1-2f46-4a58-95ff-80de5992a3f2","Type":"ContainerStarted","Data":"05899c3bbb314d0304a201d94147260470e1149c07d7f338d8fbc664da88bc3d"}
May 21 16:33:09.400009 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:09.399809    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"
May 21 16:33:09.401510 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:09.401487    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78"
May 21 16:33:09.415472 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:09.415422    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-97d9d477-2bt78" podStartSLOduration=1.722184017 podStartE2EDuration="6.415406195s" podCreationTimestamp="2026-05-21 16:33:03 +0000 UTC" firstStartedPulling="2026-05-21 16:33:04.297758704 +0000 UTC m=+74.768879667" lastFinishedPulling="2026-05-21 16:33:08.990980886 +0000 UTC m=+79.462101845" observedRunningTime="2026-05-21 16:33:09.414873187 +0000 UTC m=+79.885994162" watchObservedRunningTime="2026-05-21 16:33:09.415406195 +0000 UTC m=+79.886527175"
May 21 16:33:31.382926 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:31.382895    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mr4zm"
May 21 16:33:32.248099 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:32.248029    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:33:32.248281 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:32.248135    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:33:32.248281 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:33:32.248203    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 16:33:32.248281 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:33:32.248229    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 16:33:32.248281 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:33:32.248273    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert podName:96db6d8f-de2e-4b26-8601-d4994f37e209 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:36.248257404 +0000 UTC m=+166.719378360 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert") pod "ingress-canary-7kpdj" (UID: "96db6d8f-de2e-4b26-8601-d4994f37e209") : secret "canary-serving-cert" not found
May 21 16:33:32.248439 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:33:32.248295    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls podName:52b56561-b24c-4152-ba4d-c89944871373 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:36.248279945 +0000 UTC m=+166.719400919 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls") pod "dns-default-dblrj" (UID: "52b56561-b24c-4152-ba4d-c89944871373") : secret "dns-default-metrics-tls" not found
May 21 16:33:58.831712 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:33:58.831669    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:33:58.832189 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:33:58.831785    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 21 16:33:58.832189 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:33:58.831853    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs podName:bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f nodeName:}" failed. No retries permitted until 2026-05-21 16:36:00.831837192 +0000 UTC m=+251.302958150 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs") pod "network-metrics-daemon-hrrn5" (UID: "bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f") : secret "metrics-daemon-secret" not found
May 21 16:34:11.726877 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.726834    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"]
May 21 16:34:11.729210 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.729192    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68fc7c8669-4grz2"]
May 21 16:34:11.729364 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.729346    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:11.731089 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.731070    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6d97cc9d7b-6vfk9"]
May 21 16:34:11.731226 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.731207    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.732794 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.732779    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.733828 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.733812    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
May 21 16:34:11.733919 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.733849    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 21 16:34:11.736836 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.736810    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 21 16:34:11.736980 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.736960    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r2549\""
May 21 16:34:11.737079 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.736903    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
May 21 16:34:11.737147 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.737075    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8vm2s\""
May 21 16:34:11.737147 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.736966    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 21 16:34:11.737147 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.736858    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
May 21 16:34:11.737306 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.737152    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 21 16:34:11.737306 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.736965    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
May 21 16:34:11.737306 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.737288    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 21 16:34:11.738391 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.738372    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
May 21 16:34:11.738391 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.738401    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 21 16:34:11.738544 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.738422    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
May 21 16:34:11.738585 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.738568    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 21 16:34:11.738627 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.738616    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-sdx7l\""
May 21 16:34:11.742640 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.742613    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 21 16:34:11.743132 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.743109    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"]
May 21 16:34:11.744246 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.744223    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d97cc9d7b-6vfk9"]
May 21 16:34:11.763380 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.763347    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68fc7c8669-4grz2"]
May 21 16:34:11.820433 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.820387    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b88pc\" (UniqueName: \"kubernetes.io/projected/39cbf8cb-d27b-4e6c-8abc-603723d275fc-kube-api-access-b88pc\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:11.820433 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.820433    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-ca-trust-extracted\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.820696 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.820457    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-certificates\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.820696 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.820549    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/39cbf8cb-d27b-4e6c-8abc-603723d275fc-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:11.820696 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.820606    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:11.820696 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.820666    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.820886 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.820706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-stats-auth\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.820886 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.820761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-installation-pull-secrets\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.820962 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.820868    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-trusted-ca\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.821005 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.820962    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.821005 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.820987    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.821108 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.821005    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-bound-sa-token\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.821108 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.821021    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z5mk\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-kube-api-access-2z5mk\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.821189 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.821107    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fswqj\" (UniqueName: \"kubernetes.io/projected/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-kube-api-access-fswqj\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.821189 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.821155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-default-certificate\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.821189 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.821180    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-image-registry-private-configuration\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.821703 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.821684    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"]
May 21 16:34:11.823617 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.823603    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"
May 21 16:34:11.826123 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.826101    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
May 21 16:34:11.826259 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.826167    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
May 21 16:34:11.826449 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.826432    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-f66j4\""
May 21 16:34:11.826527 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.826438    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
May 21 16:34:11.826607 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.826591    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
May 21 16:34:11.833838 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.833810    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"]
May 21 16:34:11.922013 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.921947    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-bound-sa-token\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.922013 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922017    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-ca-trust-extracted\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.922363 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922036    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-certificates\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.922363 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/271119ee-1b3f-422f-adba-30db5318d9ec-config\") pod \"service-ca-operator-57b5b68b8-vwwlb\" (UID: \"271119ee-1b3f-422f-adba-30db5318d9ec\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"
May 21 16:34:11.922363 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922111    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-installation-pull-secrets\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.922363 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922250    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.922363 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922285    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.922363 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922315    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fswqj\" (UniqueName: \"kubernetes.io/projected/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-kube-api-access-fswqj\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.922363 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922345    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-image-registry-private-configuration\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.922707 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:11.922378    2562 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 16:34:11.922707 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:11.922402    2562 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68fc7c8669-4grz2: secret "image-registry-tls" not found
May 21 16:34:11.922707 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:11.922405    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 21 16:34:11.922707 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922380    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2z5mk\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-kube-api-access-2z5mk\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.922707 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:11.922465    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls podName:d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:12.422445007 +0000 UTC m=+142.893566026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls") pod "image-registry-68fc7c8669-4grz2" (UID: "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5") : secret "image-registry-tls" not found
May 21 16:34:11.922707 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922513    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-ca-trust-extracted\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.922707 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:11.922557    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs podName:7ab80794-ebdf-4d85-bcb2-748a4da45ae6 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:12.422536529 +0000 UTC m=+142.893657501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs") pod "router-default-6d97cc9d7b-6vfk9" (UID: "7ab80794-ebdf-4d85-bcb2-748a4da45ae6") : secret "router-metrics-certs-default" not found
May 21 16:34:11.922707 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922628    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b88pc\" (UniqueName: \"kubernetes.io/projected/39cbf8cb-d27b-4e6c-8abc-603723d275fc-kube-api-access-b88pc\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:11.922707 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922693    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/39cbf8cb-d27b-4e6c-8abc-603723d275fc-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:11.923170 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922722    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:11.923170 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922753    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.923170 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922779    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-stats-auth\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.923170 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922825    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-certificates\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.923170 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922846    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-trusted-ca\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.923170 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:11.922889    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 21 16:34:11.923170 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:11.922937    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls podName:39cbf8cb-d27b-4e6c-8abc-603723d275fc nodeName:}" failed. No retries permitted until 2026-05-21 16:34:12.422923746 +0000 UTC m=+142.894044704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-hswx2" (UID: "39cbf8cb-d27b-4e6c-8abc-603723d275fc") : secret "cluster-monitoring-operator-tls" not found
May 21 16:34:11.923170 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.922891    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/271119ee-1b3f-422f-adba-30db5318d9ec-serving-cert\") pod \"service-ca-operator-57b5b68b8-vwwlb\" (UID: \"271119ee-1b3f-422f-adba-30db5318d9ec\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"
May 21 16:34:11.923170 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:11.922959    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle podName:7ab80794-ebdf-4d85-bcb2-748a4da45ae6 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:12.422943999 +0000 UTC m=+142.894064979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle") pod "router-default-6d97cc9d7b-6vfk9" (UID: "7ab80794-ebdf-4d85-bcb2-748a4da45ae6") : configmap references non-existent config key: service-ca.crt
May 21 16:34:11.923170 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.923014    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-default-certificate\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.923170 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.923068    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh86n\" (UniqueName: \"kubernetes.io/projected/271119ee-1b3f-422f-adba-30db5318d9ec-kube-api-access-fh86n\") pod \"service-ca-operator-57b5b68b8-vwwlb\" (UID: \"271119ee-1b3f-422f-adba-30db5318d9ec\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"
May 21 16:34:11.923888 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.923460    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/39cbf8cb-d27b-4e6c-8abc-603723d275fc-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:11.924181 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.924156    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-trusted-ca\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.925396 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.925371    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-stats-auth\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.925515 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.925437    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-installation-pull-secrets\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.925515 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.925475    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-image-registry-private-configuration\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.925664 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.925568    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-default-certificate\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.934888 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.934860    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-bound-sa-token\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:11.935334 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.935313    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fswqj\" (UniqueName: \"kubernetes.io/projected/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-kube-api-access-fswqj\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:11.935387 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.935316    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b88pc\" (UniqueName: \"kubernetes.io/projected/39cbf8cb-d27b-4e6c-8abc-603723d275fc-kube-api-access-b88pc\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:11.935600 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:11.935574    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z5mk\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-kube-api-access-2z5mk\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:12.023489 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:12.023454    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/271119ee-1b3f-422f-adba-30db5318d9ec-serving-cert\") pod \"service-ca-operator-57b5b68b8-vwwlb\" (UID: \"271119ee-1b3f-422f-adba-30db5318d9ec\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"
May 21 16:34:12.023641 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:12.023497    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fh86n\" (UniqueName: \"kubernetes.io/projected/271119ee-1b3f-422f-adba-30db5318d9ec-kube-api-access-fh86n\") pod \"service-ca-operator-57b5b68b8-vwwlb\" (UID: \"271119ee-1b3f-422f-adba-30db5318d9ec\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"
May 21 16:34:12.023641 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:12.023520    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/271119ee-1b3f-422f-adba-30db5318d9ec-config\") pod \"service-ca-operator-57b5b68b8-vwwlb\" (UID: \"271119ee-1b3f-422f-adba-30db5318d9ec\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"
May 21 16:34:12.024030 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:12.024013    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/271119ee-1b3f-422f-adba-30db5318d9ec-config\") pod \"service-ca-operator-57b5b68b8-vwwlb\" (UID: \"271119ee-1b3f-422f-adba-30db5318d9ec\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"
May 21 16:34:12.026022 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:12.026002    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/271119ee-1b3f-422f-adba-30db5318d9ec-serving-cert\") pod \"service-ca-operator-57b5b68b8-vwwlb\" (UID: \"271119ee-1b3f-422f-adba-30db5318d9ec\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"
May 21 16:34:12.032042 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:12.032015    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh86n\" (UniqueName: \"kubernetes.io/projected/271119ee-1b3f-422f-adba-30db5318d9ec-kube-api-access-fh86n\") pod \"service-ca-operator-57b5b68b8-vwwlb\" (UID: \"271119ee-1b3f-422f-adba-30db5318d9ec\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"
May 21 16:34:12.132267 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:12.132228    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"
May 21 16:34:12.259936 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:12.259902    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb"]
May 21 16:34:12.263633 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:34:12.263607    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod271119ee_1b3f_422f_adba_30db5318d9ec.slice/crio-fa38c54fd406e1d10a942331b8d26c916296eb610b3177a366a9ac227da75b34 WatchSource:0}: Error finding container fa38c54fd406e1d10a942331b8d26c916296eb610b3177a366a9ac227da75b34: Status 404 returned error can't find the container with id fa38c54fd406e1d10a942331b8d26c916296eb610b3177a366a9ac227da75b34
May 21 16:34:12.426680 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:12.426582    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:12.426680 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:12.426623    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:12.426882 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:12.426690    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:12.426882 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:12.426715    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:12.426882 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:12.426765    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 21 16:34:12.426882 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:12.426813    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle podName:7ab80794-ebdf-4d85-bcb2-748a4da45ae6 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:13.426790705 +0000 UTC m=+143.897911675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle") pod "router-default-6d97cc9d7b-6vfk9" (UID: "7ab80794-ebdf-4d85-bcb2-748a4da45ae6") : configmap references non-existent config key: service-ca.crt
May 21 16:34:12.426882 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:12.426836    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 21 16:34:12.426882 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:12.426836    2562 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 16:34:12.426882 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:12.426857    2562 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68fc7c8669-4grz2: secret "image-registry-tls" not found
May 21 16:34:12.426882 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:12.426839    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls podName:39cbf8cb-d27b-4e6c-8abc-603723d275fc nodeName:}" failed. No retries permitted until 2026-05-21 16:34:13.426829298 +0000 UTC m=+143.897950261 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-hswx2" (UID: "39cbf8cb-d27b-4e6c-8abc-603723d275fc") : secret "cluster-monitoring-operator-tls" not found
May 21 16:34:12.427226 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:12.426904    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs podName:7ab80794-ebdf-4d85-bcb2-748a4da45ae6 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:13.426890856 +0000 UTC m=+143.898011813 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs") pod "router-default-6d97cc9d7b-6vfk9" (UID: "7ab80794-ebdf-4d85-bcb2-748a4da45ae6") : secret "router-metrics-certs-default" not found
May 21 16:34:12.427226 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:12.426914    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls podName:d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:13.426908492 +0000 UTC m=+143.898029450 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls") pod "image-registry-68fc7c8669-4grz2" (UID: "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5") : secret "image-registry-tls" not found
May 21 16:34:12.519344 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:12.519306    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb" event={"ID":"271119ee-1b3f-422f-adba-30db5318d9ec","Type":"ContainerStarted","Data":"fa38c54fd406e1d10a942331b8d26c916296eb610b3177a366a9ac227da75b34"}
May 21 16:34:13.433688 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:13.433653    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:13.434156 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:13.433700    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:13.434156 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:13.433751    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:13.434156 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:13.433778    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:13.434156 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:13.433781    2562 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 16:34:13.434156 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:13.433804    2562 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68fc7c8669-4grz2: secret "image-registry-tls" not found
May 21 16:34:13.434156 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:13.433851    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 21 16:34:13.434156 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:13.433878    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls podName:d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:15.433856401 +0000 UTC m=+145.904977363 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls") pod "image-registry-68fc7c8669-4grz2" (UID: "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5") : secret "image-registry-tls" not found
May 21 16:34:13.434156 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:13.433851    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 21 16:34:13.434156 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:13.433905    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle podName:7ab80794-ebdf-4d85-bcb2-748a4da45ae6 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:15.433890654 +0000 UTC m=+145.905011632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle") pod "router-default-6d97cc9d7b-6vfk9" (UID: "7ab80794-ebdf-4d85-bcb2-748a4da45ae6") : configmap references non-existent config key: service-ca.crt
May 21 16:34:13.434156 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:13.433924    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls podName:39cbf8cb-d27b-4e6c-8abc-603723d275fc nodeName:}" failed. No retries permitted until 2026-05-21 16:34:15.433913641 +0000 UTC m=+145.905034602 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-hswx2" (UID: "39cbf8cb-d27b-4e6c-8abc-603723d275fc") : secret "cluster-monitoring-operator-tls" not found
May 21 16:34:13.434156 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:13.433941    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs podName:7ab80794-ebdf-4d85-bcb2-748a4da45ae6 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:15.433930264 +0000 UTC m=+145.905051222 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs") pod "router-default-6d97cc9d7b-6vfk9" (UID: "7ab80794-ebdf-4d85-bcb2-748a4da45ae6") : secret "router-metrics-certs-default" not found
May 21 16:34:15.454742 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:15.454697    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:15.454742 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:15.454743    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:15.455255 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:15.454866    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle podName:7ab80794-ebdf-4d85-bcb2-748a4da45ae6 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:19.454849991 +0000 UTC m=+149.925970954 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle") pod "router-default-6d97cc9d7b-6vfk9" (UID: "7ab80794-ebdf-4d85-bcb2-748a4da45ae6") : configmap references non-existent config key: service-ca.crt
May 21 16:34:15.455255 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:15.454888    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 21 16:34:15.455255 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:15.454948    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:15.455255 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:15.454959    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls podName:39cbf8cb-d27b-4e6c-8abc-603723d275fc nodeName:}" failed. No retries permitted until 2026-05-21 16:34:19.45494217 +0000 UTC m=+149.926063130 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-hswx2" (UID: "39cbf8cb-d27b-4e6c-8abc-603723d275fc") : secret "cluster-monitoring-operator-tls" not found
May 21 16:34:15.455255 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:15.454980    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:15.455255 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:15.455026    2562 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 16:34:15.455255 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:15.455037    2562 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68fc7c8669-4grz2: secret "image-registry-tls" not found
May 21 16:34:15.455255 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:15.455099    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls podName:d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:19.455088182 +0000 UTC m=+149.926209143 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls") pod "image-registry-68fc7c8669-4grz2" (UID: "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5") : secret "image-registry-tls" not found
May 21 16:34:15.455255 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:15.455102    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 21 16:34:15.455255 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:15.455137    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs podName:7ab80794-ebdf-4d85-bcb2-748a4da45ae6 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:19.45512715 +0000 UTC m=+149.926248107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs") pod "router-default-6d97cc9d7b-6vfk9" (UID: "7ab80794-ebdf-4d85-bcb2-748a4da45ae6") : secret "router-metrics-certs-default" not found
May 21 16:34:15.527100 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:15.527064    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb" event={"ID":"271119ee-1b3f-422f-adba-30db5318d9ec","Type":"ContainerStarted","Data":"e0d9608c0805f3853b17f9563c8187813de3192daf134a7e888dd339651d73f7"}
May 21 16:34:15.544427 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:15.544376    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb" podStartSLOduration=1.4820873319999999 podStartE2EDuration="4.544360947s" podCreationTimestamp="2026-05-21 16:34:11 +0000 UTC" firstStartedPulling="2026-05-21 16:34:12.265433915 +0000 UTC m=+142.736554872" lastFinishedPulling="2026-05-21 16:34:15.32770753 +0000 UTC m=+145.798828487" observedRunningTime="2026-05-21 16:34:15.54374931 +0000 UTC m=+146.014870290" watchObservedRunningTime="2026-05-21 16:34:15.544360947 +0000 UTC m=+146.015481925"
May 21 16:34:19.137571 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:19.137530    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pmgp7_118ef813-488d-4e5f-a9a5-9e6579c03825/dns-node-resolver/0.log"
May 21 16:34:19.486680 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:19.486575    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:19.486680 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:19.486623    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:19.486680 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:19.486662    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:19.486680 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:19.486681    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:19.486991 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:19.486740    2562 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 16:34:19.486991 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:19.486761    2562 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68fc7c8669-4grz2: secret "image-registry-tls" not found
May 21 16:34:19.486991 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:19.486809    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 21 16:34:19.486991 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:19.486814    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 21 16:34:19.486991 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:19.486822    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls podName:d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:27.486806343 +0000 UTC m=+157.957927299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls") pod "image-registry-68fc7c8669-4grz2" (UID: "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5") : secret "image-registry-tls" not found
May 21 16:34:19.486991 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:19.486882    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle podName:7ab80794-ebdf-4d85-bcb2-748a4da45ae6 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:27.486867114 +0000 UTC m=+157.957988070 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle") pod "router-default-6d97cc9d7b-6vfk9" (UID: "7ab80794-ebdf-4d85-bcb2-748a4da45ae6") : configmap references non-existent config key: service-ca.crt
May 21 16:34:19.486991 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:19.486892    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs podName:7ab80794-ebdf-4d85-bcb2-748a4da45ae6 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:27.486886587 +0000 UTC m=+157.958007543 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs") pod "router-default-6d97cc9d7b-6vfk9" (UID: "7ab80794-ebdf-4d85-bcb2-748a4da45ae6") : secret "router-metrics-certs-default" not found
May 21 16:34:19.486991 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:19.486908    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls podName:39cbf8cb-d27b-4e6c-8abc-603723d275fc nodeName:}" failed. No retries permitted until 2026-05-21 16:34:27.48690262 +0000 UTC m=+157.958023576 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-hswx2" (UID: "39cbf8cb-d27b-4e6c-8abc-603723d275fc") : secret "cluster-monitoring-operator-tls" not found
May 21 16:34:20.138850 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:20.138822    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jslnj_24b5a0be-656b-4d0f-8b47-7f43220f88a8/node-ca/0.log"
May 21 16:34:27.549728 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:27.549683    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:27.549728 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:27.549729    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:27.550303 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:27.549773    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:27.550303 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:27.549803    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:27.550303 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:27.549863    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 21 16:34:27.550303 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:27.549931    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs podName:7ab80794-ebdf-4d85-bcb2-748a4da45ae6 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:43.549913403 +0000 UTC m=+174.021034382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs") pod "router-default-6d97cc9d7b-6vfk9" (UID: "7ab80794-ebdf-4d85-bcb2-748a4da45ae6") : secret "router-metrics-certs-default" not found
May 21 16:34:27.550303 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:27.549938    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 21 16:34:27.550303 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:27.549950    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle podName:7ab80794-ebdf-4d85-bcb2-748a4da45ae6 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:43.549941967 +0000 UTC m=+174.021062931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle") pod "router-default-6d97cc9d7b-6vfk9" (UID: "7ab80794-ebdf-4d85-bcb2-748a4da45ae6") : configmap references non-existent config key: service-ca.crt
May 21 16:34:27.550303 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:27.550001    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls podName:39cbf8cb-d27b-4e6c-8abc-603723d275fc nodeName:}" failed. No retries permitted until 2026-05-21 16:34:43.549981453 +0000 UTC m=+174.021102423 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-hswx2" (UID: "39cbf8cb-d27b-4e6c-8abc-603723d275fc") : secret "cluster-monitoring-operator-tls" not found
May 21 16:34:27.552261 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:27.552240    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls\") pod \"image-registry-68fc7c8669-4grz2\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") " pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:27.653316 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:27.653267    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:27.782976 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:27.782942    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68fc7c8669-4grz2"]
May 21 16:34:27.785971 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:34:27.785941    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd98d5bed_f2ad_4cf8_b975_8bd08f64bbf5.slice/crio-e6391ec0d601431b682fa7752e4e5e8be9372547ac5c4f37c1e321f5acb6d6f4 WatchSource:0}: Error finding container e6391ec0d601431b682fa7752e4e5e8be9372547ac5c4f37c1e321f5acb6d6f4: Status 404 returned error can't find the container with id e6391ec0d601431b682fa7752e4e5e8be9372547ac5c4f37c1e321f5acb6d6f4
May 21 16:34:28.553668 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:28.553629    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68fc7c8669-4grz2" event={"ID":"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5","Type":"ContainerStarted","Data":"67c122da6fbe1adce9f2ef446eaa332bcf7bde46ab120281d66a145cd756d274"}
May 21 16:34:28.553668 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:28.553673    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68fc7c8669-4grz2" event={"ID":"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5","Type":"ContainerStarted","Data":"e6391ec0d601431b682fa7752e4e5e8be9372547ac5c4f37c1e321f5acb6d6f4"}
May 21 16:34:28.554177 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:28.553769    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:28.577027 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:28.576979    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68fc7c8669-4grz2" podStartSLOduration=17.57696447 podStartE2EDuration="17.57696447s" podCreationTimestamp="2026-05-21 16:34:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 16:34:28.575899146 +0000 UTC m=+159.047020126" watchObservedRunningTime="2026-05-21 16:34:28.57696447 +0000 UTC m=+159.048085448"
May 21 16:34:31.396752 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:31.396706    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dblrj" podUID="52b56561-b24c-4152-ba4d-c89944871373"
May 21 16:34:31.412003 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:31.411976    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-7kpdj" podUID="96db6d8f-de2e-4b26-8601-d4994f37e209"
May 21 16:34:31.558983 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:31.558954    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dblrj"
May 21 16:34:33.114706 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:33.114656    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-hrrn5" podUID="bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f"
May 21 16:34:36.326216 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:36.326165    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:34:36.326603 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:36.326249    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:34:36.328662 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:36.328640    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52b56561-b24c-4152-ba4d-c89944871373-metrics-tls\") pod \"dns-default-dblrj\" (UID: \"52b56561-b24c-4152-ba4d-c89944871373\") " pod="openshift-dns/dns-default-dblrj"
May 21 16:34:36.328852 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:36.328833    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96db6d8f-de2e-4b26-8601-d4994f37e209-cert\") pod \"ingress-canary-7kpdj\" (UID: \"96db6d8f-de2e-4b26-8601-d4994f37e209\") " pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:34:36.362697 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:36.362666    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tnm5s\""
May 21 16:34:36.370756 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:36.370729    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dblrj"
May 21 16:34:36.497888 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:36.497850    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dblrj"]
May 21 16:34:36.500827 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:34:36.500786    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b56561_b24c_4152_ba4d_c89944871373.slice/crio-9b507482c2333c4e63d044b75545d3fb505b604a195b851875640e1b30ecee86 WatchSource:0}: Error finding container 9b507482c2333c4e63d044b75545d3fb505b604a195b851875640e1b30ecee86: Status 404 returned error can't find the container with id 9b507482c2333c4e63d044b75545d3fb505b604a195b851875640e1b30ecee86
May 21 16:34:36.570856 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:36.570814    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dblrj" event={"ID":"52b56561-b24c-4152-ba4d-c89944871373","Type":"ContainerStarted","Data":"9b507482c2333c4e63d044b75545d3fb505b604a195b851875640e1b30ecee86"}
May 21 16:34:38.577926 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:38.577888    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dblrj" event={"ID":"52b56561-b24c-4152-ba4d-c89944871373","Type":"ContainerStarted","Data":"a66048dd849eae81854366e6dfdbf4dbe87cb300c12cbc192b99c39f5f47ec48"}
May 21 16:34:38.577926 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:38.577928    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dblrj" event={"ID":"52b56561-b24c-4152-ba4d-c89944871373","Type":"ContainerStarted","Data":"b5458d2e63cac425304b91e413bf8702999bbaa5a1413f4bbb419a45e6447928"}
May 21 16:34:38.578474 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:38.578006    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dblrj"
May 21 16:34:38.595327 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:38.595268    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-dblrj" podStartSLOduration=129.159809825 podStartE2EDuration="2m10.595250228s" podCreationTimestamp="2026-05-21 16:32:28 +0000 UTC" firstStartedPulling="2026-05-21 16:34:36.502842012 +0000 UTC m=+166.973962969" lastFinishedPulling="2026-05-21 16:34:37.938282412 +0000 UTC m=+168.409403372" observedRunningTime="2026-05-21 16:34:38.594880112 +0000 UTC m=+169.066001090" watchObservedRunningTime="2026-05-21 16:34:38.595250228 +0000 UTC m=+169.066371212"
May 21 16:34:40.979432 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:40.979397    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68fc7c8669-4grz2"]
May 21 16:34:41.002799 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.002762    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5nr7k"]
May 21 16:34:41.004964 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.004942    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.007459 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.007439    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 21 16:34:41.007567 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.007523    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 21 16:34:41.007811 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.007795    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 21 16:34:41.007857 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.007808    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c2kc8\""
May 21 16:34:41.007946 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.007852    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 21 16:34:41.018583 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.018554    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5nr7k"]
May 21 16:34:41.163431 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.163389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.163576 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.163455    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-crio-socket\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.163576 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.163476    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv28x\" (UniqueName: \"kubernetes.io/projected/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-kube-api-access-rv28x\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.163576 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.163528    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-data-volume\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.163576 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.163545    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.264210 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.264125    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-data-volume\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.264210 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.264158    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.264386 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.264212    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.264386 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.264260    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-crio-socket\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.264386 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.264278    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rv28x\" (UniqueName: \"kubernetes.io/projected/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-kube-api-access-rv28x\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.264386 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.264351    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-crio-socket\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.264545 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.264525    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-data-volume\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.264786 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.264763    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.266672 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.266655    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.272433 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.272410    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv28x\" (UniqueName: \"kubernetes.io/projected/e24c6d4b-077c-49ee-b96c-47d0ffec0f61-kube-api-access-rv28x\") pod \"insights-runtime-extractor-5nr7k\" (UID: \"e24c6d4b-077c-49ee-b96c-47d0ffec0f61\") " pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.314131 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.314102    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5nr7k"
May 21 16:34:41.468316 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.468284    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5nr7k"]
May 21 16:34:41.471636 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:34:41.471609    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24c6d4b_077c_49ee_b96c_47d0ffec0f61.slice/crio-ce62718902394804e65d6b13befac8299bdc598fd100a3b2e82b05f703110c15 WatchSource:0}: Error finding container ce62718902394804e65d6b13befac8299bdc598fd100a3b2e82b05f703110c15: Status 404 returned error can't find the container with id ce62718902394804e65d6b13befac8299bdc598fd100a3b2e82b05f703110c15
May 21 16:34:41.586607 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.586566    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5nr7k" event={"ID":"e24c6d4b-077c-49ee-b96c-47d0ffec0f61","Type":"ContainerStarted","Data":"a9d08360662985931d0873d1ba085b0efa66c8b3e42fba4f29b7693dc81f50cf"}
May 21 16:34:41.586607 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:41.586603    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5nr7k" event={"ID":"e24c6d4b-077c-49ee-b96c-47d0ffec0f61","Type":"ContainerStarted","Data":"ce62718902394804e65d6b13befac8299bdc598fd100a3b2e82b05f703110c15"}
May 21 16:34:43.581928 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:43.581878    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:43.581928 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:43.581926    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:43.582361 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:43.582011    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:43.582622 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:43.582602    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-service-ca-bundle\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:43.584611 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:43.584586    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39cbf8cb-d27b-4e6c-8abc-603723d275fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-hswx2\" (UID: \"39cbf8cb-d27b-4e6c-8abc-603723d275fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:43.584729 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:43.584640    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ab80794-ebdf-4d85-bcb2-748a4da45ae6-metrics-certs\") pod \"router-default-6d97cc9d7b-6vfk9\" (UID: \"7ab80794-ebdf-4d85-bcb2-748a4da45ae6\") " pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:43.593070 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:43.593027    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5nr7k" event={"ID":"e24c6d4b-077c-49ee-b96c-47d0ffec0f61","Type":"ContainerStarted","Data":"f5c181773a1d3ab189ced9daf8d27e2b7aa3dbb2289b7a8dc1364b20902de668"}
May 21 16:34:43.839496 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:43.839409    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"
May 21 16:34:43.847280 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:43.847240    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:43.996278 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:43.996234    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2"]
May 21 16:34:44.010492 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:44.010441    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d97cc9d7b-6vfk9"]
May 21 16:34:44.104342 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:44.104255    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:34:44.359504 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:34:44.359425    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39cbf8cb_d27b_4e6c_8abc_603723d275fc.slice/crio-9bd94ed79075c5fef518b9480dca5867ad6d6f99759bf4a94d5d92400ffbe160 WatchSource:0}: Error finding container 9bd94ed79075c5fef518b9480dca5867ad6d6f99759bf4a94d5d92400ffbe160: Status 404 returned error can't find the container with id 9bd94ed79075c5fef518b9480dca5867ad6d6f99759bf4a94d5d92400ffbe160
May 21 16:34:44.360130 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:34:44.360109    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ab80794_ebdf_4d85_bcb2_748a4da45ae6.slice/crio-7f1c7f2f988190d0a2ec732c188d883286973da3f389c046434fa08a6efb3f2b WatchSource:0}: Error finding container 7f1c7f2f988190d0a2ec732c188d883286973da3f389c046434fa08a6efb3f2b: Status 404 returned error can't find the container with id 7f1c7f2f988190d0a2ec732c188d883286973da3f389c046434fa08a6efb3f2b
May 21 16:34:44.598315 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:44.598266    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5nr7k" event={"ID":"e24c6d4b-077c-49ee-b96c-47d0ffec0f61","Type":"ContainerStarted","Data":"4dbb0ab1c5ef88018c99744d6568c66ab9d1cd15e6b30ce5f22dcdb6c9519e8d"}
May 21 16:34:44.599671 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:44.599645    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9" event={"ID":"7ab80794-ebdf-4d85-bcb2-748a4da45ae6","Type":"ContainerStarted","Data":"4f17c3e5682481098bac56dcbac23a252518ef91d33bb1707f83ae592edcb402"}
May 21 16:34:44.599789 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:44.599679    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9" event={"ID":"7ab80794-ebdf-4d85-bcb2-748a4da45ae6","Type":"ContainerStarted","Data":"7f1c7f2f988190d0a2ec732c188d883286973da3f389c046434fa08a6efb3f2b"}
May 21 16:34:44.600654 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:44.600632    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2" event={"ID":"39cbf8cb-d27b-4e6c-8abc-603723d275fc","Type":"ContainerStarted","Data":"9bd94ed79075c5fef518b9480dca5867ad6d6f99759bf4a94d5d92400ffbe160"}
May 21 16:34:44.615235 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:44.615138    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5nr7k" podStartSLOduration=1.746902363 podStartE2EDuration="4.615120986s" podCreationTimestamp="2026-05-21 16:34:40 +0000 UTC" firstStartedPulling="2026-05-21 16:34:41.517909361 +0000 UTC m=+171.989030318" lastFinishedPulling="2026-05-21 16:34:44.386127981 +0000 UTC m=+174.857248941" observedRunningTime="2026-05-21 16:34:44.614335467 +0000 UTC m=+175.085456447" watchObservedRunningTime="2026-05-21 16:34:44.615120986 +0000 UTC m=+175.086241965"
May 21 16:34:44.636483 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:44.636438    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9" podStartSLOduration=33.636421317 podStartE2EDuration="33.636421317s" podCreationTimestamp="2026-05-21 16:34:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 16:34:44.635334701 +0000 UTC m=+175.106455680" watchObservedRunningTime="2026-05-21 16:34:44.636421317 +0000 UTC m=+175.107542288"
May 21 16:34:44.848306 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:44.848262    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:44.850783 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:44.850761    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:45.104182 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:45.104149    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:34:45.106740 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:45.106721    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c67tk\""
May 21 16:34:45.114612 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:45.114599    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7kpdj"
May 21 16:34:45.241293 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:45.241263    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7kpdj"]
May 21 16:34:45.244326 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:34:45.244299    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96db6d8f_de2e_4b26_8601_d4994f37e209.slice/crio-ac94058a084af70e10bc8b64e57c85ab67bfb0719066ce142f675743303f7e5e WatchSource:0}: Error finding container ac94058a084af70e10bc8b64e57c85ab67bfb0719066ce142f675743303f7e5e: Status 404 returned error can't find the container with id ac94058a084af70e10bc8b64e57c85ab67bfb0719066ce142f675743303f7e5e
May 21 16:34:45.607415 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:45.607349    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7kpdj" event={"ID":"96db6d8f-de2e-4b26-8601-d4994f37e209","Type":"ContainerStarted","Data":"ac94058a084af70e10bc8b64e57c85ab67bfb0719066ce142f675743303f7e5e"}
May 21 16:34:45.607926 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:45.607900    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:45.609335 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:45.609314    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6d97cc9d7b-6vfk9"
May 21 16:34:47.614932 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:47.614890    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7kpdj" event={"ID":"96db6d8f-de2e-4b26-8601-d4994f37e209","Type":"ContainerStarted","Data":"e194596a08845ad08b796100fda44f2eeba5ab9c50d10ef17b2f240e8140dfee"}
May 21 16:34:47.616362 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:47.616341    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2" event={"ID":"39cbf8cb-d27b-4e6c-8abc-603723d275fc","Type":"ContainerStarted","Data":"f9e9f495e9235ad1803e7c4fdc6026426ff9f4af9af61271f6cbae36123b809b"}
May 21 16:34:47.632841 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:47.632796    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7kpdj" podStartSLOduration=137.837728852 podStartE2EDuration="2m19.632777168s" podCreationTimestamp="2026-05-21 16:32:28 +0000 UTC" firstStartedPulling="2026-05-21 16:34:45.251069278 +0000 UTC m=+175.722190254" lastFinishedPulling="2026-05-21 16:34:47.046117609 +0000 UTC m=+177.517238570" observedRunningTime="2026-05-21 16:34:47.632197819 +0000 UTC m=+178.103318813" watchObservedRunningTime="2026-05-21 16:34:47.632777168 +0000 UTC m=+178.103898147"
May 21 16:34:47.646716 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:47.646665    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-hswx2" podStartSLOduration=33.963481345 podStartE2EDuration="36.646647523s" podCreationTimestamp="2026-05-21 16:34:11 +0000 UTC" firstStartedPulling="2026-05-21 16:34:44.36133624 +0000 UTC m=+174.832457214" lastFinishedPulling="2026-05-21 16:34:47.044502421 +0000 UTC m=+177.515623392" observedRunningTime="2026-05-21 16:34:47.646566462 +0000 UTC m=+178.117687441" watchObservedRunningTime="2026-05-21 16:34:47.646647523 +0000 UTC m=+178.117768503"
May 21 16:34:48.583808 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:48.583775    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dblrj"
May 21 16:34:50.601356 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.601320    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"]
May 21 16:34:50.604143 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.604125    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:50.606916 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.606888    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
May 21 16:34:50.608103 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.608085    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 21 16:34:50.608180 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.608159    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-h4hhc\""
May 21 16:34:50.608226 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.608159    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
May 21 16:34:50.612688 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.612649    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"]
May 21 16:34:50.634802 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.634770    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3967222d-b640-482f-ae29-8bc9102f7067-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:50.634949 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.634825    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3967222d-b640-482f-ae29-8bc9102f7067-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:50.634949 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.634857    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3967222d-b640-482f-ae29-8bc9102f7067-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:50.634949 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.634901    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp9h2\" (UniqueName: \"kubernetes.io/projected/3967222d-b640-482f-ae29-8bc9102f7067-kube-api-access-jp9h2\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:50.735664 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.735620    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3967222d-b640-482f-ae29-8bc9102f7067-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:50.735813 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.735674    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3967222d-b640-482f-ae29-8bc9102f7067-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:50.735813 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.735698    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jp9h2\" (UniqueName: \"kubernetes.io/projected/3967222d-b640-482f-ae29-8bc9102f7067-kube-api-access-jp9h2\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:50.735813 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.735726    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3967222d-b640-482f-ae29-8bc9102f7067-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:50.735958 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:50.735826    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
May 21 16:34:50.735958 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:50.735882    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3967222d-b640-482f-ae29-8bc9102f7067-prometheus-operator-tls podName:3967222d-b640-482f-ae29-8bc9102f7067 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:51.235867442 +0000 UTC m=+181.706988398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3967222d-b640-482f-ae29-8bc9102f7067-prometheus-operator-tls") pod "prometheus-operator-7779bbdd78-d8qpk" (UID: "3967222d-b640-482f-ae29-8bc9102f7067") : secret "prometheus-operator-tls" not found
May 21 16:34:50.736400 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.736371    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3967222d-b640-482f-ae29-8bc9102f7067-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:50.738148 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.738127    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3967222d-b640-482f-ae29-8bc9102f7067-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:50.747746 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.747718    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp9h2\" (UniqueName: \"kubernetes.io/projected/3967222d-b640-482f-ae29-8bc9102f7067-kube-api-access-jp9h2\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:50.985305 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:50.985234    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:34:51.239740 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:51.239643    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3967222d-b640-482f-ae29-8bc9102f7067-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:51.239905 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:51.239769    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
May 21 16:34:51.239905 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:51.239824    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3967222d-b640-482f-ae29-8bc9102f7067-prometheus-operator-tls podName:3967222d-b640-482f-ae29-8bc9102f7067 nodeName:}" failed. No retries permitted until 2026-05-21 16:34:52.239808696 +0000 UTC m=+182.710929653 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3967222d-b640-482f-ae29-8bc9102f7067-prometheus-operator-tls") pod "prometheus-operator-7779bbdd78-d8qpk" (UID: "3967222d-b640-482f-ae29-8bc9102f7067") : secret "prometheus-operator-tls" not found
May 21 16:34:52.246303 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:52.246256    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3967222d-b640-482f-ae29-8bc9102f7067-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:52.248824 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:52.248795    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3967222d-b640-482f-ae29-8bc9102f7067-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-d8qpk\" (UID: \"3967222d-b640-482f-ae29-8bc9102f7067\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:52.412558 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:52.412522    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"
May 21 16:34:52.553192 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:52.553105    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk"]
May 21 16:34:52.555698 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:34:52.555667    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3967222d_b640_482f_ae29_8bc9102f7067.slice/crio-b51c38f1dac0d7f64369cfeea283ce5ff11f5f5457354f3058d1e215d26939e0 WatchSource:0}: Error finding container b51c38f1dac0d7f64369cfeea283ce5ff11f5f5457354f3058d1e215d26939e0: Status 404 returned error can't find the container with id b51c38f1dac0d7f64369cfeea283ce5ff11f5f5457354f3058d1e215d26939e0
May 21 16:34:52.633219 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:52.633176    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk" event={"ID":"3967222d-b640-482f-ae29-8bc9102f7067","Type":"ContainerStarted","Data":"b51c38f1dac0d7f64369cfeea283ce5ff11f5f5457354f3058d1e215d26939e0"}
May 21 16:34:54.639996 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:54.639966    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk" event={"ID":"3967222d-b640-482f-ae29-8bc9102f7067","Type":"ContainerStarted","Data":"dde704a4bbece5f8f8e3b1f70342712c25da757cadc6c5daa59baa48f0aa5427"}
May 21 16:34:54.639996 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:54.640000    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk" event={"ID":"3967222d-b640-482f-ae29-8bc9102f7067","Type":"ContainerStarted","Data":"0cdb88864c06623f7c75ffa3214889155c3b3572fdfd7a90db2782d28f4efb5e"}
May 21 16:34:54.657411 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:54.657352    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7779bbdd78-d8qpk" podStartSLOduration=3.337646561 podStartE2EDuration="4.657335028s" podCreationTimestamp="2026-05-21 16:34:50 +0000 UTC" firstStartedPulling="2026-05-21 16:34:52.557507646 +0000 UTC m=+183.028628603" lastFinishedPulling="2026-05-21 16:34:53.877196113 +0000 UTC m=+184.348317070" observedRunningTime="2026-05-21 16:34:54.656512342 +0000 UTC m=+185.127633322" watchObservedRunningTime="2026-05-21 16:34:54.657335028 +0000 UTC m=+185.128456009"
May 21 16:34:56.972588 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:56.972556    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"]
May 21 16:34:56.974861 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:56.974843    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:56.977288 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:56.977261    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
May 21 16:34:56.977417 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:56.977336    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-v8rcq\""
May 21 16:34:56.977417 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:56.977338    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
May 21 16:34:56.989300 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:56.989274    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lmhtg"]
May 21 16:34:56.991442 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:56.991409    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:56.993079 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:56.993035    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"]
May 21 16:34:56.993960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:56.993937    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 21 16:34:56.993960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:56.993952    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 21 16:34:56.994135 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:56.994088    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 21 16:34:56.994200 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:56.994185    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4j794\""
May 21 16:34:57.085858 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.085827    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-sys\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.086043 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.085871    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-root\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.086043 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.085894    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9139e24-9adb-4c84-a365-6a80796df35b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-hjzqz\" (UID: \"d9139e24-9adb-4c84-a365-6a80796df35b\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:57.086043 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.085960    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-accelerators-collector-config\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.086504 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.086065    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfl6k\" (UniqueName: \"kubernetes.io/projected/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-kube-api-access-rfl6k\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.086504 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.086122    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9139e24-9adb-4c84-a365-6a80796df35b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-hjzqz\" (UID: \"d9139e24-9adb-4c84-a365-6a80796df35b\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:57.086504 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.086157    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-tls\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.086504 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.086185    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9139e24-9adb-4c84-a365-6a80796df35b-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-hjzqz\" (UID: \"d9139e24-9adb-4c84-a365-6a80796df35b\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:57.086504 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.086213    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z99lw\" (UniqueName: \"kubernetes.io/projected/d9139e24-9adb-4c84-a365-6a80796df35b-kube-api-access-z99lw\") pod \"openshift-state-metrics-9c959559f-hjzqz\" (UID: \"d9139e24-9adb-4c84-a365-6a80796df35b\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:57.086504 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.086244    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.086504 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.086310    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-wtmp\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.086504 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.086342    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-textfile\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.086504 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.086365    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-metrics-client-ca\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.187307 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.187256    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-root\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.187490 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.187317    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9139e24-9adb-4c84-a365-6a80796df35b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-hjzqz\" (UID: \"d9139e24-9adb-4c84-a365-6a80796df35b\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:57.187490 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.187351    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-accelerators-collector-config\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.187490 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.187354    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-root\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.187490 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.187381    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rfl6k\" (UniqueName: \"kubernetes.io/projected/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-kube-api-access-rfl6k\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.187490 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.187468    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9139e24-9adb-4c84-a365-6a80796df35b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-hjzqz\" (UID: \"d9139e24-9adb-4c84-a365-6a80796df35b\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:57.187759 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:57.187593    2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
May 21 16:34:57.187759 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:34:57.187706    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9139e24-9adb-4c84-a365-6a80796df35b-openshift-state-metrics-tls podName:d9139e24-9adb-4c84-a365-6a80796df35b nodeName:}" failed. No retries permitted until 2026-05-21 16:34:57.687682321 +0000 UTC m=+188.158803298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d9139e24-9adb-4c84-a365-6a80796df35b-openshift-state-metrics-tls") pod "openshift-state-metrics-9c959559f-hjzqz" (UID: "d9139e24-9adb-4c84-a365-6a80796df35b") : secret "openshift-state-metrics-tls" not found
May 21 16:34:57.187928 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.187900    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-tls\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.188175 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.187952    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9139e24-9adb-4c84-a365-6a80796df35b-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-hjzqz\" (UID: \"d9139e24-9adb-4c84-a365-6a80796df35b\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:57.188175 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.187986    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z99lw\" (UniqueName: \"kubernetes.io/projected/d9139e24-9adb-4c84-a365-6a80796df35b-kube-api-access-z99lw\") pod \"openshift-state-metrics-9c959559f-hjzqz\" (UID: \"d9139e24-9adb-4c84-a365-6a80796df35b\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:57.188175 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.188008    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.188175 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.188041    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-wtmp\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.188175 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.188103    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-textfile\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.188175 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.188137    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-metrics-client-ca\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.188175 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.188158    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-accelerators-collector-config\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.188641 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.188182    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-sys\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.188641 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.188268    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-wtmp\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.188641 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.188276    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-sys\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.188641 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.188412    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-textfile\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.188939 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.188730    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9139e24-9adb-4c84-a365-6a80796df35b-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-hjzqz\" (UID: \"d9139e24-9adb-4c84-a365-6a80796df35b\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:57.188939 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.188931    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-metrics-client-ca\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.190366 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.190332    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-tls\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.190450 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.190371    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9139e24-9adb-4c84-a365-6a80796df35b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-hjzqz\" (UID: \"d9139e24-9adb-4c84-a365-6a80796df35b\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:57.190911 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.190892    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.204660 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.204626    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfl6k\" (UniqueName: \"kubernetes.io/projected/b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9-kube-api-access-rfl6k\") pod \"node-exporter-lmhtg\" (UID: \"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9\") " pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.204818 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.204698    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z99lw\" (UniqueName: \"kubernetes.io/projected/d9139e24-9adb-4c84-a365-6a80796df35b-kube-api-access-z99lw\") pod \"openshift-state-metrics-9c959559f-hjzqz\" (UID: \"d9139e24-9adb-4c84-a365-6a80796df35b\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:57.301739 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.301698    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lmhtg"
May 21 16:34:57.309908 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:34:57.309881    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0cc0f3e_7e26_4759_a3f0_9001acc6c2a9.slice/crio-1af12ecb5009672282041dbf5e24497c0f1f077c7c4bd4ffc98886d3d64041da WatchSource:0}: Error finding container 1af12ecb5009672282041dbf5e24497c0f1f077c7c4bd4ffc98886d3d64041da: Status 404 returned error can't find the container with id 1af12ecb5009672282041dbf5e24497c0f1f077c7c4bd4ffc98886d3d64041da
May 21 16:34:57.649555 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.649474    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lmhtg" event={"ID":"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9","Type":"ContainerStarted","Data":"1af12ecb5009672282041dbf5e24497c0f1f077c7c4bd4ffc98886d3d64041da"}
May 21 16:34:57.692115 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.692077    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9139e24-9adb-4c84-a365-6a80796df35b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-hjzqz\" (UID: \"d9139e24-9adb-4c84-a365-6a80796df35b\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:57.694717 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.694673    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9139e24-9adb-4c84-a365-6a80796df35b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-hjzqz\" (UID: \"d9139e24-9adb-4c84-a365-6a80796df35b\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:57.884485 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:57.884447    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"
May 21 16:34:58.055474 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:58.055444    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz"]
May 21 16:34:58.058784 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:34:58.058745    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9139e24_9adb_4c84_a365_6a80796df35b.slice/crio-04845a3741f196671e196d89c385e101548a645dd36f8dc5ef953998217ea692 WatchSource:0}: Error finding container 04845a3741f196671e196d89c385e101548a645dd36f8dc5ef953998217ea692: Status 404 returned error can't find the container with id 04845a3741f196671e196d89c385e101548a645dd36f8dc5ef953998217ea692
May 21 16:34:58.653677 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:58.653579    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz" event={"ID":"d9139e24-9adb-4c84-a365-6a80796df35b","Type":"ContainerStarted","Data":"f9e5a2369ce973a197a7bb4ac64df9ab02184b3386e0fdb4089499e73ec71313"}
May 21 16:34:58.653677 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:58.653617    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz" event={"ID":"d9139e24-9adb-4c84-a365-6a80796df35b","Type":"ContainerStarted","Data":"331e2fabd3ab410c7e339e0c68549636570617b7eea18f9279e9b1820e9a1134"}
May 21 16:34:58.653677 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:58.653626    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz" event={"ID":"d9139e24-9adb-4c84-a365-6a80796df35b","Type":"ContainerStarted","Data":"04845a3741f196671e196d89c385e101548a645dd36f8dc5ef953998217ea692"}
May 21 16:34:58.655003 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:58.654977    2562 generic.go:358] "Generic (PLEG): container finished" podID="b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9" containerID="3b6455c9b2f58b90ab7508851174b1bfc22a102c3914184aa4a7df2fddef9748" exitCode=0
May 21 16:34:58.655160 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:58.655013    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lmhtg" event={"ID":"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9","Type":"ContainerDied","Data":"3b6455c9b2f58b90ab7508851174b1bfc22a102c3914184aa4a7df2fddef9748"}
May 21 16:34:59.044950 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.044912    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-77b996bc84-rtsb6"]
May 21 16:34:59.047658 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.047636    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.050412 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.050377    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
May 21 16:34:59.051510 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.051484    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-b7eftogoi4148\""
May 21 16:34:59.051814 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.051793    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
May 21 16:34:59.052310 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.052291    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
May 21 16:34:59.052402 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.052333    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
May 21 16:34:59.052402 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.052347    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
May 21 16:34:59.052840 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.052821    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-57jjf\""
May 21 16:34:59.061625 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.061598    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-77b996bc84-rtsb6"]
May 21 16:34:59.103115 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.103083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f0704d0-1996-4674-a74c-d0c3cb8710d0-metrics-client-ca\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.103294 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.103126    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.103294 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.103230    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-grpc-tls\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.103294 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.103275    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.103479 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.103339    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.103479 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.103378    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-tls\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.103479 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.103411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.103479 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.103462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6f2l\" (UniqueName: \"kubernetes.io/projected/1f0704d0-1996-4674-a74c-d0c3cb8710d0-kube-api-access-d6f2l\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.204174 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.204136    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d6f2l\" (UniqueName: \"kubernetes.io/projected/1f0704d0-1996-4674-a74c-d0c3cb8710d0-kube-api-access-d6f2l\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.204359 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.204205    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f0704d0-1996-4674-a74c-d0c3cb8710d0-metrics-client-ca\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.204359 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.204250    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.204359 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.204313    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-grpc-tls\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.204531 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.204440    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.204531 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.204505    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.204632 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.204544    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-tls\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.204632 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.204569    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.205067 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.205003    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f0704d0-1996-4674-a74c-d0c3cb8710d0-metrics-client-ca\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.207187 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.207136    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.207443 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.207420    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-grpc-tls\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.207687 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.207667    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.207795 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.207779    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-tls\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.207964 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.207943    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.207999 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.207954    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f0704d0-1996-4674-a74c-d0c3cb8710d0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.218549 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.218526    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6f2l\" (UniqueName: \"kubernetes.io/projected/1f0704d0-1996-4674-a74c-d0c3cb8710d0-kube-api-access-d6f2l\") pod \"thanos-querier-77b996bc84-rtsb6\" (UID: \"1f0704d0-1996-4674-a74c-d0c3cb8710d0\") " pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.359859 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.359773    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:34:59.631463 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.631367    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-77b996bc84-rtsb6"]
May 21 16:34:59.634923 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:34:59.634895    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f0704d0_1996_4674_a74c_d0c3cb8710d0.slice/crio-5eecc776280bad19e53683d383e6a237bb093b49c513c581b27181822306f14b WatchSource:0}: Error finding container 5eecc776280bad19e53683d383e6a237bb093b49c513c581b27181822306f14b: Status 404 returned error can't find the container with id 5eecc776280bad19e53683d383e6a237bb093b49c513c581b27181822306f14b
May 21 16:34:59.660096 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.660042    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lmhtg" event={"ID":"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9","Type":"ContainerStarted","Data":"9bb4ae987b487a145ef083d67e26d10e219e528a5487a6982fa170c187871f4d"}
May 21 16:34:59.660280 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.660108    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lmhtg" event={"ID":"b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9","Type":"ContainerStarted","Data":"c4a35c9bfdbd29a53598f07bf14e5b389c7a0e795aa9f5babecec5e6167d5d3a"}
May 21 16:34:59.661898 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.661869    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz" event={"ID":"d9139e24-9adb-4c84-a365-6a80796df35b","Type":"ContainerStarted","Data":"add64f7d575bdcfac34d767469ec277816bff682809531b2fb3806c94d2b806b"}
May 21 16:34:59.662841 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.662821    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6" event={"ID":"1f0704d0-1996-4674-a74c-d0c3cb8710d0","Type":"ContainerStarted","Data":"5eecc776280bad19e53683d383e6a237bb093b49c513c581b27181822306f14b"}
May 21 16:34:59.681645 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.681591    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lmhtg" podStartSLOduration=2.6436880560000002 podStartE2EDuration="3.681576781s" podCreationTimestamp="2026-05-21 16:34:56 +0000 UTC" firstStartedPulling="2026-05-21 16:34:57.311698365 +0000 UTC m=+187.782819323" lastFinishedPulling="2026-05-21 16:34:58.349587088 +0000 UTC m=+188.820708048" observedRunningTime="2026-05-21 16:34:59.681182941 +0000 UTC m=+190.152303899" watchObservedRunningTime="2026-05-21 16:34:59.681576781 +0000 UTC m=+190.152697760"
May 21 16:34:59.704507 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:34:59.704453    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-9c959559f-hjzqz" podStartSLOduration=2.583717989 podStartE2EDuration="3.704435142s" podCreationTimestamp="2026-05-21 16:34:56 +0000 UTC" firstStartedPulling="2026-05-21 16:34:58.384246023 +0000 UTC m=+188.855366983" lastFinishedPulling="2026-05-21 16:34:59.504963179 +0000 UTC m=+189.976084136" observedRunningTime="2026-05-21 16:34:59.703775345 +0000 UTC m=+190.174896323" watchObservedRunningTime="2026-05-21 16:34:59.704435142 +0000 UTC m=+190.175556120"
May 21 16:35:02.676230 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:02.676193    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6" event={"ID":"1f0704d0-1996-4674-a74c-d0c3cb8710d0","Type":"ContainerStarted","Data":"0ea19bdca4cb5e4b1b016875b35117d938683a43f88c87014876bb540350cf44"}
May 21 16:35:02.676230 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:02.676232    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6" event={"ID":"1f0704d0-1996-4674-a74c-d0c3cb8710d0","Type":"ContainerStarted","Data":"6d210298a92b25d500275eef60178857a19d8febd5437639dc000b3b273ab68e"}
May 21 16:35:02.676569 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:02.676241    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6" event={"ID":"1f0704d0-1996-4674-a74c-d0c3cb8710d0","Type":"ContainerStarted","Data":"9cdb39ccba73ca6b657101b4313551e84bbb5677c0fde9f0be3343719aeb287b"}
May 21 16:35:02.676569 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:02.676249    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6" event={"ID":"1f0704d0-1996-4674-a74c-d0c3cb8710d0","Type":"ContainerStarted","Data":"e3d756f6d3ea04f563ac6438296852e78a1c4e86aa9b012d5d5a8677f2c864d7"}
May 21 16:35:03.340874 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.340831    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 21 16:35:03.344263 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.344238    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.349803 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.349778    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2bl5725n64ikv\""
May 21 16:35:03.350960 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.350931    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 21 16:35:03.351257 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.351236    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 21 16:35:03.351454 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.351435    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 21 16:35:03.351674 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.351655    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 21 16:35:03.351889 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.351867    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 21 16:35:03.352135 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.352116    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 21 16:35:03.352367 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.352348    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 21 16:35:03.352636 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.352610    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nwptl\""
May 21 16:35:03.352934 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.352887    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 21 16:35:03.353143 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.353128    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 21 16:35:03.353348 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.353319    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 21 16:35:03.353348 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.353341    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 21 16:35:03.353470 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.353450    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 21 16:35:03.363581 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.363243    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 21 16:35:03.376248 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.376214    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 21 16:35:03.443086 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443029    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443086 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443090    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-config\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443317 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443119    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d443dcd-5f3d-4277-80fe-57ca419c7201-config-out\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443317 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443197    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443317 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443243    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443317 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443293    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443317 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443523 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443342    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443523 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443367    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443523 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443388    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d443dcd-5f3d-4277-80fe-57ca419c7201-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443523 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443523 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443427    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443523 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443458    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-web-config\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443523 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443501    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443765 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443535    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443765 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443553    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443765 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443581    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.443765 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.443610    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bdgr\" (UniqueName: \"kubernetes.io/projected/2d443dcd-5f3d-4277-80fe-57ca419c7201-kube-api-access-8bdgr\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.544467 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.544432    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8bdgr\" (UniqueName: \"kubernetes.io/projected/2d443dcd-5f3d-4277-80fe-57ca419c7201-kube-api-access-8bdgr\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.544650 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.544480    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.544650 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.544621    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-config\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.544743 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.544668    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d443dcd-5f3d-4277-80fe-57ca419c7201-config-out\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.544743 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.544714    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.544844 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.544750    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.544844 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.544815    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.544950 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.544842    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.545036 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.545009    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.545142 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.545097    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.545221 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.545144    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d443dcd-5f3d-4277-80fe-57ca419c7201-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.545221 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.545171    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.545221 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.545206    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.545373 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.545229    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-web-config\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.545373 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.545252    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.545373 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.545262    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.545373 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.545313    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.545373 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.545337    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.545789 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.545374    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.545789 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.545429    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.546090 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.546068    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.547404 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.547372    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.548454 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.548419    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.548724 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.548597    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-config\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.548792 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.548760    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d443dcd-5f3d-4277-80fe-57ca419c7201-config-out\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.549559 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.549460    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.549559 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.549519    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.550526 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.550303    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.550526 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.550467    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d443dcd-5f3d-4277-80fe-57ca419c7201-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.550676 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.550533    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.550936 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.550916    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.551130 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.551111    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.551397 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.551374    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.551463 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.551403    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.552196 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.552178    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-web-config\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.563289 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.563261    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bdgr\" (UniqueName: \"kubernetes.io/projected/2d443dcd-5f3d-4277-80fe-57ca419c7201-kube-api-access-8bdgr\") pod \"prometheus-k8s-0\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.656321 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.656221    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:03.684963 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.684928    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6" event={"ID":"1f0704d0-1996-4674-a74c-d0c3cb8710d0","Type":"ContainerStarted","Data":"32904b87989f08316991b4a709e83f2b37583ee30d1708d29bf5bb20394fbae1"}
May 21 16:35:03.685395 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.684971    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6" event={"ID":"1f0704d0-1996-4674-a74c-d0c3cb8710d0","Type":"ContainerStarted","Data":"e1572b5a3833b42861c65e615bb9d7b84a0dc93813711979eef6c725abd385a8"}
May 21 16:35:03.685395 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.685115    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:35:03.710118 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.710033    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6" podStartSLOduration=1.747608155 podStartE2EDuration="4.71001146s" podCreationTimestamp="2026-05-21 16:34:59 +0000 UTC" firstStartedPulling="2026-05-21 16:34:59.637691772 +0000 UTC m=+190.108812731" lastFinishedPulling="2026-05-21 16:35:02.600095077 +0000 UTC m=+193.071216036" observedRunningTime="2026-05-21 16:35:03.708833252 +0000 UTC m=+194.179954230" watchObservedRunningTime="2026-05-21 16:35:03.71001146 +0000 UTC m=+194.181132443"
May 21 16:35:03.819447 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:03.819410    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 21 16:35:03.822266 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:35:03.822234    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d443dcd_5f3d_4277_80fe_57ca419c7201.slice/crio-361def981f5e4413290c493c9771bd5da9c5169ad5361c83da5132a1aabe2319 WatchSource:0}: Error finding container 361def981f5e4413290c493c9771bd5da9c5169ad5361c83da5132a1aabe2319: Status 404 returned error can't find the container with id 361def981f5e4413290c493c9771bd5da9c5169ad5361c83da5132a1aabe2319
May 21 16:35:04.689177 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:04.689133    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerStarted","Data":"361def981f5e4413290c493c9771bd5da9c5169ad5361c83da5132a1aabe2319"}
May 21 16:35:05.692963 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:05.692922    2562 generic.go:358] "Generic (PLEG): container finished" podID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerID="fc11d8035d22420420ee23aed08a7081c40b0331c4d87b06b52bdd66b988d782" exitCode=0
May 21 16:35:05.693373 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:05.693008    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerDied","Data":"fc11d8035d22420420ee23aed08a7081c40b0331c4d87b06b52bdd66b988d782"}
May 21 16:35:05.998919 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:05.998824    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-68fc7c8669-4grz2" podUID="d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" containerName="registry" containerID="cri-o://67c122da6fbe1adce9f2ef446eaa332bcf7bde46ab120281d66a145cd756d274" gracePeriod=30
May 21 16:35:06.263205 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.263132    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:35:06.369763 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.369729    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2z5mk\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-kube-api-access-2z5mk\") pod \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") "
May 21 16:35:06.369763 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.369780    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-ca-trust-extracted\") pod \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") "
May 21 16:35:06.370004 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.369825    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-certificates\") pod \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") "
May 21 16:35:06.370004 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.369859    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-trusted-ca\") pod \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") "
May 21 16:35:06.370004 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.369888    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-bound-sa-token\") pod \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") "
May 21 16:35:06.370004 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.369927    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-installation-pull-secrets\") pod \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") "
May 21 16:35:06.370004 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.369959    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls\") pod \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") "
May 21 16:35:06.370004 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.369993    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-image-registry-private-configuration\") pod \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\" (UID: \"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5\") "
May 21 16:35:06.370453 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.370427    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" (UID: "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 21 16:35:06.370561 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.370512    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" (UID: "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 21 16:35:06.372589 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.372559    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-kube-api-access-2z5mk" (OuterVolumeSpecName: "kube-api-access-2z5mk") pod "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" (UID: "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5"). InnerVolumeSpecName "kube-api-access-2z5mk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 16:35:06.372589 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.372571    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" (UID: "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 16:35:06.372852 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.372832    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" (UID: "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 16:35:06.373163 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.373140    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" (UID: "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 16:35:06.373251 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.373174    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" (UID: "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 16:35:06.380655 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.380598    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" (UID: "d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 21 16:35:06.471132 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.471089    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-ca-trust-extracted\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:35:06.471132 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.471131    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-certificates\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:35:06.471324 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.471146    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-trusted-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:35:06.471324 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.471159    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-bound-sa-token\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:35:06.471324 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.471171    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-installation-pull-secrets\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:35:06.471324 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.471183    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-registry-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:35:06.471324 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.471196    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-image-registry-private-configuration\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:35:06.471324 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.471209    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2z5mk\" (UniqueName: \"kubernetes.io/projected/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5-kube-api-access-2z5mk\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:35:06.696721 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.696688    2562 generic.go:358] "Generic (PLEG): container finished" podID="d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" containerID="67c122da6fbe1adce9f2ef446eaa332bcf7bde46ab120281d66a145cd756d274" exitCode=0
May 21 16:35:06.697145 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.696747    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68fc7c8669-4grz2"
May 21 16:35:06.697145 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.696773    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68fc7c8669-4grz2" event={"ID":"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5","Type":"ContainerDied","Data":"67c122da6fbe1adce9f2ef446eaa332bcf7bde46ab120281d66a145cd756d274"}
May 21 16:35:06.697145 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.696811    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68fc7c8669-4grz2" event={"ID":"d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5","Type":"ContainerDied","Data":"e6391ec0d601431b682fa7752e4e5e8be9372547ac5c4f37c1e321f5acb6d6f4"}
May 21 16:35:06.697145 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.696831    2562 scope.go:117] "RemoveContainer" containerID="67c122da6fbe1adce9f2ef446eaa332bcf7bde46ab120281d66a145cd756d274"
May 21 16:35:06.705026 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.705005    2562 scope.go:117] "RemoveContainer" containerID="67c122da6fbe1adce9f2ef446eaa332bcf7bde46ab120281d66a145cd756d274"
May 21 16:35:06.705398 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:35:06.705376    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67c122da6fbe1adce9f2ef446eaa332bcf7bde46ab120281d66a145cd756d274\": container with ID starting with 67c122da6fbe1adce9f2ef446eaa332bcf7bde46ab120281d66a145cd756d274 not found: ID does not exist" containerID="67c122da6fbe1adce9f2ef446eaa332bcf7bde46ab120281d66a145cd756d274"
May 21 16:35:06.705465 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.705407    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67c122da6fbe1adce9f2ef446eaa332bcf7bde46ab120281d66a145cd756d274"} err="failed to get container status \"67c122da6fbe1adce9f2ef446eaa332bcf7bde46ab120281d66a145cd756d274\": rpc error: code = NotFound desc = could not find container \"67c122da6fbe1adce9f2ef446eaa332bcf7bde46ab120281d66a145cd756d274\": container with ID starting with 67c122da6fbe1adce9f2ef446eaa332bcf7bde46ab120281d66a145cd756d274 not found: ID does not exist"
May 21 16:35:06.720730 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.720697    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68fc7c8669-4grz2"]
May 21 16:35:06.724535 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:06.724513    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-68fc7c8669-4grz2"]
May 21 16:35:08.108483 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:08.108448    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" path="/var/lib/kubelet/pods/d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5/volumes"
May 21 16:35:08.706033 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:08.705997    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerStarted","Data":"6088922f1fcc3e0014f90946c4d173df5c40359535da0801d8f1205fa662afec"}
May 21 16:35:09.695860 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:09.695834    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-77b996bc84-rtsb6"
May 21 16:35:09.718569 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:09.718526    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerStarted","Data":"a29fdffd724e34b7895febe0318b760643995d7b071f80406d3bb083ab669915"}
May 21 16:35:09.718569 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:09.718574    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerStarted","Data":"02f0145e3ce048dfee15cadd61327e317fcaac4830e06337ead313baa9248975"}
May 21 16:35:09.718770 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:09.718588    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerStarted","Data":"92ade71e706fc38f06632bd1ccfca540f25c4dda3cf088cad91d36786f291533"}
May 21 16:35:09.718770 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:09.718602    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerStarted","Data":"eaff75812ff5a71ef11eebe6ba844cacaf4f717061c144d5787597fdb8e80383"}
May 21 16:35:09.718770 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:09.718613    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerStarted","Data":"3f999a716cb0c6e695de00a36df5dd5aab974ca2d820dc06029b3afb7e238195"}
May 21 16:35:09.749528 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:09.749474    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.9701297960000002 podStartE2EDuration="6.74945554s" podCreationTimestamp="2026-05-21 16:35:03 +0000 UTC" firstStartedPulling="2026-05-21 16:35:03.826191755 +0000 UTC m=+194.297312714" lastFinishedPulling="2026-05-21 16:35:08.605517501 +0000 UTC m=+199.076638458" observedRunningTime="2026-05-21 16:35:09.747883736 +0000 UTC m=+200.219004718" watchObservedRunningTime="2026-05-21 16:35:09.74945554 +0000 UTC m=+200.220576553"
May 21 16:35:13.656738 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:13.656695    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:35:36.797401 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:36.797367    2562 generic.go:358] "Generic (PLEG): container finished" podID="271119ee-1b3f-422f-adba-30db5318d9ec" containerID="e0d9608c0805f3853b17f9563c8187813de3192daf134a7e888dd339651d73f7" exitCode=0
May 21 16:35:36.797794 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:36.797442    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb" event={"ID":"271119ee-1b3f-422f-adba-30db5318d9ec","Type":"ContainerDied","Data":"e0d9608c0805f3853b17f9563c8187813de3192daf134a7e888dd339651d73f7"}
May 21 16:35:36.797843 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:36.797805    2562 scope.go:117] "RemoveContainer" containerID="e0d9608c0805f3853b17f9563c8187813de3192daf134a7e888dd339651d73f7"
May 21 16:35:37.802257 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:35:37.802215    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-vwwlb" event={"ID":"271119ee-1b3f-422f-adba-30db5318d9ec","Type":"ContainerStarted","Data":"0aab45c0ec5631cd235285271ba60c8adc52322f6f59d053a909d328eee49a0d"}
May 21 16:36:00.843461 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:00.843424    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:36:00.845954 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:00.845926    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f-metrics-certs\") pod \"network-metrics-daemon-hrrn5\" (UID: \"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f\") " pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:36:00.908620 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:00.908591    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tq9vg\""
May 21 16:36:00.915894 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:00.915865    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrn5"
May 21 16:36:01.086872 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:01.086842    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hrrn5"]
May 21 16:36:01.090161 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:36:01.090121    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd3e235e_1b4f_47b0_ad4f_527c6d3b2f9f.slice/crio-448667c3d1b4f01eaeff776982fd9d5ee820e68fb0b7e4707061a72fc61da7b9 WatchSource:0}: Error finding container 448667c3d1b4f01eaeff776982fd9d5ee820e68fb0b7e4707061a72fc61da7b9: Status 404 returned error can't find the container with id 448667c3d1b4f01eaeff776982fd9d5ee820e68fb0b7e4707061a72fc61da7b9
May 21 16:36:01.872157 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:01.872114    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hrrn5" event={"ID":"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f","Type":"ContainerStarted","Data":"448667c3d1b4f01eaeff776982fd9d5ee820e68fb0b7e4707061a72fc61da7b9"}
May 21 16:36:02.877123 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:02.877079    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hrrn5" event={"ID":"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f","Type":"ContainerStarted","Data":"47bad72fd6e8095a294faf48adae7d1968f4513b3d02b3dec646e1af72a36ee2"}
May 21 16:36:02.877123 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:02.877126    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hrrn5" event={"ID":"bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f","Type":"ContainerStarted","Data":"f25c51db23b9f2a7439c0454d5e159f974999d76214a12fb137b614811ce221d"}
May 21 16:36:02.896067 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:02.896007    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hrrn5" podStartSLOduration=251.908094156 podStartE2EDuration="4m12.895989373s" podCreationTimestamp="2026-05-21 16:31:50 +0000 UTC" firstStartedPulling="2026-05-21 16:36:01.092615683 +0000 UTC m=+251.563736639" lastFinishedPulling="2026-05-21 16:36:02.080510899 +0000 UTC m=+252.551631856" observedRunningTime="2026-05-21 16:36:02.895897782 +0000 UTC m=+253.367018818" watchObservedRunningTime="2026-05-21 16:36:02.895989373 +0000 UTC m=+253.367110354"
May 21 16:36:03.656808 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:03.656747    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:03.673238 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:03.673202    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:03.895719 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:03.895684    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:21.652986 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.652902    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 21 16:36:21.653609 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.653394    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="prometheus" containerID="cri-o://6088922f1fcc3e0014f90946c4d173df5c40359535da0801d8f1205fa662afec" gracePeriod=600
May 21 16:36:21.653609 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.653411    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="thanos-sidecar" containerID="cri-o://eaff75812ff5a71ef11eebe6ba844cacaf4f717061c144d5787597fdb8e80383" gracePeriod=600
May 21 16:36:21.653609 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.653431    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="kube-rbac-proxy" containerID="cri-o://02f0145e3ce048dfee15cadd61327e317fcaac4830e06337ead313baa9248975" gracePeriod=600
May 21 16:36:21.653609 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.653499    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="kube-rbac-proxy-web" containerID="cri-o://92ade71e706fc38f06632bd1ccfca540f25c4dda3cf088cad91d36786f291533" gracePeriod=600
May 21 16:36:21.653609 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.653528    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="config-reloader" containerID="cri-o://3f999a716cb0c6e695de00a36df5dd5aab974ca2d820dc06029b3afb7e238195" gracePeriod=600
May 21 16:36:21.653609 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.653579    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a29fdffd724e34b7895febe0318b760643995d7b071f80406d3bb083ab669915" gracePeriod=600
May 21 16:36:21.938210 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.938120    2562 generic.go:358] "Generic (PLEG): container finished" podID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerID="a29fdffd724e34b7895febe0318b760643995d7b071f80406d3bb083ab669915" exitCode=0
May 21 16:36:21.938210 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.938155    2562 generic.go:358] "Generic (PLEG): container finished" podID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerID="02f0145e3ce048dfee15cadd61327e317fcaac4830e06337ead313baa9248975" exitCode=0
May 21 16:36:21.938210 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.938165    2562 generic.go:358] "Generic (PLEG): container finished" podID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerID="eaff75812ff5a71ef11eebe6ba844cacaf4f717061c144d5787597fdb8e80383" exitCode=0
May 21 16:36:21.938210 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.938176    2562 generic.go:358] "Generic (PLEG): container finished" podID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerID="3f999a716cb0c6e695de00a36df5dd5aab974ca2d820dc06029b3afb7e238195" exitCode=0
May 21 16:36:21.938210 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.938185    2562 generic.go:358] "Generic (PLEG): container finished" podID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerID="6088922f1fcc3e0014f90946c4d173df5c40359535da0801d8f1205fa662afec" exitCode=0
May 21 16:36:21.938486 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.938175    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerDied","Data":"a29fdffd724e34b7895febe0318b760643995d7b071f80406d3bb083ab669915"}
May 21 16:36:21.938486 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.938259    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerDied","Data":"02f0145e3ce048dfee15cadd61327e317fcaac4830e06337ead313baa9248975"}
May 21 16:36:21.938486 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.938286    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerDied","Data":"eaff75812ff5a71ef11eebe6ba844cacaf4f717061c144d5787597fdb8e80383"}
May 21 16:36:21.938486 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.938300    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerDied","Data":"3f999a716cb0c6e695de00a36df5dd5aab974ca2d820dc06029b3afb7e238195"}
May 21 16:36:21.938486 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:21.938313    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerDied","Data":"6088922f1fcc3e0014f90946c4d173df5c40359535da0801d8f1205fa662afec"}
May 21 16:36:22.906252 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:22.906225    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:22.946045 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:22.946011    2562 generic.go:358] "Generic (PLEG): container finished" podID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerID="92ade71e706fc38f06632bd1ccfca540f25c4dda3cf088cad91d36786f291533" exitCode=0
May 21 16:36:22.946234 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:22.946095    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerDied","Data":"92ade71e706fc38f06632bd1ccfca540f25c4dda3cf088cad91d36786f291533"}
May 21 16:36:22.946234 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:22.946141    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d443dcd-5f3d-4277-80fe-57ca419c7201","Type":"ContainerDied","Data":"361def981f5e4413290c493c9771bd5da9c5169ad5361c83da5132a1aabe2319"}
May 21 16:36:22.946234 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:22.946162    2562 scope.go:117] "RemoveContainer" containerID="a29fdffd724e34b7895febe0318b760643995d7b071f80406d3bb083ab669915"
May 21 16:36:22.946234 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:22.946186    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:22.955906 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:22.955879    2562 scope.go:117] "RemoveContainer" containerID="02f0145e3ce048dfee15cadd61327e317fcaac4830e06337ead313baa9248975"
May 21 16:36:22.964880 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:22.964858    2562 scope.go:117] "RemoveContainer" containerID="92ade71e706fc38f06632bd1ccfca540f25c4dda3cf088cad91d36786f291533"
May 21 16:36:22.972356 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:22.972334    2562 scope.go:117] "RemoveContainer" containerID="eaff75812ff5a71ef11eebe6ba844cacaf4f717061c144d5787597fdb8e80383"
May 21 16:36:22.979917 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:22.979896    2562 scope.go:117] "RemoveContainer" containerID="3f999a716cb0c6e695de00a36df5dd5aab974ca2d820dc06029b3afb7e238195"
May 21 16:36:22.990782 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:22.990762    2562 scope.go:117] "RemoveContainer" containerID="6088922f1fcc3e0014f90946c4d173df5c40359535da0801d8f1205fa662afec"
May 21 16:36:22.998390 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:22.998359    2562 scope.go:117] "RemoveContainer" containerID="fc11d8035d22420420ee23aed08a7081c40b0331c4d87b06b52bdd66b988d782"
May 21 16:36:23.005108 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.005043    2562 scope.go:117] "RemoveContainer" containerID="a29fdffd724e34b7895febe0318b760643995d7b071f80406d3bb083ab669915"
May 21 16:36:23.005405 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:36:23.005385    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a29fdffd724e34b7895febe0318b760643995d7b071f80406d3bb083ab669915\": container with ID starting with a29fdffd724e34b7895febe0318b760643995d7b071f80406d3bb083ab669915 not found: ID does not exist" containerID="a29fdffd724e34b7895febe0318b760643995d7b071f80406d3bb083ab669915"
May 21 16:36:23.005479 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.005418    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a29fdffd724e34b7895febe0318b760643995d7b071f80406d3bb083ab669915"} err="failed to get container status \"a29fdffd724e34b7895febe0318b760643995d7b071f80406d3bb083ab669915\": rpc error: code = NotFound desc = could not find container \"a29fdffd724e34b7895febe0318b760643995d7b071f80406d3bb083ab669915\": container with ID starting with a29fdffd724e34b7895febe0318b760643995d7b071f80406d3bb083ab669915 not found: ID does not exist"
May 21 16:36:23.005479 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.005441    2562 scope.go:117] "RemoveContainer" containerID="02f0145e3ce048dfee15cadd61327e317fcaac4830e06337ead313baa9248975"
May 21 16:36:23.005719 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:36:23.005700    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02f0145e3ce048dfee15cadd61327e317fcaac4830e06337ead313baa9248975\": container with ID starting with 02f0145e3ce048dfee15cadd61327e317fcaac4830e06337ead313baa9248975 not found: ID does not exist" containerID="02f0145e3ce048dfee15cadd61327e317fcaac4830e06337ead313baa9248975"
May 21 16:36:23.005767 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.005726    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02f0145e3ce048dfee15cadd61327e317fcaac4830e06337ead313baa9248975"} err="failed to get container status \"02f0145e3ce048dfee15cadd61327e317fcaac4830e06337ead313baa9248975\": rpc error: code = NotFound desc = could not find container \"02f0145e3ce048dfee15cadd61327e317fcaac4830e06337ead313baa9248975\": container with ID starting with 02f0145e3ce048dfee15cadd61327e317fcaac4830e06337ead313baa9248975 not found: ID does not exist"
May 21 16:36:23.005767 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.005743    2562 scope.go:117] "RemoveContainer" containerID="92ade71e706fc38f06632bd1ccfca540f25c4dda3cf088cad91d36786f291533"
May 21 16:36:23.005964 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:36:23.005945    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92ade71e706fc38f06632bd1ccfca540f25c4dda3cf088cad91d36786f291533\": container with ID starting with 92ade71e706fc38f06632bd1ccfca540f25c4dda3cf088cad91d36786f291533 not found: ID does not exist" containerID="92ade71e706fc38f06632bd1ccfca540f25c4dda3cf088cad91d36786f291533"
May 21 16:36:23.006029 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.005974    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92ade71e706fc38f06632bd1ccfca540f25c4dda3cf088cad91d36786f291533"} err="failed to get container status \"92ade71e706fc38f06632bd1ccfca540f25c4dda3cf088cad91d36786f291533\": rpc error: code = NotFound desc = could not find container \"92ade71e706fc38f06632bd1ccfca540f25c4dda3cf088cad91d36786f291533\": container with ID starting with 92ade71e706fc38f06632bd1ccfca540f25c4dda3cf088cad91d36786f291533 not found: ID does not exist"
May 21 16:36:23.006029 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.005998    2562 scope.go:117] "RemoveContainer" containerID="eaff75812ff5a71ef11eebe6ba844cacaf4f717061c144d5787597fdb8e80383"
May 21 16:36:23.006251 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:36:23.006237    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaff75812ff5a71ef11eebe6ba844cacaf4f717061c144d5787597fdb8e80383\": container with ID starting with eaff75812ff5a71ef11eebe6ba844cacaf4f717061c144d5787597fdb8e80383 not found: ID does not exist" containerID="eaff75812ff5a71ef11eebe6ba844cacaf4f717061c144d5787597fdb8e80383"
May 21 16:36:23.006297 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.006255    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaff75812ff5a71ef11eebe6ba844cacaf4f717061c144d5787597fdb8e80383"} err="failed to get container status \"eaff75812ff5a71ef11eebe6ba844cacaf4f717061c144d5787597fdb8e80383\": rpc error: code = NotFound desc = could not find container \"eaff75812ff5a71ef11eebe6ba844cacaf4f717061c144d5787597fdb8e80383\": container with ID starting with eaff75812ff5a71ef11eebe6ba844cacaf4f717061c144d5787597fdb8e80383 not found: ID does not exist"
May 21 16:36:23.006297 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.006267    2562 scope.go:117] "RemoveContainer" containerID="3f999a716cb0c6e695de00a36df5dd5aab974ca2d820dc06029b3afb7e238195"
May 21 16:36:23.006480 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:36:23.006461    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f999a716cb0c6e695de00a36df5dd5aab974ca2d820dc06029b3afb7e238195\": container with ID starting with 3f999a716cb0c6e695de00a36df5dd5aab974ca2d820dc06029b3afb7e238195 not found: ID does not exist" containerID="3f999a716cb0c6e695de00a36df5dd5aab974ca2d820dc06029b3afb7e238195"
May 21 16:36:23.006545 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.006488    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f999a716cb0c6e695de00a36df5dd5aab974ca2d820dc06029b3afb7e238195"} err="failed to get container status \"3f999a716cb0c6e695de00a36df5dd5aab974ca2d820dc06029b3afb7e238195\": rpc error: code = NotFound desc = could not find container \"3f999a716cb0c6e695de00a36df5dd5aab974ca2d820dc06029b3afb7e238195\": container with ID starting with 3f999a716cb0c6e695de00a36df5dd5aab974ca2d820dc06029b3afb7e238195 not found: ID does not exist"
May 21 16:36:23.006545 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.006511    2562 scope.go:117] "RemoveContainer" containerID="6088922f1fcc3e0014f90946c4d173df5c40359535da0801d8f1205fa662afec"
May 21 16:36:23.006749 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:36:23.006735    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6088922f1fcc3e0014f90946c4d173df5c40359535da0801d8f1205fa662afec\": container with ID starting with 6088922f1fcc3e0014f90946c4d173df5c40359535da0801d8f1205fa662afec not found: ID does not exist" containerID="6088922f1fcc3e0014f90946c4d173df5c40359535da0801d8f1205fa662afec"
May 21 16:36:23.006789 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.006751    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6088922f1fcc3e0014f90946c4d173df5c40359535da0801d8f1205fa662afec"} err="failed to get container status \"6088922f1fcc3e0014f90946c4d173df5c40359535da0801d8f1205fa662afec\": rpc error: code = NotFound desc = could not find container \"6088922f1fcc3e0014f90946c4d173df5c40359535da0801d8f1205fa662afec\": container with ID starting with 6088922f1fcc3e0014f90946c4d173df5c40359535da0801d8f1205fa662afec not found: ID does not exist"
May 21 16:36:23.006789 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.006762    2562 scope.go:117] "RemoveContainer" containerID="fc11d8035d22420420ee23aed08a7081c40b0331c4d87b06b52bdd66b988d782"
May 21 16:36:23.006942 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:36:23.006924    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc11d8035d22420420ee23aed08a7081c40b0331c4d87b06b52bdd66b988d782\": container with ID starting with fc11d8035d22420420ee23aed08a7081c40b0331c4d87b06b52bdd66b988d782 not found: ID does not exist" containerID="fc11d8035d22420420ee23aed08a7081c40b0331c4d87b06b52bdd66b988d782"
May 21 16:36:23.006987 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.006945    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc11d8035d22420420ee23aed08a7081c40b0331c4d87b06b52bdd66b988d782"} err="failed to get container status \"fc11d8035d22420420ee23aed08a7081c40b0331c4d87b06b52bdd66b988d782\": rpc error: code = NotFound desc = could not find container \"fc11d8035d22420420ee23aed08a7081c40b0331c4d87b06b52bdd66b988d782\": container with ID starting with fc11d8035d22420420ee23aed08a7081c40b0331c4d87b06b52bdd66b988d782 not found: ID does not exist"
May 21 16:36:23.027302 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027268    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.027302 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027308    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-config\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.027537 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027325    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-thanos-prometheus-http-client-file\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.027537 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027364    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-grpc-tls\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.027537 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027456    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-kubelet-serving-ca-bundle\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.027537 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027507    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-metrics-client-certs\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.027743 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027540    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-k8s-db\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.027743 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027567    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d443dcd-5f3d-4277-80fe-57ca419c7201-tls-assets\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.027743 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027612    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-kube-rbac-proxy\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.027743 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027636    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-k8s-rulefiles-0\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.027743 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027670    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-serving-certs-ca-bundle\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.027743 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027714    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d443dcd-5f3d-4277-80fe-57ca419c7201-config-out\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.028032 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027745    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-metrics-client-ca\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.028032 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027772    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-web-config\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.028032 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027811    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.028032 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027844    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-tls\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.028032 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027869    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bdgr\" (UniqueName: \"kubernetes.io/projected/2d443dcd-5f3d-4277-80fe-57ca419c7201-kube-api-access-8bdgr\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.028032 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027904    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-trusted-ca-bundle\") pod \"2d443dcd-5f3d-4277-80fe-57ca419c7201\" (UID: \"2d443dcd-5f3d-4277-80fe-57ca419c7201\") "
May 21 16:36:23.028032 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.027900    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 21 16:36:23.028405 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.028358    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.028581 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.028488    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 21 16:36:23.028878 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.028747    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 21 16:36:23.028878 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.028802    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 21 16:36:23.030101 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.030071    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 21 16:36:23.031315 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.031252    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 16:36:23.031497 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.031474    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 16:36:23.031576 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.031539    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d443dcd-5f3d-4277-80fe-57ca419c7201-config-out" (OuterVolumeSpecName: "config-out") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 21 16:36:23.031720 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.031690    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 16:36:23.031806 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.031765    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 21 16:36:23.032041 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.031995    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 16:36:23.032279 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.032247    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 16:36:23.032823 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.032791    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 16:36:23.032920 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.032900    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-config" (OuterVolumeSpecName: "config") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 16:36:23.033207 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.033182    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d443dcd-5f3d-4277-80fe-57ca419c7201-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 16:36:23.033860 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.033832    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d443dcd-5f3d-4277-80fe-57ca419c7201-kube-api-access-8bdgr" (OuterVolumeSpecName: "kube-api-access-8bdgr") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "kube-api-access-8bdgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 16:36:23.033860 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.033833    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 16:36:23.044492 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.044466    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-web-config" (OuterVolumeSpecName: "web-config") pod "2d443dcd-5f3d-4277-80fe-57ca419c7201" (UID: "2d443dcd-5f3d-4277-80fe-57ca419c7201"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 16:36:23.129121 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129080    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d443dcd-5f3d-4277-80fe-57ca419c7201-config-out\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129121 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129113    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-metrics-client-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129121 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129123    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-web-config\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129133    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129143    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129151    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bdgr\" (UniqueName: \"kubernetes.io/projected/2d443dcd-5f3d-4277-80fe-57ca419c7201-kube-api-access-8bdgr\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129160    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129169    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129178    2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-config\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129187    2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129196    2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-grpc-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129204    2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-metrics-client-certs\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129212    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-k8s-db\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129220    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d443dcd-5f3d-4277-80fe-57ca419c7201-tls-assets\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129229    2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d443dcd-5f3d-4277-80fe-57ca419c7201-secret-kube-rbac-proxy\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129244    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.129357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.129253    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d443dcd-5f3d-4277-80fe-57ca419c7201-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:36:23.269898 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.269859    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 21 16:36:23.273854 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.273822    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 21 16:36:23.297432 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297400    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 21 16:36:23.297706 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297694    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="kube-rbac-proxy"
May 21 16:36:23.297764 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297708    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="kube-rbac-proxy"
May 21 16:36:23.297764 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297718    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="kube-rbac-proxy-thanos"
May 21 16:36:23.297764 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297723    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="kube-rbac-proxy-thanos"
May 21 16:36:23.297764 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297735    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="kube-rbac-proxy-web"
May 21 16:36:23.297764 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297740    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="kube-rbac-proxy-web"
May 21 16:36:23.297764 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297747    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="thanos-sidecar"
May 21 16:36:23.297764 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297751    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="thanos-sidecar"
May 21 16:36:23.297764 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297758    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" containerName="registry"
May 21 16:36:23.297764 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297763    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" containerName="registry"
May 21 16:36:23.298021 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297769    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="prometheus"
May 21 16:36:23.298021 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297774    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="prometheus"
May 21 16:36:23.298021 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297780    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="config-reloader"
May 21 16:36:23.298021 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297784    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="config-reloader"
May 21 16:36:23.298021 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297792    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="init-config-reloader"
May 21 16:36:23.298021 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297797    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="init-config-reloader"
May 21 16:36:23.298021 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297837    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="prometheus"
May 21 16:36:23.298021 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297844    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="kube-rbac-proxy"
May 21 16:36:23.298021 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297851    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="config-reloader"
May 21 16:36:23.298021 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297856    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="thanos-sidecar"
May 21 16:36:23.298021 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297862    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="kube-rbac-proxy-thanos"
May 21 16:36:23.298021 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297867    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="d98d5bed-f2ad-4cf8-b975-8bd08f64bbf5" containerName="registry"
May 21 16:36:23.298021 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.297874    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" containerName="kube-rbac-proxy-web"
May 21 16:36:23.302926 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.302905    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.305684 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.305661    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 21 16:36:23.305805 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.305755    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 21 16:36:23.305909 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.305889    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 21 16:36:23.305971 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.305959    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 21 16:36:23.306082 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.306030    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nwptl\""
May 21 16:36:23.306156 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.306085    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 21 16:36:23.306216 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.306198    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 21 16:36:23.306293 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.306269    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 21 16:36:23.306436 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.306420    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 21 16:36:23.306508 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.306492    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 21 16:36:23.306596 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.306581    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 21 16:36:23.306646 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.306586    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 21 16:36:23.306704 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.306691    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2bl5725n64ikv\""
May 21 16:36:23.309876 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.309856    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 21 16:36:23.311731 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.311710    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 21 16:36:23.314629 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.314607    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 21 16:36:23.432252 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432215    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432429 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432429 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432287    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7ff357c-5f9a-41fa-9c30-a71accf31969-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432429 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432319    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432429 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432338    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-config\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432429 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432356    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432589 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432417    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432589 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432589 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432509    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432589 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432528    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432589 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432568    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432589 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432584    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432763 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7ff357c-5f9a-41fa-9c30-a71accf31969-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432763 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432632    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432763 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7nn6\" (UniqueName: \"kubernetes.io/projected/d7ff357c-5f9a-41fa-9c30-a71accf31969-kube-api-access-x7nn6\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432763 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432678    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-web-config\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432763 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7ff357c-5f9a-41fa-9c30-a71accf31969-config-out\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.432763 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.432726    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.533334 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.533301    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7ff357c-5f9a-41fa-9c30-a71accf31969-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.533456 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.533339    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.533456 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.533357    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-config\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.533527 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.533472    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.533527 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.533503    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.533595 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.533535    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.533595 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.533565    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.533802 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.533769    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7ff357c-5f9a-41fa-9c30-a71accf31969-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.533957 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.533832    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.533957 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.533885    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.533957 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.533915    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.533957 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.533938    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7ff357c-5f9a-41fa-9c30-a71accf31969-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.534337 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.533982    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.534337 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.534021    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x7nn6\" (UniqueName: \"kubernetes.io/projected/d7ff357c-5f9a-41fa-9c30-a71accf31969-kube-api-access-x7nn6\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.534337 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.534093    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-web-config\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.534337 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.534129    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7ff357c-5f9a-41fa-9c30-a71accf31969-config-out\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.534337 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.534205    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.534337 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.534356    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.534663 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.534407    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.534663 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.534539    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.535329 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.535265    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.536839 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.536809    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.536931 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.536818    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.536999 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.536927    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.536999 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.536932    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-config\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.537168 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.537147    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7ff357c-5f9a-41fa-9c30-a71accf31969-config-out\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.537358 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.537334    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.537591 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.537562    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.537813 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.537773    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.537919 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.537901    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.538008 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.537989    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.539224 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.539206    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-web-config\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.539895 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.539875    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7ff357c-5f9a-41fa-9c30-a71accf31969-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.540063 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.540026    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7ff357c-5f9a-41fa-9c30-a71accf31969-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.540115 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.540085    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7ff357c-5f9a-41fa-9c30-a71accf31969-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.545738 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.545717    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7nn6\" (UniqueName: \"kubernetes.io/projected/d7ff357c-5f9a-41fa-9c30-a71accf31969-kube-api-access-x7nn6\") pod \"prometheus-k8s-0\" (UID: \"d7ff357c-5f9a-41fa-9c30-a71accf31969\") " pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.614398 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.614353    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:23.753861 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.753834    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 21 16:36:23.756282 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:36:23.756252    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ff357c_5f9a_41fa_9c30_a71accf31969.slice/crio-c3e8c1aa5beeebdb92a9683e0d012d8518659a63d887f95c416386397a68205e WatchSource:0}: Error finding container c3e8c1aa5beeebdb92a9683e0d012d8518659a63d887f95c416386397a68205e: Status 404 returned error can't find the container with id c3e8c1aa5beeebdb92a9683e0d012d8518659a63d887f95c416386397a68205e
May 21 16:36:23.950813 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.950778    2562 generic.go:358] "Generic (PLEG): container finished" podID="d7ff357c-5f9a-41fa-9c30-a71accf31969" containerID="fa98a49e7c5b848cc8c48f539bd188968701635cb5c0f361ee0f397f6b260c4e" exitCode=0
May 21 16:36:23.951211 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.950869    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7ff357c-5f9a-41fa-9c30-a71accf31969","Type":"ContainerDied","Data":"fa98a49e7c5b848cc8c48f539bd188968701635cb5c0f361ee0f397f6b260c4e"}
May 21 16:36:23.951211 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:23.950908    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7ff357c-5f9a-41fa-9c30-a71accf31969","Type":"ContainerStarted","Data":"c3e8c1aa5beeebdb92a9683e0d012d8518659a63d887f95c416386397a68205e"}
May 21 16:36:24.110263 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:24.110215    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d443dcd-5f3d-4277-80fe-57ca419c7201" path="/var/lib/kubelet/pods/2d443dcd-5f3d-4277-80fe-57ca419c7201/volumes"
May 21 16:36:24.957442 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:24.957401    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7ff357c-5f9a-41fa-9c30-a71accf31969","Type":"ContainerStarted","Data":"1fb10511182baf55b8df1c2f6e2c002292d62a362ab407f70e5bb932b788c9d7"}
May 21 16:36:24.957442 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:24.957446    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7ff357c-5f9a-41fa-9c30-a71accf31969","Type":"ContainerStarted","Data":"fab8bd3e8bef0679a9ab6f66231e948e437bc283e7ed66bd884500b903b9443f"}
May 21 16:36:24.957832 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:24.957461    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7ff357c-5f9a-41fa-9c30-a71accf31969","Type":"ContainerStarted","Data":"482e106b1d1492d10ce86164b3cbd6be44fa04b517240f350cd6911f51f0fd96"}
May 21 16:36:24.957832 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:24.957474    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7ff357c-5f9a-41fa-9c30-a71accf31969","Type":"ContainerStarted","Data":"2069c42a56d171e9b070d37356b5dea6b19cee5cd021abc32c5ea69539b03d19"}
May 21 16:36:24.957832 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:24.957486    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7ff357c-5f9a-41fa-9c30-a71accf31969","Type":"ContainerStarted","Data":"3f002e214b766cd69d3bd0748160a93ed0e34ff8299966a603ec3e75e4b37275"}
May 21 16:36:24.957832 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:24.957498    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7ff357c-5f9a-41fa-9c30-a71accf31969","Type":"ContainerStarted","Data":"7525cfec5a82dc7156ebffd418d4f4d663472bef5a01794d55a1e02529f06dc4"}
May 21 16:36:24.986440 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:24.986392    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.986374367 podStartE2EDuration="1.986374367s" podCreationTimestamp="2026-05-21 16:36:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 16:36:24.984225992 +0000 UTC m=+275.455346990" watchObservedRunningTime="2026-05-21 16:36:24.986374367 +0000 UTC m=+275.457495348"
May 21 16:36:28.615374 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:28.615336    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:36:50.015584 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:36:50.015555    2562 kubelet.go:1628] "Image garbage collection succeeded"
May 21 16:37:23.614701 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:37:23.614645    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:37:23.630924 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:37:23.630894    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:37:24.147167 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:37:24.147137    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 21 16:38:10.253144 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.253100    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"]
May 21 16:38:10.256139 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.256121    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"
May 21 16:38:10.258979 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.258956    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
May 21 16:38:10.259117 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.258955    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
May 21 16:38:10.259117 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.259033    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wnrl7\""
May 21 16:38:10.259250 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.259171    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
May 21 16:38:10.259403 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.259388    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
May 21 16:38:10.272498 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.272475    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"]
May 21 16:38:10.453152 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.453105    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/31a41f19-a348-4bff-b005-0b0de9bac330-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb\" (UID: \"31a41f19-a348-4bff-b005-0b0de9bac330\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"
May 21 16:38:10.453350 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.453176    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgltc\" (UniqueName: \"kubernetes.io/projected/31a41f19-a348-4bff-b005-0b0de9bac330-kube-api-access-cgltc\") pod \"opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb\" (UID: \"31a41f19-a348-4bff-b005-0b0de9bac330\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"
May 21 16:38:10.453350 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.453279    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31a41f19-a348-4bff-b005-0b0de9bac330-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb\" (UID: \"31a41f19-a348-4bff-b005-0b0de9bac330\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"
May 21 16:38:10.554181 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.554138    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31a41f19-a348-4bff-b005-0b0de9bac330-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb\" (UID: \"31a41f19-a348-4bff-b005-0b0de9bac330\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"
May 21 16:38:10.554181 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.554190    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/31a41f19-a348-4bff-b005-0b0de9bac330-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb\" (UID: \"31a41f19-a348-4bff-b005-0b0de9bac330\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"
May 21 16:38:10.554456 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.554213    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cgltc\" (UniqueName: \"kubernetes.io/projected/31a41f19-a348-4bff-b005-0b0de9bac330-kube-api-access-cgltc\") pod \"opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb\" (UID: \"31a41f19-a348-4bff-b005-0b0de9bac330\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"
May 21 16:38:10.556769 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.556738    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/31a41f19-a348-4bff-b005-0b0de9bac330-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb\" (UID: \"31a41f19-a348-4bff-b005-0b0de9bac330\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"
May 21 16:38:10.556889 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.556802    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31a41f19-a348-4bff-b005-0b0de9bac330-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb\" (UID: \"31a41f19-a348-4bff-b005-0b0de9bac330\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"
May 21 16:38:10.562436 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.562413    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgltc\" (UniqueName: \"kubernetes.io/projected/31a41f19-a348-4bff-b005-0b0de9bac330-kube-api-access-cgltc\") pod \"opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb\" (UID: \"31a41f19-a348-4bff-b005-0b0de9bac330\") " pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"
May 21 16:38:10.566175 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.566155    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"
May 21 16:38:10.699373 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.699340    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"]
May 21 16:38:10.703126 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:38:10.703097    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31a41f19_a348_4bff_b005_0b0de9bac330.slice/crio-c1ec40e747de2a84b834d15745c1ca3a7efcd889593f5ed171efd5739415cd9b WatchSource:0}: Error finding container c1ec40e747de2a84b834d15745c1ca3a7efcd889593f5ed171efd5739415cd9b: Status 404 returned error can't find the container with id c1ec40e747de2a84b834d15745c1ca3a7efcd889593f5ed171efd5739415cd9b
May 21 16:38:10.705083 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:10.705067    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 21 16:38:11.266138 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:11.266098    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb" event={"ID":"31a41f19-a348-4bff-b005-0b0de9bac330","Type":"ContainerStarted","Data":"c1ec40e747de2a84b834d15745c1ca3a7efcd889593f5ed171efd5739415cd9b"}
May 21 16:38:14.276966 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:14.276934    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb" event={"ID":"31a41f19-a348-4bff-b005-0b0de9bac330","Type":"ContainerStarted","Data":"872b35e5b446b25905a6b748531d6245f0007763af927b276ad2f2d908767bed"}
May 21 16:38:14.277363 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:14.277073    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"
May 21 16:38:14.297928 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:14.297879    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb" podStartSLOduration=1.6656753420000001 podStartE2EDuration="4.297865306s" podCreationTimestamp="2026-05-21 16:38:10 +0000 UTC" firstStartedPulling="2026-05-21 16:38:10.705188289 +0000 UTC m=+381.176309247" lastFinishedPulling="2026-05-21 16:38:13.337378246 +0000 UTC m=+383.808499211" observedRunningTime="2026-05-21 16:38:14.295391946 +0000 UTC m=+384.766513123" watchObservedRunningTime="2026-05-21 16:38:14.297865306 +0000 UTC m=+384.768986285"
May 21 16:38:25.282820 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:25.282789    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb"
May 21 16:38:28.164633 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.164598    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"]
May 21 16:38:28.167993 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.167971    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"
May 21 16:38:28.170568 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.170545    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
May 21 16:38:28.170697 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.170545    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
May 21 16:38:28.170697 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.170611    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8z9q4\""
May 21 16:38:28.177649 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.177625    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"]
May 21 16:38:28.200347 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.200318    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24be0444-c553-410d-89b6-731303d5f2fc-tls-certs\") pod \"kube-auth-proxy-5df99877bf-pqrbw\" (UID: \"24be0444-c553-410d-89b6-731303d5f2fc\") " pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"
May 21 16:38:28.200347 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.200352    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24be0444-c553-410d-89b6-731303d5f2fc-tmp\") pod \"kube-auth-proxy-5df99877bf-pqrbw\" (UID: \"24be0444-c553-410d-89b6-731303d5f2fc\") " pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"
May 21 16:38:28.200532 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.200380    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dglnk\" (UniqueName: \"kubernetes.io/projected/24be0444-c553-410d-89b6-731303d5f2fc-kube-api-access-dglnk\") pod \"kube-auth-proxy-5df99877bf-pqrbw\" (UID: \"24be0444-c553-410d-89b6-731303d5f2fc\") " pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"
May 21 16:38:28.301178 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.301142    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dglnk\" (UniqueName: \"kubernetes.io/projected/24be0444-c553-410d-89b6-731303d5f2fc-kube-api-access-dglnk\") pod \"kube-auth-proxy-5df99877bf-pqrbw\" (UID: \"24be0444-c553-410d-89b6-731303d5f2fc\") " pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"
May 21 16:38:28.301362 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.301213    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24be0444-c553-410d-89b6-731303d5f2fc-tls-certs\") pod \"kube-auth-proxy-5df99877bf-pqrbw\" (UID: \"24be0444-c553-410d-89b6-731303d5f2fc\") " pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"
May 21 16:38:28.301362 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.301234    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24be0444-c553-410d-89b6-731303d5f2fc-tmp\") pod \"kube-auth-proxy-5df99877bf-pqrbw\" (UID: \"24be0444-c553-410d-89b6-731303d5f2fc\") " pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"
May 21 16:38:28.303702 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.303673    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24be0444-c553-410d-89b6-731303d5f2fc-tmp\") pod \"kube-auth-proxy-5df99877bf-pqrbw\" (UID: \"24be0444-c553-410d-89b6-731303d5f2fc\") " pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"
May 21 16:38:28.303905 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.303887    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24be0444-c553-410d-89b6-731303d5f2fc-tls-certs\") pod \"kube-auth-proxy-5df99877bf-pqrbw\" (UID: \"24be0444-c553-410d-89b6-731303d5f2fc\") " pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"
May 21 16:38:28.309454 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.309434    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dglnk\" (UniqueName: \"kubernetes.io/projected/24be0444-c553-410d-89b6-731303d5f2fc-kube-api-access-dglnk\") pod \"kube-auth-proxy-5df99877bf-pqrbw\" (UID: \"24be0444-c553-410d-89b6-731303d5f2fc\") " pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"
May 21 16:38:28.478751 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.478655    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"
May 21 16:38:28.608941 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:28.608911    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw"]
May 21 16:38:28.611832 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:38:28.611800    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24be0444_c553_410d_89b6_731303d5f2fc.slice/crio-61edd8226f6392b4c2745ad831c0a1bd253e62b92e1a63343483c5b810752438 WatchSource:0}: Error finding container 61edd8226f6392b4c2745ad831c0a1bd253e62b92e1a63343483c5b810752438: Status 404 returned error can't find the container with id 61edd8226f6392b4c2745ad831c0a1bd253e62b92e1a63343483c5b810752438
May 21 16:38:29.321944 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:29.321900    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw" event={"ID":"24be0444-c553-410d-89b6-731303d5f2fc","Type":"ContainerStarted","Data":"61edd8226f6392b4c2745ad831c0a1bd253e62b92e1a63343483c5b810752438"}
May 21 16:38:29.954627 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:29.954586    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"]
May 21 16:38:29.962759 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:29.962689    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:29.966013 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:29.965970    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"]
May 21 16:38:29.966762 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:29.966602    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-gglb6\""
May 21 16:38:29.966762 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:29.966649    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
May 21 16:38:29.966762 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:29.966609    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
May 21 16:38:29.966762 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:29.966716    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
May 21 16:38:29.967123 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:29.966924    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
May 21 16:38:29.967123 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:29.966967    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
May 21 16:38:30.017075 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.017018    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0578f9da-02bf-4346-bdf9-69a1284f912f-cert\") pod \"lws-controller-manager-67b4fd8c8c-ld9cp\" (UID: \"0578f9da-02bf-4346-bdf9-69a1284f912f\") " pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:30.017274 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.017224    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0578f9da-02bf-4346-bdf9-69a1284f912f-manager-config\") pod \"lws-controller-manager-67b4fd8c8c-ld9cp\" (UID: \"0578f9da-02bf-4346-bdf9-69a1284f912f\") " pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:30.017338 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.017281    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcls8\" (UniqueName: \"kubernetes.io/projected/0578f9da-02bf-4346-bdf9-69a1284f912f-kube-api-access-tcls8\") pod \"lws-controller-manager-67b4fd8c8c-ld9cp\" (UID: \"0578f9da-02bf-4346-bdf9-69a1284f912f\") " pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:30.017338 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.017319    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0578f9da-02bf-4346-bdf9-69a1284f912f-metrics-cert\") pod \"lws-controller-manager-67b4fd8c8c-ld9cp\" (UID: \"0578f9da-02bf-4346-bdf9-69a1284f912f\") " pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:30.117895 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.117785    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0578f9da-02bf-4346-bdf9-69a1284f912f-cert\") pod \"lws-controller-manager-67b4fd8c8c-ld9cp\" (UID: \"0578f9da-02bf-4346-bdf9-69a1284f912f\") " pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:30.117895 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.117878    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0578f9da-02bf-4346-bdf9-69a1284f912f-manager-config\") pod \"lws-controller-manager-67b4fd8c8c-ld9cp\" (UID: \"0578f9da-02bf-4346-bdf9-69a1284f912f\") " pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:30.118242 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.117913    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tcls8\" (UniqueName: \"kubernetes.io/projected/0578f9da-02bf-4346-bdf9-69a1284f912f-kube-api-access-tcls8\") pod \"lws-controller-manager-67b4fd8c8c-ld9cp\" (UID: \"0578f9da-02bf-4346-bdf9-69a1284f912f\") " pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:30.118242 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.117940    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0578f9da-02bf-4346-bdf9-69a1284f912f-metrics-cert\") pod \"lws-controller-manager-67b4fd8c8c-ld9cp\" (UID: \"0578f9da-02bf-4346-bdf9-69a1284f912f\") " pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:30.118847 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.118764    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0578f9da-02bf-4346-bdf9-69a1284f912f-manager-config\") pod \"lws-controller-manager-67b4fd8c8c-ld9cp\" (UID: \"0578f9da-02bf-4346-bdf9-69a1284f912f\") " pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:30.121641 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.121568    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0578f9da-02bf-4346-bdf9-69a1284f912f-metrics-cert\") pod \"lws-controller-manager-67b4fd8c8c-ld9cp\" (UID: \"0578f9da-02bf-4346-bdf9-69a1284f912f\") " pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:30.122997 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.122973    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0578f9da-02bf-4346-bdf9-69a1284f912f-cert\") pod \"lws-controller-manager-67b4fd8c8c-ld9cp\" (UID: \"0578f9da-02bf-4346-bdf9-69a1284f912f\") " pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:30.135440 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.135386    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcls8\" (UniqueName: \"kubernetes.io/projected/0578f9da-02bf-4346-bdf9-69a1284f912f-kube-api-access-tcls8\") pod \"lws-controller-manager-67b4fd8c8c-ld9cp\" (UID: \"0578f9da-02bf-4346-bdf9-69a1284f912f\") " pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:30.276322 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.276285    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:30.684332 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:30.684174    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"]
May 21 16:38:30.686909 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:38:30.686879    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0578f9da_02bf_4346_bdf9_69a1284f912f.slice/crio-df434decbe51aa4d4554b39e4689544498080777e4ef4d71859fbd6f8644be06 WatchSource:0}: Error finding container df434decbe51aa4d4554b39e4689544498080777e4ef4d71859fbd6f8644be06: Status 404 returned error can't find the container with id df434decbe51aa4d4554b39e4689544498080777e4ef4d71859fbd6f8644be06
May 21 16:38:31.329969 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:31.329932    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp" event={"ID":"0578f9da-02bf-4346-bdf9-69a1284f912f","Type":"ContainerStarted","Data":"df434decbe51aa4d4554b39e4689544498080777e4ef4d71859fbd6f8644be06"}
May 21 16:38:32.334619 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:32.334576    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw" event={"ID":"24be0444-c553-410d-89b6-731303d5f2fc","Type":"ContainerStarted","Data":"eb286077bec58aa3789092e6faf265602317d7b52677928d1d3e5b71576d0399"}
May 21 16:38:32.354714 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:32.354650    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5df99877bf-pqrbw" podStartSLOduration=1.5335228760000001 podStartE2EDuration="4.354634954s" podCreationTimestamp="2026-05-21 16:38:28 +0000 UTC" firstStartedPulling="2026-05-21 16:38:28.613599183 +0000 UTC m=+399.084720150" lastFinishedPulling="2026-05-21 16:38:31.434711271 +0000 UTC m=+401.905832228" observedRunningTime="2026-05-21 16:38:32.351940117 +0000 UTC m=+402.823061123" watchObservedRunningTime="2026-05-21 16:38:32.354634954 +0000 UTC m=+402.825755932"
May 21 16:38:33.339535 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:33.339493    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp" event={"ID":"0578f9da-02bf-4346-bdf9-69a1284f912f","Type":"ContainerStarted","Data":"d6b1d9e5019bc7fa7c94fc1aab27182294c657ac4c482639f6a59b626da6f99d"}
May 21 16:38:33.339950 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:33.339780    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:38:33.355123 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:33.354902    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp" podStartSLOduration=1.817588476 podStartE2EDuration="4.354888113s" podCreationTimestamp="2026-05-21 16:38:29 +0000 UTC" firstStartedPulling="2026-05-21 16:38:30.688816762 +0000 UTC m=+401.159937723" lastFinishedPulling="2026-05-21 16:38:33.226116401 +0000 UTC m=+403.697237360" observedRunningTime="2026-05-21 16:38:33.354829502 +0000 UTC m=+403.825950482" watchObservedRunningTime="2026-05-21 16:38:33.354888113 +0000 UTC m=+403.826009092"
May 21 16:38:44.345757 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:38:44.345721    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-67b4fd8c8c-ld9cp"
May 21 16:40:11.074335 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.074296    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"]
May 21 16:40:11.076355 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.076336    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"
May 21 16:40:11.080338 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.080310    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 21 16:40:11.081455 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.081436    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
May 21 16:40:11.081455 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.081449    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 21 16:40:11.081633 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.081478    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-l4pgl\""
May 21 16:40:11.081633 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.081536    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
May 21 16:40:11.091927 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.091899    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"]
May 21 16:40:11.183371 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.183332    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e80afd22-59f9-4646-bd74-ecf27ecd3b4d-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-z6jdq\" (UID: \"e80afd22-59f9-4646-bd74-ecf27ecd3b4d\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"
May 21 16:40:11.183371 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.183374    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e80afd22-59f9-4646-bd74-ecf27ecd3b4d-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-z6jdq\" (UID: \"e80afd22-59f9-4646-bd74-ecf27ecd3b4d\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"
May 21 16:40:11.183641 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.183415    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv9d2\" (UniqueName: \"kubernetes.io/projected/e80afd22-59f9-4646-bd74-ecf27ecd3b4d-kube-api-access-nv9d2\") pod \"kuadrant-console-plugin-6cb54b5c86-z6jdq\" (UID: \"e80afd22-59f9-4646-bd74-ecf27ecd3b4d\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"
May 21 16:40:11.284124 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.284088    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nv9d2\" (UniqueName: \"kubernetes.io/projected/e80afd22-59f9-4646-bd74-ecf27ecd3b4d-kube-api-access-nv9d2\") pod \"kuadrant-console-plugin-6cb54b5c86-z6jdq\" (UID: \"e80afd22-59f9-4646-bd74-ecf27ecd3b4d\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"
May 21 16:40:11.284313 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.284187    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e80afd22-59f9-4646-bd74-ecf27ecd3b4d-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-z6jdq\" (UID: \"e80afd22-59f9-4646-bd74-ecf27ecd3b4d\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"
May 21 16:40:11.284313 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.284212    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e80afd22-59f9-4646-bd74-ecf27ecd3b4d-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-z6jdq\" (UID: \"e80afd22-59f9-4646-bd74-ecf27ecd3b4d\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"
May 21 16:40:11.284313 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:40:11.284306    2562 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
May 21 16:40:11.284479 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:40:11.284379    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e80afd22-59f9-4646-bd74-ecf27ecd3b4d-plugin-serving-cert podName:e80afd22-59f9-4646-bd74-ecf27ecd3b4d nodeName:}" failed. No retries permitted until 2026-05-21 16:40:11.784358524 +0000 UTC m=+502.255479495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/e80afd22-59f9-4646-bd74-ecf27ecd3b4d-plugin-serving-cert") pod "kuadrant-console-plugin-6cb54b5c86-z6jdq" (UID: "e80afd22-59f9-4646-bd74-ecf27ecd3b4d") : secret "plugin-serving-cert" not found
May 21 16:40:11.285011 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.284988    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e80afd22-59f9-4646-bd74-ecf27ecd3b4d-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-z6jdq\" (UID: \"e80afd22-59f9-4646-bd74-ecf27ecd3b4d\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"
May 21 16:40:11.293300 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.293267    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv9d2\" (UniqueName: \"kubernetes.io/projected/e80afd22-59f9-4646-bd74-ecf27ecd3b4d-kube-api-access-nv9d2\") pod \"kuadrant-console-plugin-6cb54b5c86-z6jdq\" (UID: \"e80afd22-59f9-4646-bd74-ecf27ecd3b4d\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"
May 21 16:40:11.789239 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.789194    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e80afd22-59f9-4646-bd74-ecf27ecd3b4d-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-z6jdq\" (UID: \"e80afd22-59f9-4646-bd74-ecf27ecd3b4d\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"
May 21 16:40:11.791789 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.791768    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e80afd22-59f9-4646-bd74-ecf27ecd3b4d-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-z6jdq\" (UID: \"e80afd22-59f9-4646-bd74-ecf27ecd3b4d\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"
May 21 16:40:11.994395 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:11.994347    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"
May 21 16:40:12.125905 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:12.125871    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq"]
May 21 16:40:12.128496 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:40:12.128467    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode80afd22_59f9_4646_bd74_ecf27ecd3b4d.slice/crio-d316911108b17651e6e8ac4c2b646ec0e1cb723950c695c8314a42f0ba553179 WatchSource:0}: Error finding container d316911108b17651e6e8ac4c2b646ec0e1cb723950c695c8314a42f0ba553179: Status 404 returned error can't find the container with id d316911108b17651e6e8ac4c2b646ec0e1cb723950c695c8314a42f0ba553179
May 21 16:40:12.650385 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:12.650342    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq" event={"ID":"e80afd22-59f9-4646-bd74-ecf27ecd3b4d","Type":"ContainerStarted","Data":"d316911108b17651e6e8ac4c2b646ec0e1cb723950c695c8314a42f0ba553179"}
May 21 16:40:36.742432 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:36.742396    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq" event={"ID":"e80afd22-59f9-4646-bd74-ecf27ecd3b4d","Type":"ContainerStarted","Data":"83f9beacdea979261ce9d79675eb158bb4ccaf9ea3206ef26c43a6ef6abdfa4a"}
May 21 16:40:36.757713 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:36.757655    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z6jdq" podStartSLOduration=1.6395670340000001 podStartE2EDuration="25.757640768s" podCreationTimestamp="2026-05-21 16:40:11 +0000 UTC" firstStartedPulling="2026-05-21 16:40:12.129870766 +0000 UTC m=+502.600991736" lastFinishedPulling="2026-05-21 16:40:36.247944511 +0000 UTC m=+526.719065470" observedRunningTime="2026-05-21 16:40:36.756375227 +0000 UTC m=+527.227496205" watchObservedRunningTime="2026-05-21 16:40:36.757640768 +0000 UTC m=+527.228761747"
May 21 16:40:54.510349 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:54.510305    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-rrczd"]
May 21 16:40:54.520772 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:54.520746    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd"
May 21 16:40:54.523400 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:54.523367    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
May 21 16:40:54.524376 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:54.524353    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-rrczd"]
May 21 16:40:54.562686 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:54.562654    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55304f34-8676-465a-8ef9-f98d0b71645a-config-file\") pod \"limitador-limitador-7d549b5b-rrczd\" (UID: \"55304f34-8676-465a-8ef9-f98d0b71645a\") " pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd"
May 21 16:40:54.562867 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:54.562710    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmk7v\" (UniqueName: \"kubernetes.io/projected/55304f34-8676-465a-8ef9-f98d0b71645a-kube-api-access-fmk7v\") pod \"limitador-limitador-7d549b5b-rrczd\" (UID: \"55304f34-8676-465a-8ef9-f98d0b71645a\") " pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd"
May 21 16:40:54.603355 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:54.603318    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-rrczd"]
May 21 16:40:54.663398 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:54.663357    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55304f34-8676-465a-8ef9-f98d0b71645a-config-file\") pod \"limitador-limitador-7d549b5b-rrczd\" (UID: \"55304f34-8676-465a-8ef9-f98d0b71645a\") " pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd"
May 21 16:40:54.663619 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:54.663420    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fmk7v\" (UniqueName: \"kubernetes.io/projected/55304f34-8676-465a-8ef9-f98d0b71645a-kube-api-access-fmk7v\") pod \"limitador-limitador-7d549b5b-rrczd\" (UID: \"55304f34-8676-465a-8ef9-f98d0b71645a\") " pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd"
May 21 16:40:54.664044 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:54.664025    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55304f34-8676-465a-8ef9-f98d0b71645a-config-file\") pod \"limitador-limitador-7d549b5b-rrczd\" (UID: \"55304f34-8676-465a-8ef9-f98d0b71645a\") " pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd"
May 21 16:40:54.671224 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:54.671201    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmk7v\" (UniqueName: \"kubernetes.io/projected/55304f34-8676-465a-8ef9-f98d0b71645a-kube-api-access-fmk7v\") pod \"limitador-limitador-7d549b5b-rrczd\" (UID: \"55304f34-8676-465a-8ef9-f98d0b71645a\") " pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd"
May 21 16:40:54.831562 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:54.831534    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd"
May 21 16:40:54.958285 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:54.958256    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-rrczd"]
May 21 16:40:54.960617 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:40:54.960582    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55304f34_8676_465a_8ef9_f98d0b71645a.slice/crio-d08c8f4a6a4584b0c1e36a490595206ab0d16cd67be8ace258d94cb93e124990 WatchSource:0}: Error finding container d08c8f4a6a4584b0c1e36a490595206ab0d16cd67be8ace258d94cb93e124990: Status 404 returned error can't find the container with id d08c8f4a6a4584b0c1e36a490595206ab0d16cd67be8ace258d94cb93e124990
May 21 16:40:55.306822 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:55.306783    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-m2fc5"]
May 21 16:40:55.311777 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:55.311753    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-m2fc5"
May 21 16:40:55.314425 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:55.314334    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-g9xmg\""
May 21 16:40:55.316004 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:55.315982    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-m2fc5"]
May 21 16:40:55.370392 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:55.370357    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbw4c\" (UniqueName: \"kubernetes.io/projected/ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4-kube-api-access-dbw4c\") pod \"authorino-f99f4b5cd-m2fc5\" (UID: \"ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4\") " pod="kuadrant-system/authorino-f99f4b5cd-m2fc5"
May 21 16:40:55.470879 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:55.470840    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dbw4c\" (UniqueName: \"kubernetes.io/projected/ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4-kube-api-access-dbw4c\") pod \"authorino-f99f4b5cd-m2fc5\" (UID: \"ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4\") " pod="kuadrant-system/authorino-f99f4b5cd-m2fc5"
May 21 16:40:55.479379 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:55.479346    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbw4c\" (UniqueName: \"kubernetes.io/projected/ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4-kube-api-access-dbw4c\") pod \"authorino-f99f4b5cd-m2fc5\" (UID: \"ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4\") " pod="kuadrant-system/authorino-f99f4b5cd-m2fc5"
May 21 16:40:55.621860 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:55.621779    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-m2fc5"
May 21 16:40:55.767868 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:55.766829    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-m2fc5"]
May 21 16:40:55.771399 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:40:55.771236    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea8e8a3f_9a26_4741_8ea1_89fbe47e92c4.slice/crio-d24b91dd6b564b517e71fa0fd9b8b66260956d09292ce95343bfa058c047d373 WatchSource:0}: Error finding container d24b91dd6b564b517e71fa0fd9b8b66260956d09292ce95343bfa058c047d373: Status 404 returned error can't find the container with id d24b91dd6b564b517e71fa0fd9b8b66260956d09292ce95343bfa058c047d373
May 21 16:40:55.808585 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:55.808523    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd" event={"ID":"55304f34-8676-465a-8ef9-f98d0b71645a","Type":"ContainerStarted","Data":"d08c8f4a6a4584b0c1e36a490595206ab0d16cd67be8ace258d94cb93e124990"}
May 21 16:40:55.814752 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:40:55.814714    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-m2fc5" event={"ID":"ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4","Type":"ContainerStarted","Data":"d24b91dd6b564b517e71fa0fd9b8b66260956d09292ce95343bfa058c047d373"}
May 21 16:41:00.028603 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:00.028564    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-m2fc5"]
May 21 16:41:01.839701 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:01.839670    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-m2fc5" event={"ID":"ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4","Type":"ContainerStarted","Data":"9906c8a4d66b7f286dd4a76f31f0c26a7e241ea651ceff289a1dfcd3f143d70f"}
May 21 16:41:01.840236 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:01.839749    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-m2fc5" podUID="ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4" containerName="authorino" containerID="cri-o://9906c8a4d66b7f286dd4a76f31f0c26a7e241ea651ceff289a1dfcd3f143d70f" gracePeriod=30
May 21 16:41:01.841414 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:01.841388    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd" event={"ID":"55304f34-8676-465a-8ef9-f98d0b71645a","Type":"ContainerStarted","Data":"7d7534b4269cd9e2949bf2ca65a8e5b1d3a3ee90ff1b3d265d9d3512313387fa"}
May 21 16:41:01.841525 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:01.841474    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd"
May 21 16:41:01.854134 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:01.854086    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-m2fc5" podStartSLOduration=1.824055752 podStartE2EDuration="6.854042629s" podCreationTimestamp="2026-05-21 16:40:55 +0000 UTC" firstStartedPulling="2026-05-21 16:40:55.772731987 +0000 UTC m=+546.243852958" lastFinishedPulling="2026-05-21 16:41:00.802718879 +0000 UTC m=+551.273839835" observedRunningTime="2026-05-21 16:41:01.853286288 +0000 UTC m=+552.324407271" watchObservedRunningTime="2026-05-21 16:41:01.854042629 +0000 UTC m=+552.325163674"
May 21 16:41:01.868797 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:01.868743    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd" podStartSLOduration=2.020753506 podStartE2EDuration="7.868725987s" podCreationTimestamp="2026-05-21 16:40:54 +0000 UTC" firstStartedPulling="2026-05-21 16:40:54.962469306 +0000 UTC m=+545.433590266" lastFinishedPulling="2026-05-21 16:41:00.810441789 +0000 UTC m=+551.281562747" observedRunningTime="2026-05-21 16:41:01.867477258 +0000 UTC m=+552.338598239" watchObservedRunningTime="2026-05-21 16:41:01.868725987 +0000 UTC m=+552.339846966"
May 21 16:41:02.165111 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:02.165084    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-m2fc5"
May 21 16:41:02.237332 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:02.237291    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbw4c\" (UniqueName: \"kubernetes.io/projected/ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4-kube-api-access-dbw4c\") pod \"ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4\" (UID: \"ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4\") "
May 21 16:41:02.239651 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:02.239624    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4-kube-api-access-dbw4c" (OuterVolumeSpecName: "kube-api-access-dbw4c") pod "ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4" (UID: "ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4"). InnerVolumeSpecName "kube-api-access-dbw4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 16:41:02.338205 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:02.338160    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbw4c\" (UniqueName: \"kubernetes.io/projected/ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4-kube-api-access-dbw4c\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:41:02.845407 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:02.845359    2562 generic.go:358] "Generic (PLEG): container finished" podID="ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4" containerID="9906c8a4d66b7f286dd4a76f31f0c26a7e241ea651ceff289a1dfcd3f143d70f" exitCode=0
May 21 16:41:02.845801 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:02.845414    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-m2fc5"
May 21 16:41:02.845801 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:02.845449    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-m2fc5" event={"ID":"ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4","Type":"ContainerDied","Data":"9906c8a4d66b7f286dd4a76f31f0c26a7e241ea651ceff289a1dfcd3f143d70f"}
May 21 16:41:02.845801 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:02.845490    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-m2fc5" event={"ID":"ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4","Type":"ContainerDied","Data":"d24b91dd6b564b517e71fa0fd9b8b66260956d09292ce95343bfa058c047d373"}
May 21 16:41:02.845801 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:02.845511    2562 scope.go:117] "RemoveContainer" containerID="9906c8a4d66b7f286dd4a76f31f0c26a7e241ea651ceff289a1dfcd3f143d70f"
May 21 16:41:02.853334 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:02.853312    2562 scope.go:117] "RemoveContainer" containerID="9906c8a4d66b7f286dd4a76f31f0c26a7e241ea651ceff289a1dfcd3f143d70f"
May 21 16:41:02.853628 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:41:02.853607    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9906c8a4d66b7f286dd4a76f31f0c26a7e241ea651ceff289a1dfcd3f143d70f\": container with ID starting with 9906c8a4d66b7f286dd4a76f31f0c26a7e241ea651ceff289a1dfcd3f143d70f not found: ID does not exist" containerID="9906c8a4d66b7f286dd4a76f31f0c26a7e241ea651ceff289a1dfcd3f143d70f"
May 21 16:41:02.853704 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:02.853638    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9906c8a4d66b7f286dd4a76f31f0c26a7e241ea651ceff289a1dfcd3f143d70f"} err="failed to get container status \"9906c8a4d66b7f286dd4a76f31f0c26a7e241ea651ceff289a1dfcd3f143d70f\": rpc error: code = NotFound desc = could not find container \"9906c8a4d66b7f286dd4a76f31f0c26a7e241ea651ceff289a1dfcd3f143d70f\": container with ID starting with 9906c8a4d66b7f286dd4a76f31f0c26a7e241ea651ceff289a1dfcd3f143d70f not found: ID does not exist"
May 21 16:41:02.866513 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:02.866481    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-m2fc5"]
May 21 16:41:02.869714 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:02.869691    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-m2fc5"]
May 21 16:41:04.109172 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:04.109139    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4" path="/var/lib/kubelet/pods/ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4/volumes"
May 21 16:41:09.656584 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:09.656553    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-rrczd"]
May 21 16:41:09.656957 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:09.656836    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd" podUID="55304f34-8676-465a-8ef9-f98d0b71645a" containerName="limitador" containerID="cri-o://7d7534b4269cd9e2949bf2ca65a8e5b1d3a3ee90ff1b3d265d9d3512313387fa" gracePeriod=30
May 21 16:41:09.658963 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:09.658940    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd"
May 21 16:41:10.203222 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.203198    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd"
May 21 16:41:10.308918 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.308885    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55304f34-8676-465a-8ef9-f98d0b71645a-config-file\") pod \"55304f34-8676-465a-8ef9-f98d0b71645a\" (UID: \"55304f34-8676-465a-8ef9-f98d0b71645a\") "
May 21 16:41:10.309121 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.308972    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmk7v\" (UniqueName: \"kubernetes.io/projected/55304f34-8676-465a-8ef9-f98d0b71645a-kube-api-access-fmk7v\") pod \"55304f34-8676-465a-8ef9-f98d0b71645a\" (UID: \"55304f34-8676-465a-8ef9-f98d0b71645a\") "
May 21 16:41:10.309329 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.309303    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55304f34-8676-465a-8ef9-f98d0b71645a-config-file" (OuterVolumeSpecName: "config-file") pod "55304f34-8676-465a-8ef9-f98d0b71645a" (UID: "55304f34-8676-465a-8ef9-f98d0b71645a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 21 16:41:10.311187 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.311164    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55304f34-8676-465a-8ef9-f98d0b71645a-kube-api-access-fmk7v" (OuterVolumeSpecName: "kube-api-access-fmk7v") pod "55304f34-8676-465a-8ef9-f98d0b71645a" (UID: "55304f34-8676-465a-8ef9-f98d0b71645a"). InnerVolumeSpecName "kube-api-access-fmk7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 16:41:10.409509 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.409474    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmk7v\" (UniqueName: \"kubernetes.io/projected/55304f34-8676-465a-8ef9-f98d0b71645a-kube-api-access-fmk7v\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:41:10.409509 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.409504    2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55304f34-8676-465a-8ef9-f98d0b71645a-config-file\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:41:10.872455 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.872425    2562 generic.go:358] "Generic (PLEG): container finished" podID="55304f34-8676-465a-8ef9-f98d0b71645a" containerID="7d7534b4269cd9e2949bf2ca65a8e5b1d3a3ee90ff1b3d265d9d3512313387fa" exitCode=0
May 21 16:41:10.872894 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.872493    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd"
May 21 16:41:10.872894 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.872495    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd" event={"ID":"55304f34-8676-465a-8ef9-f98d0b71645a","Type":"ContainerDied","Data":"7d7534b4269cd9e2949bf2ca65a8e5b1d3a3ee90ff1b3d265d9d3512313387fa"}
May 21 16:41:10.872894 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.872593    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-rrczd" event={"ID":"55304f34-8676-465a-8ef9-f98d0b71645a","Type":"ContainerDied","Data":"d08c8f4a6a4584b0c1e36a490595206ab0d16cd67be8ace258d94cb93e124990"}
May 21 16:41:10.872894 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.872609    2562 scope.go:117] "RemoveContainer" containerID="7d7534b4269cd9e2949bf2ca65a8e5b1d3a3ee90ff1b3d265d9d3512313387fa"
May 21 16:41:10.881198 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.881167    2562 scope.go:117] "RemoveContainer" containerID="7d7534b4269cd9e2949bf2ca65a8e5b1d3a3ee90ff1b3d265d9d3512313387fa"
May 21 16:41:10.881447 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:41:10.881430    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d7534b4269cd9e2949bf2ca65a8e5b1d3a3ee90ff1b3d265d9d3512313387fa\": container with ID starting with 7d7534b4269cd9e2949bf2ca65a8e5b1d3a3ee90ff1b3d265d9d3512313387fa not found: ID does not exist" containerID="7d7534b4269cd9e2949bf2ca65a8e5b1d3a3ee90ff1b3d265d9d3512313387fa"
May 21 16:41:10.881485 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.881457    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7534b4269cd9e2949bf2ca65a8e5b1d3a3ee90ff1b3d265d9d3512313387fa"} err="failed to get container status \"7d7534b4269cd9e2949bf2ca65a8e5b1d3a3ee90ff1b3d265d9d3512313387fa\": rpc error: code = NotFound desc = could not find container \"7d7534b4269cd9e2949bf2ca65a8e5b1d3a3ee90ff1b3d265d9d3512313387fa\": container with ID starting with 7d7534b4269cd9e2949bf2ca65a8e5b1d3a3ee90ff1b3d265d9d3512313387fa not found: ID does not exist"
May 21 16:41:10.892369 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.892338    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-rrczd"]
May 21 16:41:10.896401 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:10.896379    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-rrczd"]
May 21 16:41:12.108770 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:12.108741    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55304f34-8676-465a-8ef9-f98d0b71645a" path="/var/lib/kubelet/pods/55304f34-8676-465a-8ef9-f98d0b71645a/volumes"
May 21 16:41:15.614871 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.614839    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-9q4qf"]
May 21 16:41:15.615361 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.615343    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55304f34-8676-465a-8ef9-f98d0b71645a" containerName="limitador"
May 21 16:41:15.615427 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.615362    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="55304f34-8676-465a-8ef9-f98d0b71645a" containerName="limitador"
May 21 16:41:15.615427 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.615376    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4" containerName="authorino"
May 21 16:41:15.615427 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.615384    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4" containerName="authorino"
May 21 16:41:15.615567 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.615474    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="ea8e8a3f-9a26-4741-8ea1-89fbe47e92c4" containerName="authorino"
May 21 16:41:15.615567 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.615490    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="55304f34-8676-465a-8ef9-f98d0b71645a" containerName="limitador"
May 21 16:41:15.619357 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.619335    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-9q4qf"
May 21 16:41:15.621722 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.621703    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
May 21 16:41:15.621825 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.621771    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-pwtbj\""
May 21 16:41:15.626138 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.626113    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-9q4qf"]
May 21 16:41:15.760117 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.760079    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e2acde12-a04d-4635-abde-9cd950fa97f9-data\") pod \"postgres-68c4fbbd6f-9q4qf\" (UID: \"e2acde12-a04d-4635-abde-9cd950fa97f9\") " pod="opendatahub/postgres-68c4fbbd6f-9q4qf"
May 21 16:41:15.760318 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.760213    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xlsf\" (UniqueName: \"kubernetes.io/projected/e2acde12-a04d-4635-abde-9cd950fa97f9-kube-api-access-5xlsf\") pod \"postgres-68c4fbbd6f-9q4qf\" (UID: \"e2acde12-a04d-4635-abde-9cd950fa97f9\") " pod="opendatahub/postgres-68c4fbbd6f-9q4qf"
May 21 16:41:15.861461 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.861410    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5xlsf\" (UniqueName: \"kubernetes.io/projected/e2acde12-a04d-4635-abde-9cd950fa97f9-kube-api-access-5xlsf\") pod \"postgres-68c4fbbd6f-9q4qf\" (UID: \"e2acde12-a04d-4635-abde-9cd950fa97f9\") " pod="opendatahub/postgres-68c4fbbd6f-9q4qf"
May 21 16:41:15.861615 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.861487    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e2acde12-a04d-4635-abde-9cd950fa97f9-data\") pod \"postgres-68c4fbbd6f-9q4qf\" (UID: \"e2acde12-a04d-4635-abde-9cd950fa97f9\") " pod="opendatahub/postgres-68c4fbbd6f-9q4qf"
May 21 16:41:15.861824 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.861809    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e2acde12-a04d-4635-abde-9cd950fa97f9-data\") pod \"postgres-68c4fbbd6f-9q4qf\" (UID: \"e2acde12-a04d-4635-abde-9cd950fa97f9\") " pod="opendatahub/postgres-68c4fbbd6f-9q4qf"
May 21 16:41:15.870277 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.870212    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xlsf\" (UniqueName: \"kubernetes.io/projected/e2acde12-a04d-4635-abde-9cd950fa97f9-kube-api-access-5xlsf\") pod \"postgres-68c4fbbd6f-9q4qf\" (UID: \"e2acde12-a04d-4635-abde-9cd950fa97f9\") " pod="opendatahub/postgres-68c4fbbd6f-9q4qf"
May 21 16:41:15.932495 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:15.932454    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-9q4qf"
May 21 16:41:16.062720 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:16.062624    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-9q4qf"]
May 21 16:41:16.065316 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:41:16.065290    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2acde12_a04d_4635_abde_9cd950fa97f9.slice/crio-864a98c4bc9ac344b8155f94ca89f55301ae05fb523a3fb12adeaa0412fa301a WatchSource:0}: Error finding container 864a98c4bc9ac344b8155f94ca89f55301ae05fb523a3fb12adeaa0412fa301a: Status 404 returned error can't find the container with id 864a98c4bc9ac344b8155f94ca89f55301ae05fb523a3fb12adeaa0412fa301a
May 21 16:41:16.894750 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:16.894711    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-9q4qf" event={"ID":"e2acde12-a04d-4635-abde-9cd950fa97f9","Type":"ContainerStarted","Data":"864a98c4bc9ac344b8155f94ca89f55301ae05fb523a3fb12adeaa0412fa301a"}
May 21 16:41:21.913207 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:21.913161    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-9q4qf" event={"ID":"e2acde12-a04d-4635-abde-9cd950fa97f9","Type":"ContainerStarted","Data":"1a663fc02f819f4c9b05e6a7c3083522c74666e1af37bc887d523e5b82683d90"}
May 21 16:41:21.913606 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:21.913306    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-9q4qf"
May 21 16:41:21.930183 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:21.930124    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-9q4qf" podStartSLOduration=1.8949332349999999 podStartE2EDuration="6.930108019s" podCreationTimestamp="2026-05-21 16:41:15 +0000 UTC" firstStartedPulling="2026-05-21 16:41:16.066727961 +0000 UTC m=+566.537848921" lastFinishedPulling="2026-05-21 16:41:21.101902748 +0000 UTC m=+571.573023705" observedRunningTime="2026-05-21 16:41:21.927492965 +0000 UTC m=+572.398613942" watchObservedRunningTime="2026-05-21 16:41:21.930108019 +0000 UTC m=+572.401228997"
May 21 16:41:27.945514 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:27.945483    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-9q4qf"
May 21 16:41:28.471078 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.471016    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-zbh5p"]
May 21 16:41:28.480510 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.480480    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-zbh5p"]
May 21 16:41:28.480669 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.480583    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-zbh5p"
May 21 16:41:28.482829 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.482803    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-g9xmg\""
May 21 16:41:28.669430 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.669396    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c26pm\" (UniqueName: \"kubernetes.io/projected/9ef68ceb-f2e9-46ef-844d-846d36fa11e1-kube-api-access-c26pm\") pod \"authorino-8b475cf9f-zbh5p\" (UID: \"9ef68ceb-f2e9-46ef-844d-846d36fa11e1\") " pod="kuadrant-system/authorino-8b475cf9f-zbh5p"
May 21 16:41:28.702393 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.702354    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-zbh5p"]
May 21 16:41:28.702601 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:41:28.702582    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-c26pm], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-8b475cf9f-zbh5p" podUID="9ef68ceb-f2e9-46ef-844d-846d36fa11e1"
May 21 16:41:28.727871 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.727796    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-fc789fc84-8245f"]
May 21 16:41:28.731423 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.731403    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fc789fc84-8245f"
May 21 16:41:28.739648 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.739622    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fc789fc84-8245f"]
May 21 16:41:28.770669 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.770637    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c26pm\" (UniqueName: \"kubernetes.io/projected/9ef68ceb-f2e9-46ef-844d-846d36fa11e1-kube-api-access-c26pm\") pod \"authorino-8b475cf9f-zbh5p\" (UID: \"9ef68ceb-f2e9-46ef-844d-846d36fa11e1\") " pod="kuadrant-system/authorino-8b475cf9f-zbh5p"
May 21 16:41:28.778869 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.778832    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c26pm\" (UniqueName: \"kubernetes.io/projected/9ef68ceb-f2e9-46ef-844d-846d36fa11e1-kube-api-access-c26pm\") pod \"authorino-8b475cf9f-zbh5p\" (UID: \"9ef68ceb-f2e9-46ef-844d-846d36fa11e1\") " pod="kuadrant-system/authorino-8b475cf9f-zbh5p"
May 21 16:41:28.871864 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.871823    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlpqd\" (UniqueName: \"kubernetes.io/projected/a349ca00-a3f2-4486-a987-3cf6b1f820dc-kube-api-access-tlpqd\") pod \"authorino-fc789fc84-8245f\" (UID: \"a349ca00-a3f2-4486-a987-3cf6b1f820dc\") " pod="kuadrant-system/authorino-fc789fc84-8245f"
May 21 16:41:28.934547 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.934518    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-zbh5p"
May 21 16:41:28.939462 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.939442    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-zbh5p"
May 21 16:41:28.972604 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.972561    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tlpqd\" (UniqueName: \"kubernetes.io/projected/a349ca00-a3f2-4486-a987-3cf6b1f820dc-kube-api-access-tlpqd\") pod \"authorino-fc789fc84-8245f\" (UID: \"a349ca00-a3f2-4486-a987-3cf6b1f820dc\") " pod="kuadrant-system/authorino-fc789fc84-8245f"
May 21 16:41:28.981340 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.981265    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlpqd\" (UniqueName: \"kubernetes.io/projected/a349ca00-a3f2-4486-a987-3cf6b1f820dc-kube-api-access-tlpqd\") pod \"authorino-fc789fc84-8245f\" (UID: \"a349ca00-a3f2-4486-a987-3cf6b1f820dc\") " pod="kuadrant-system/authorino-fc789fc84-8245f"
May 21 16:41:28.982360 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.982338    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-fc789fc84-8245f"]
May 21 16:41:28.982583 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:28.982571    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fc789fc84-8245f"
May 21 16:41:29.008986 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.008958    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6b875bd744-khql5"]
May 21 16:41:29.013947 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.013924    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b875bd744-khql5"
May 21 16:41:29.016696 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.016661    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
May 21 16:41:29.019098 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.019068    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b875bd744-khql5"]
May 21 16:41:29.073499 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.073472    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c26pm\" (UniqueName: \"kubernetes.io/projected/9ef68ceb-f2e9-46ef-844d-846d36fa11e1-kube-api-access-c26pm\") pod \"9ef68ceb-f2e9-46ef-844d-846d36fa11e1\" (UID: \"9ef68ceb-f2e9-46ef-844d-846d36fa11e1\") "
May 21 16:41:29.075795 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.075765    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ef68ceb-f2e9-46ef-844d-846d36fa11e1-kube-api-access-c26pm" (OuterVolumeSpecName: "kube-api-access-c26pm") pod "9ef68ceb-f2e9-46ef-844d-846d36fa11e1" (UID: "9ef68ceb-f2e9-46ef-844d-846d36fa11e1"). InnerVolumeSpecName "kube-api-access-c26pm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 16:41:29.121419 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.121294    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-fc789fc84-8245f"]
May 21 16:41:29.124024 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:41:29.124000    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda349ca00_a3f2_4486_a987_3cf6b1f820dc.slice/crio-de09a239a4b304bf592e6c6faf3b5a481f057c037970e562e72d4e9120662b9e WatchSource:0}: Error finding container de09a239a4b304bf592e6c6faf3b5a481f057c037970e562e72d4e9120662b9e: Status 404 returned error can't find the container with id de09a239a4b304bf592e6c6faf3b5a481f057c037970e562e72d4e9120662b9e
May 21 16:41:29.174330 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.174292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6-tls-cert\") pod \"authorino-6b875bd744-khql5\" (UID: \"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6\") " pod="kuadrant-system/authorino-6b875bd744-khql5"
May 21 16:41:29.174492 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.174341    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxwjh\" (UniqueName: \"kubernetes.io/projected/314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6-kube-api-access-vxwjh\") pod \"authorino-6b875bd744-khql5\" (UID: \"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6\") " pod="kuadrant-system/authorino-6b875bd744-khql5"
May 21 16:41:29.174492 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.174447    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c26pm\" (UniqueName: \"kubernetes.io/projected/9ef68ceb-f2e9-46ef-844d-846d36fa11e1-kube-api-access-c26pm\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:41:29.275271 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.275237    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6-tls-cert\") pod \"authorino-6b875bd744-khql5\" (UID: \"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6\") " pod="kuadrant-system/authorino-6b875bd744-khql5"
May 21 16:41:29.275412 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.275287    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vxwjh\" (UniqueName: \"kubernetes.io/projected/314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6-kube-api-access-vxwjh\") pod \"authorino-6b875bd744-khql5\" (UID: \"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6\") " pod="kuadrant-system/authorino-6b875bd744-khql5"
May 21 16:41:29.277786 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.277764    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6-tls-cert\") pod \"authorino-6b875bd744-khql5\" (UID: \"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6\") " pod="kuadrant-system/authorino-6b875bd744-khql5"
May 21 16:41:29.283313 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.283288    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxwjh\" (UniqueName: \"kubernetes.io/projected/314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6-kube-api-access-vxwjh\") pod \"authorino-6b875bd744-khql5\" (UID: \"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6\") " pod="kuadrant-system/authorino-6b875bd744-khql5"
May 21 16:41:29.326834 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.326797    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b875bd744-khql5"
May 21 16:41:29.457778 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.457632    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b875bd744-khql5"]
May 21 16:41:29.460388 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:41:29.460361    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod314a48a4_9c7b_4ef2_b5f3_fb1fed23c1f6.slice/crio-d76d71de3578fbf4fa7537e40dcc688451ee8b9c8878a8398345f9cccbcf6e7a WatchSource:0}: Error finding container d76d71de3578fbf4fa7537e40dcc688451ee8b9c8878a8398345f9cccbcf6e7a: Status 404 returned error can't find the container with id d76d71de3578fbf4fa7537e40dcc688451ee8b9c8878a8398345f9cccbcf6e7a
May 21 16:41:29.939567 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.939531    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fc789fc84-8245f" event={"ID":"a349ca00-a3f2-4486-a987-3cf6b1f820dc","Type":"ContainerStarted","Data":"1e6aab9d6982e775cab17ebe61401fa9c2caf01a3a70a8f53b13de04086d2754"}
May 21 16:41:29.939776 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.939574    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fc789fc84-8245f" event={"ID":"a349ca00-a3f2-4486-a987-3cf6b1f820dc","Type":"ContainerStarted","Data":"de09a239a4b304bf592e6c6faf3b5a481f057c037970e562e72d4e9120662b9e"}
May 21 16:41:29.939776 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.939593    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-fc789fc84-8245f" podUID="a349ca00-a3f2-4486-a987-3cf6b1f820dc" containerName="authorino" containerID="cri-o://1e6aab9d6982e775cab17ebe61401fa9c2caf01a3a70a8f53b13de04086d2754" gracePeriod=30
May 21 16:41:29.940980 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.940950    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b875bd744-khql5" event={"ID":"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6","Type":"ContainerStarted","Data":"156b94b08abab8027d2a6fc2ba1bc0469d0467cb700f6348e57a273b383edafa"}
May 21 16:41:29.941106 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.940993    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b875bd744-khql5" event={"ID":"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6","Type":"ContainerStarted","Data":"d76d71de3578fbf4fa7537e40dcc688451ee8b9c8878a8398345f9cccbcf6e7a"}
May 21 16:41:29.941106 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.940968    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-zbh5p"
May 21 16:41:29.953870 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.953833    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-fc789fc84-8245f" podStartSLOduration=1.597955198 podStartE2EDuration="1.95382008s" podCreationTimestamp="2026-05-21 16:41:28 +0000 UTC" firstStartedPulling="2026-05-21 16:41:29.125256836 +0000 UTC m=+579.596377796" lastFinishedPulling="2026-05-21 16:41:29.481121719 +0000 UTC m=+579.952242678" observedRunningTime="2026-05-21 16:41:29.953137498 +0000 UTC m=+580.424258477" watchObservedRunningTime="2026-05-21 16:41:29.95382008 +0000 UTC m=+580.424941107"
May 21 16:41:29.972745 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.972718    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-zbh5p"]
May 21 16:41:29.975682 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.975653    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-zbh5p"]
May 21 16:41:29.987007 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:29.986970    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-6b875bd744-khql5" podStartSLOduration=1.684882666 podStartE2EDuration="1.986956773s" podCreationTimestamp="2026-05-21 16:41:28 +0000 UTC" firstStartedPulling="2026-05-21 16:41:29.461856598 +0000 UTC m=+579.932977555" lastFinishedPulling="2026-05-21 16:41:29.763930705 +0000 UTC m=+580.235051662" observedRunningTime="2026-05-21 16:41:29.98583606 +0000 UTC m=+580.456957040" watchObservedRunningTime="2026-05-21 16:41:29.986956773 +0000 UTC m=+580.458077752"
May 21 16:41:30.117727 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.117683    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ef68ceb-f2e9-46ef-844d-846d36fa11e1" path="/var/lib/kubelet/pods/9ef68ceb-f2e9-46ef-844d-846d36fa11e1/volumes"
May 21 16:41:30.219193 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.219168    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fc789fc84-8245f"
May 21 16:41:30.295988 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.295940    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlpqd\" (UniqueName: \"kubernetes.io/projected/a349ca00-a3f2-4486-a987-3cf6b1f820dc-kube-api-access-tlpqd\") pod \"a349ca00-a3f2-4486-a987-3cf6b1f820dc\" (UID: \"a349ca00-a3f2-4486-a987-3cf6b1f820dc\") "
May 21 16:41:30.298232 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.298205    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a349ca00-a3f2-4486-a987-3cf6b1f820dc-kube-api-access-tlpqd" (OuterVolumeSpecName: "kube-api-access-tlpqd") pod "a349ca00-a3f2-4486-a987-3cf6b1f820dc" (UID: "a349ca00-a3f2-4486-a987-3cf6b1f820dc"). InnerVolumeSpecName "kube-api-access-tlpqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 16:41:30.397102 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.396994    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlpqd\" (UniqueName: \"kubernetes.io/projected/a349ca00-a3f2-4486-a987-3cf6b1f820dc-kube-api-access-tlpqd\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:41:30.946138 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.946101    2562 generic.go:358] "Generic (PLEG): container finished" podID="a349ca00-a3f2-4486-a987-3cf6b1f820dc" containerID="1e6aab9d6982e775cab17ebe61401fa9c2caf01a3a70a8f53b13de04086d2754" exitCode=0
May 21 16:41:30.946401 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.946152    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fc789fc84-8245f"
May 21 16:41:30.946401 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.946187    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fc789fc84-8245f" event={"ID":"a349ca00-a3f2-4486-a987-3cf6b1f820dc","Type":"ContainerDied","Data":"1e6aab9d6982e775cab17ebe61401fa9c2caf01a3a70a8f53b13de04086d2754"}
May 21 16:41:30.946401 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.946231    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fc789fc84-8245f" event={"ID":"a349ca00-a3f2-4486-a987-3cf6b1f820dc","Type":"ContainerDied","Data":"de09a239a4b304bf592e6c6faf3b5a481f057c037970e562e72d4e9120662b9e"}
May 21 16:41:30.946401 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.946247    2562 scope.go:117] "RemoveContainer" containerID="1e6aab9d6982e775cab17ebe61401fa9c2caf01a3a70a8f53b13de04086d2754"
May 21 16:41:30.954925 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.954908    2562 scope.go:117] "RemoveContainer" containerID="1e6aab9d6982e775cab17ebe61401fa9c2caf01a3a70a8f53b13de04086d2754"
May 21 16:41:30.955219 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:41:30.955201    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e6aab9d6982e775cab17ebe61401fa9c2caf01a3a70a8f53b13de04086d2754\": container with ID starting with 1e6aab9d6982e775cab17ebe61401fa9c2caf01a3a70a8f53b13de04086d2754 not found: ID does not exist" containerID="1e6aab9d6982e775cab17ebe61401fa9c2caf01a3a70a8f53b13de04086d2754"
May 21 16:41:30.955282 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.955228    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e6aab9d6982e775cab17ebe61401fa9c2caf01a3a70a8f53b13de04086d2754"} err="failed to get container status \"1e6aab9d6982e775cab17ebe61401fa9c2caf01a3a70a8f53b13de04086d2754\": rpc error: code = NotFound desc = could not find container \"1e6aab9d6982e775cab17ebe61401fa9c2caf01a3a70a8f53b13de04086d2754\": container with ID starting with 1e6aab9d6982e775cab17ebe61401fa9c2caf01a3a70a8f53b13de04086d2754 not found: ID does not exist"
May 21 16:41:30.966519 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.966495    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-fc789fc84-8245f"]
May 21 16:41:30.969465 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:30.969445    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-fc789fc84-8245f"]
May 21 16:41:32.108655 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:41:32.108622    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a349ca00-a3f2-4486-a987-3cf6b1f820dc" path="/var/lib/kubelet/pods/a349ca00-a3f2-4486-a987-3cf6b1f820dc/volumes"
May 21 16:42:23.670101 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.669967    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"]
May 21 16:42:23.672088 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.670356    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a349ca00-a3f2-4486-a987-3cf6b1f820dc" containerName="authorino"
May 21 16:42:23.672088 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.670370    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a349ca00-a3f2-4486-a987-3cf6b1f820dc" containerName="authorino"
May 21 16:42:23.672088 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.670430    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="a349ca00-a3f2-4486-a987-3cf6b1f820dc" containerName="authorino"
May 21 16:42:23.672393 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.672377    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.676015 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.675992    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-46n4s\""
May 21 16:42:23.676015 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.676003    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
May 21 16:42:23.676226 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.676030    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
May 21 16:42:23.676226 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.676011    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
May 21 16:42:23.683208 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.683179    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"]
May 21 16:42:23.750603 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.750572    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw6pb\" (UniqueName: \"kubernetes.io/projected/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-kube-api-access-jw6pb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.750770 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.750609    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.750770 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.750641    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.750770 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.750666    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.750770 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.750684    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.750770 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.750714    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.851328 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.851275    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.851328 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.851326    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.851569 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.851344    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.851569 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.851381    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.851569 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.851438    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jw6pb\" (UniqueName: \"kubernetes.io/projected/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-kube-api-access-jw6pb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.851569 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.851467    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.852099 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.851708    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.852099 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.851733    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.852099 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.851827    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.853919 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.853900    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.854127 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.854111    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.861459 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.861425    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw6pb\" (UniqueName: \"kubernetes.io/projected/f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be-kube-api-access-jw6pb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj\" (UID: \"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:23.983339 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:23.983254    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:24.119150 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:24.119102    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"]
May 21 16:42:24.121951 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:42:24.121924    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5f8c0f6_d2be_4ba8_8ab0_dc546ea5a0be.slice/crio-a4cc211d7b2705c29e55f39be33758c64a2026bb15851e61d7c9c6d87f3cb3a9 WatchSource:0}: Error finding container a4cc211d7b2705c29e55f39be33758c64a2026bb15851e61d7c9c6d87f3cb3a9: Status 404 returned error can't find the container with id a4cc211d7b2705c29e55f39be33758c64a2026bb15851e61d7c9c6d87f3cb3a9
May 21 16:42:25.124141 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:25.124106    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj" event={"ID":"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be","Type":"ContainerStarted","Data":"a4cc211d7b2705c29e55f39be33758c64a2026bb15851e61d7c9c6d87f3cb3a9"}
May 21 16:42:31.146841 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:31.146801    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj" event={"ID":"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be","Type":"ContainerStarted","Data":"bd6197c6843c93292c6efb7d906ea077f941d5e84db6368ee907fee743f0b276"}
May 21 16:42:37.167984 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:37.167947    2562 generic.go:358] "Generic (PLEG): container finished" podID="f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be" containerID="bd6197c6843c93292c6efb7d906ea077f941d5e84db6368ee907fee743f0b276" exitCode=0
May 21 16:42:37.168469 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:37.168029    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj" event={"ID":"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be","Type":"ContainerDied","Data":"bd6197c6843c93292c6efb7d906ea077f941d5e84db6368ee907fee743f0b276"}
May 21 16:42:39.179826 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:39.179795    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj" event={"ID":"f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be","Type":"ContainerStarted","Data":"71b93d65ef5745330a841fc146ae5c03d1f4a4f84679c72fb7512164adc8bb24"}
May 21 16:42:39.180212 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:39.180010    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:42:39.201407 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:39.201362    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj" podStartSLOduration=2.14983277 podStartE2EDuration="16.201348212s" podCreationTimestamp="2026-05-21 16:42:23 +0000 UTC" firstStartedPulling="2026-05-21 16:42:24.123656954 +0000 UTC m=+634.594777915" lastFinishedPulling="2026-05-21 16:42:38.175172396 +0000 UTC m=+648.646293357" observedRunningTime="2026-05-21 16:42:39.200180607 +0000 UTC m=+649.671301587" watchObservedRunningTime="2026-05-21 16:42:39.201348212 +0000 UTC m=+649.672469191"
May 21 16:42:50.195217 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:42:50.195187    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj"
May 21 16:43:07.056068 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.056013    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"]
May 21 16:43:07.088399 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.088369    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"]
May 21 16:43:07.088559 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.088493    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.091657 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.091635    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
May 21 16:43:07.138274 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.138239    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf9kt\" (UniqueName: \"kubernetes.io/projected/985fe487-2eba-4d43-a8f8-17142cb541f9-kube-api-access-rf9kt\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.138441 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.138305    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/985fe487-2eba-4d43-a8f8-17142cb541f9-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.138441 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.138367    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/985fe487-2eba-4d43-a8f8-17142cb541f9-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.138441 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.138391    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/985fe487-2eba-4d43-a8f8-17142cb541f9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.138602 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.138493    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/985fe487-2eba-4d43-a8f8-17142cb541f9-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.138602 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.138518    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/985fe487-2eba-4d43-a8f8-17142cb541f9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.239856 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.239813    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/985fe487-2eba-4d43-a8f8-17142cb541f9-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.239856 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.239857    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/985fe487-2eba-4d43-a8f8-17142cb541f9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.240142 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.239899    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rf9kt\" (UniqueName: \"kubernetes.io/projected/985fe487-2eba-4d43-a8f8-17142cb541f9-kube-api-access-rf9kt\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.240142 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.239925    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/985fe487-2eba-4d43-a8f8-17142cb541f9-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.240142 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.239948    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/985fe487-2eba-4d43-a8f8-17142cb541f9-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.240317 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.240142    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/985fe487-2eba-4d43-a8f8-17142cb541f9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.240317 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.240305    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/985fe487-2eba-4d43-a8f8-17142cb541f9-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.240435 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.240364    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/985fe487-2eba-4d43-a8f8-17142cb541f9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.240435 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.240402    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/985fe487-2eba-4d43-a8f8-17142cb541f9-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.242459 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.242437    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/985fe487-2eba-4d43-a8f8-17142cb541f9-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.242702 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.242684    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/985fe487-2eba-4d43-a8f8-17142cb541f9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.247802 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.247783    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf9kt\" (UniqueName: \"kubernetes.io/projected/985fe487-2eba-4d43-a8f8-17142cb541f9-kube-api-access-rf9kt\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5\" (UID: \"985fe487-2eba-4d43-a8f8-17142cb541f9\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.399684 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.399584    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:07.536961 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:07.536929    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"]
May 21 16:43:07.540908 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:43:07.540879    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985fe487_2eba_4d43_a8f8_17142cb541f9.slice/crio-7565d87a7b23e9dddb2fd96ab9171f7f9eaee0600717477d4e38c004430c191b WatchSource:0}: Error finding container 7565d87a7b23e9dddb2fd96ab9171f7f9eaee0600717477d4e38c004430c191b: Status 404 returned error can't find the container with id 7565d87a7b23e9dddb2fd96ab9171f7f9eaee0600717477d4e38c004430c191b
May 21 16:43:08.273880 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:08.273839    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5" event={"ID":"985fe487-2eba-4d43-a8f8-17142cb541f9","Type":"ContainerStarted","Data":"caeb7027b6ee0138d4d6e8503aff5219925808547cffa9d180a07df94f151e7d"}
May 21 16:43:08.273880 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:08.273880    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5" event={"ID":"985fe487-2eba-4d43-a8f8-17142cb541f9","Type":"ContainerStarted","Data":"7565d87a7b23e9dddb2fd96ab9171f7f9eaee0600717477d4e38c004430c191b"}
May 21 16:43:13.298838 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:13.298803    2562 generic.go:358] "Generic (PLEG): container finished" podID="985fe487-2eba-4d43-a8f8-17142cb541f9" containerID="caeb7027b6ee0138d4d6e8503aff5219925808547cffa9d180a07df94f151e7d" exitCode=0
May 21 16:43:13.299269 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:13.298874    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5" event={"ID":"985fe487-2eba-4d43-a8f8-17142cb541f9","Type":"ContainerDied","Data":"caeb7027b6ee0138d4d6e8503aff5219925808547cffa9d180a07df94f151e7d"}
May 21 16:43:13.299434 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:13.299420    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 21 16:43:14.304537 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:14.304500    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5" event={"ID":"985fe487-2eba-4d43-a8f8-17142cb541f9","Type":"ContainerStarted","Data":"4dede9e0b1b80874e7d5677dbda312959b5c3c16cdca06079a1b78509a9f1d81"}
May 21 16:43:14.304954 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:14.304726    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:14.322730 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:14.322678    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5" podStartSLOduration=6.982651337 podStartE2EDuration="7.322661356s" podCreationTimestamp="2026-05-21 16:43:07 +0000 UTC" firstStartedPulling="2026-05-21 16:43:13.299531472 +0000 UTC m=+683.770652429" lastFinishedPulling="2026-05-21 16:43:13.639541485 +0000 UTC m=+684.110662448" observedRunningTime="2026-05-21 16:43:14.321658904 +0000 UTC m=+684.792779882" watchObservedRunningTime="2026-05-21 16:43:14.322661356 +0000 UTC m=+684.793782334"
May 21 16:43:25.321674 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:25.321642    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5"
May 21 16:43:28.448709 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.448672    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"]
May 21 16:43:28.452411 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.452395    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.455190 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.455164    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
May 21 16:43:28.462255 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.462226    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"]
May 21 16:43:28.520722 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.520675    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.520919 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.520731    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.520919 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.520777    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.520919 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.520814    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd2q4\" (UniqueName: \"kubernetes.io/projected/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-kube-api-access-sd2q4\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.521087 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.520957    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.521087 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.520990    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.621438 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.621407    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.621438 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.621442    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.621677 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.621468    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.621677 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.621484    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.621677 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.621505    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.621677 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.621638    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sd2q4\" (UniqueName: \"kubernetes.io/projected/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-kube-api-access-sd2q4\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.621907 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.621876    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.621972 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.621908    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.621972 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.621946    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.623861 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.623833    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.624279 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.624261    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.631372 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.631351    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd2q4\" (UniqueName: \"kubernetes.io/projected/48187a78-76fb-42b9-bdb7-4e2bd1ae6305-kube-api-access-sd2q4\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f\" (UID: \"48187a78-76fb-42b9-bdb7-4e2bd1ae6305\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.764104 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.763998    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:28.895770 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:28.895729    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"]
May 21 16:43:28.898330 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:43:28.898308    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48187a78_76fb_42b9_bdb7_4e2bd1ae6305.slice/crio-70f4282456da26b65ee78ae158a095ea1ee8558b0a403bce25addc52549b6f6b WatchSource:0}: Error finding container 70f4282456da26b65ee78ae158a095ea1ee8558b0a403bce25addc52549b6f6b: Status 404 returned error can't find the container with id 70f4282456da26b65ee78ae158a095ea1ee8558b0a403bce25addc52549b6f6b
May 21 16:43:29.356830 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:29.356791    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f" event={"ID":"48187a78-76fb-42b9-bdb7-4e2bd1ae6305","Type":"ContainerStarted","Data":"e73016f374260d3036db2d69afde8eaab6f355d26feb2b1f70ea5066e8907d97"}
May 21 16:43:29.356830 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:29.356829    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f" event={"ID":"48187a78-76fb-42b9-bdb7-4e2bd1ae6305","Type":"ContainerStarted","Data":"70f4282456da26b65ee78ae158a095ea1ee8558b0a403bce25addc52549b6f6b"}
May 21 16:43:34.376471 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:34.376433    2562 generic.go:358] "Generic (PLEG): container finished" podID="48187a78-76fb-42b9-bdb7-4e2bd1ae6305" containerID="e73016f374260d3036db2d69afde8eaab6f355d26feb2b1f70ea5066e8907d97" exitCode=0
May 21 16:43:34.376856 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:34.376507    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f" event={"ID":"48187a78-76fb-42b9-bdb7-4e2bd1ae6305","Type":"ContainerDied","Data":"e73016f374260d3036db2d69afde8eaab6f355d26feb2b1f70ea5066e8907d97"}
May 21 16:43:35.384883 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:35.384852    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f" event={"ID":"48187a78-76fb-42b9-bdb7-4e2bd1ae6305","Type":"ContainerStarted","Data":"43d5cda0b6a83f37960f2fa0f4d2ab7fa705d773bdd5d3159a308504c841a533"}
May 21 16:43:35.385380 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:35.385203    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:43:35.415808 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:35.415758    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f" podStartSLOduration=7.210342466 podStartE2EDuration="7.415743499s" podCreationTimestamp="2026-05-21 16:43:28 +0000 UTC" firstStartedPulling="2026-05-21 16:43:34.377189994 +0000 UTC m=+704.848310954" lastFinishedPulling="2026-05-21 16:43:34.58259102 +0000 UTC m=+705.053711987" observedRunningTime="2026-05-21 16:43:35.415710687 +0000 UTC m=+705.886831667" watchObservedRunningTime="2026-05-21 16:43:35.415743499 +0000 UTC m=+705.886864480"
May 21 16:43:46.400611 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:43:46.400522    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f"
May 21 16:44:09.760196 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:09.760163    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b875bd744-khql5"]
May 21 16:44:09.760668 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:09.760370    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-6b875bd744-khql5" podUID="314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6" containerName="authorino" containerID="cri-o://156b94b08abab8027d2a6fc2ba1bc0469d0467cb700f6348e57a273b383edafa" gracePeriod=30
May 21 16:44:10.009501 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.009475    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b875bd744-khql5"
May 21 16:44:10.081252 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.081221    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxwjh\" (UniqueName: \"kubernetes.io/projected/314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6-kube-api-access-vxwjh\") pod \"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6\" (UID: \"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6\") "
May 21 16:44:10.081420 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.081318    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6-tls-cert\") pod \"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6\" (UID: \"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6\") "
May 21 16:44:10.083464 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.083432    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6-kube-api-access-vxwjh" (OuterVolumeSpecName: "kube-api-access-vxwjh") pod "314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6" (UID: "314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6"). InnerVolumeSpecName "kube-api-access-vxwjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 16:44:10.091328 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.091302    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6" (UID: "314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 16:44:10.182620 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.182584    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxwjh\" (UniqueName: \"kubernetes.io/projected/314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6-kube-api-access-vxwjh\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:44:10.182620 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.182615    2562 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6-tls-cert\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:44:10.502480 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.502391    2562 generic.go:358] "Generic (PLEG): container finished" podID="314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6" containerID="156b94b08abab8027d2a6fc2ba1bc0469d0467cb700f6348e57a273b383edafa" exitCode=0
May 21 16:44:10.502480 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.502443    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b875bd744-khql5" event={"ID":"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6","Type":"ContainerDied","Data":"156b94b08abab8027d2a6fc2ba1bc0469d0467cb700f6348e57a273b383edafa"}
May 21 16:44:10.502480 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.502471    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b875bd744-khql5" event={"ID":"314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6","Type":"ContainerDied","Data":"d76d71de3578fbf4fa7537e40dcc688451ee8b9c8878a8398345f9cccbcf6e7a"}
May 21 16:44:10.502750 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.502487    2562 scope.go:117] "RemoveContainer" containerID="156b94b08abab8027d2a6fc2ba1bc0469d0467cb700f6348e57a273b383edafa"
May 21 16:44:10.502750 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.502448    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b875bd744-khql5"
May 21 16:44:10.510762 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.510742    2562 scope.go:117] "RemoveContainer" containerID="156b94b08abab8027d2a6fc2ba1bc0469d0467cb700f6348e57a273b383edafa"
May 21 16:44:10.511005 ip-10-0-136-21 kubenswrapper[2562]: E0521 16:44:10.510985    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"156b94b08abab8027d2a6fc2ba1bc0469d0467cb700f6348e57a273b383edafa\": container with ID starting with 156b94b08abab8027d2a6fc2ba1bc0469d0467cb700f6348e57a273b383edafa not found: ID does not exist" containerID="156b94b08abab8027d2a6fc2ba1bc0469d0467cb700f6348e57a273b383edafa"
May 21 16:44:10.511080 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.511014    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"156b94b08abab8027d2a6fc2ba1bc0469d0467cb700f6348e57a273b383edafa"} err="failed to get container status \"156b94b08abab8027d2a6fc2ba1bc0469d0467cb700f6348e57a273b383edafa\": rpc error: code = NotFound desc = could not find container \"156b94b08abab8027d2a6fc2ba1bc0469d0467cb700f6348e57a273b383edafa\": container with ID starting with 156b94b08abab8027d2a6fc2ba1bc0469d0467cb700f6348e57a273b383edafa not found: ID does not exist"
May 21 16:44:10.518531 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.518506    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b875bd744-khql5"]
May 21 16:44:10.524622 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:10.524601    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6b875bd744-khql5"]
May 21 16:44:12.108758 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:44:12.108719    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6" path="/var/lib/kubelet/pods/314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6/volumes"
May 21 16:45:00.142719 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:00.142682    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29656365-9q2nr"]
May 21 16:45:00.143149 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:00.143030    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6" containerName="authorino"
May 21 16:45:00.143149 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:00.143042    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6" containerName="authorino"
May 21 16:45:00.143149 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:00.143125    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="314a48a4-9c7b-4ef2-b5f3-fb1fed23c1f6" containerName="authorino"
May 21 16:45:00.147916 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:00.147885    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29656365-9q2nr"
May 21 16:45:00.151714 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:00.151687    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-bzpx5\""
May 21 16:45:00.152857 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:00.152823    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29656365-9q2nr"]
May 21 16:45:00.234710 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:00.234675    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnjpq\" (UniqueName: \"kubernetes.io/projected/dd2fad01-ba6c-48cc-8001-f9c0f60e6292-kube-api-access-xnjpq\") pod \"maas-api-key-cleanup-29656365-9q2nr\" (UID: \"dd2fad01-ba6c-48cc-8001-f9c0f60e6292\") " pod="opendatahub/maas-api-key-cleanup-29656365-9q2nr"
May 21 16:45:00.335785 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:00.335747    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xnjpq\" (UniqueName: \"kubernetes.io/projected/dd2fad01-ba6c-48cc-8001-f9c0f60e6292-kube-api-access-xnjpq\") pod \"maas-api-key-cleanup-29656365-9q2nr\" (UID: \"dd2fad01-ba6c-48cc-8001-f9c0f60e6292\") " pod="opendatahub/maas-api-key-cleanup-29656365-9q2nr"
May 21 16:45:00.349242 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:00.349212    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnjpq\" (UniqueName: \"kubernetes.io/projected/dd2fad01-ba6c-48cc-8001-f9c0f60e6292-kube-api-access-xnjpq\") pod \"maas-api-key-cleanup-29656365-9q2nr\" (UID: \"dd2fad01-ba6c-48cc-8001-f9c0f60e6292\") " pod="opendatahub/maas-api-key-cleanup-29656365-9q2nr"
May 21 16:45:00.462134 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:00.462030    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29656365-9q2nr"
May 21 16:45:00.592038 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:00.592012    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29656365-9q2nr"]
May 21 16:45:00.594647 ip-10-0-136-21 kubenswrapper[2562]: W0521 16:45:00.594621    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd2fad01_ba6c_48cc_8001_f9c0f60e6292.slice/crio-0b150f9f2f05bbd68ed9f3992ff809c97c55186388f0ad51fc1aa2c686be41c2 WatchSource:0}: Error finding container 0b150f9f2f05bbd68ed9f3992ff809c97c55186388f0ad51fc1aa2c686be41c2: Status 404 returned error can't find the container with id 0b150f9f2f05bbd68ed9f3992ff809c97c55186388f0ad51fc1aa2c686be41c2
May 21 16:45:00.674032 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:00.673993    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29656365-9q2nr" event={"ID":"dd2fad01-ba6c-48cc-8001-f9c0f60e6292","Type":"ContainerStarted","Data":"0b150f9f2f05bbd68ed9f3992ff809c97c55186388f0ad51fc1aa2c686be41c2"}
May 21 16:45:03.685131 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:03.685088    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29656365-9q2nr" event={"ID":"dd2fad01-ba6c-48cc-8001-f9c0f60e6292","Type":"ContainerStarted","Data":"6400ec6a16a260b2b24eefa54fd2c4c3de8d0d6d5ef2d8e55d3a247dc59e1113"}
May 21 16:45:03.701701 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:03.701657    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29656365-9q2nr" podStartSLOduration=1.289027585 podStartE2EDuration="3.701644701s" podCreationTimestamp="2026-05-21 16:45:00 +0000 UTC" firstStartedPulling="2026-05-21 16:45:00.596828334 +0000 UTC m=+791.067949292" lastFinishedPulling="2026-05-21 16:45:03.009445444 +0000 UTC m=+793.480566408" observedRunningTime="2026-05-21 16:45:03.700602025 +0000 UTC m=+794.171723004" watchObservedRunningTime="2026-05-21 16:45:03.701644701 +0000 UTC m=+794.172765709"
May 21 16:45:04.689428 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:04.689396    2562 generic.go:358] "Generic (PLEG): container finished" podID="dd2fad01-ba6c-48cc-8001-f9c0f60e6292" containerID="6400ec6a16a260b2b24eefa54fd2c4c3de8d0d6d5ef2d8e55d3a247dc59e1113" exitCode=0
May 21 16:45:04.689888 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:04.689489    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29656365-9q2nr" event={"ID":"dd2fad01-ba6c-48cc-8001-f9c0f60e6292","Type":"ContainerDied","Data":"6400ec6a16a260b2b24eefa54fd2c4c3de8d0d6d5ef2d8e55d3a247dc59e1113"}
May 21 16:45:05.823616 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:05.823585    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29656365-9q2nr"
May 21 16:45:05.886603 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:05.886572    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnjpq\" (UniqueName: \"kubernetes.io/projected/dd2fad01-ba6c-48cc-8001-f9c0f60e6292-kube-api-access-xnjpq\") pod \"dd2fad01-ba6c-48cc-8001-f9c0f60e6292\" (UID: \"dd2fad01-ba6c-48cc-8001-f9c0f60e6292\") "
May 21 16:45:05.888783 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:05.888752    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd2fad01-ba6c-48cc-8001-f9c0f60e6292-kube-api-access-xnjpq" (OuterVolumeSpecName: "kube-api-access-xnjpq") pod "dd2fad01-ba6c-48cc-8001-f9c0f60e6292" (UID: "dd2fad01-ba6c-48cc-8001-f9c0f60e6292"). InnerVolumeSpecName "kube-api-access-xnjpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 16:45:05.988308 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:05.988225    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnjpq\" (UniqueName: \"kubernetes.io/projected/dd2fad01-ba6c-48cc-8001-f9c0f60e6292-kube-api-access-xnjpq\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 16:45:06.696868 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:06.696836    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29656365-9q2nr"
May 21 16:45:06.697024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:06.696846    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29656365-9q2nr" event={"ID":"dd2fad01-ba6c-48cc-8001-f9c0f60e6292","Type":"ContainerDied","Data":"0b150f9f2f05bbd68ed9f3992ff809c97c55186388f0ad51fc1aa2c686be41c2"}
May 21 16:45:06.697024 ip-10-0-136-21 kubenswrapper[2562]: I0521 16:45:06.696953    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b150f9f2f05bbd68ed9f3992ff809c97c55186388f0ad51fc1aa2c686be41c2"
May 21 17:00:00.137833 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.137790    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29656380-m85lk"]
May 21 17:00:00.138338 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.138199    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd2fad01-ba6c-48cc-8001-f9c0f60e6292" containerName="cleanup"
May 21 17:00:00.138338 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.138213    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd2fad01-ba6c-48cc-8001-f9c0f60e6292" containerName="cleanup"
May 21 17:00:00.138338 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.138276    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="dd2fad01-ba6c-48cc-8001-f9c0f60e6292" containerName="cleanup"
May 21 17:00:00.141202 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.141174    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29656380-m85lk"
May 21 17:00:00.143739 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.143718    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-bzpx5\""
May 21 17:00:00.156984 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.156961    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29656380-m85lk"]
May 21 17:00:00.262843 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.262797    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x45xr\" (UniqueName: \"kubernetes.io/projected/f6b73fb2-2003-4a7d-b21d-4183a777c770-kube-api-access-x45xr\") pod \"maas-api-key-cleanup-29656380-m85lk\" (UID: \"f6b73fb2-2003-4a7d-b21d-4183a777c770\") " pod="opendatahub/maas-api-key-cleanup-29656380-m85lk"
May 21 17:00:00.364234 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.364193    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x45xr\" (UniqueName: \"kubernetes.io/projected/f6b73fb2-2003-4a7d-b21d-4183a777c770-kube-api-access-x45xr\") pod \"maas-api-key-cleanup-29656380-m85lk\" (UID: \"f6b73fb2-2003-4a7d-b21d-4183a777c770\") " pod="opendatahub/maas-api-key-cleanup-29656380-m85lk"
May 21 17:00:00.373105 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.373080    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x45xr\" (UniqueName: \"kubernetes.io/projected/f6b73fb2-2003-4a7d-b21d-4183a777c770-kube-api-access-x45xr\") pod \"maas-api-key-cleanup-29656380-m85lk\" (UID: \"f6b73fb2-2003-4a7d-b21d-4183a777c770\") " pod="opendatahub/maas-api-key-cleanup-29656380-m85lk"
May 21 17:00:00.451340 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.451247    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29656380-m85lk"
May 21 17:00:00.582764 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.582738    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29656380-m85lk"]
May 21 17:00:00.585314 ip-10-0-136-21 kubenswrapper[2562]: W0521 17:00:00.585285    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6b73fb2_2003_4a7d_b21d_4183a777c770.slice/crio-d545d0709c8b170b47ab67a8a8cde810340033025361f19b566e10b663a905a3 WatchSource:0}: Error finding container d545d0709c8b170b47ab67a8a8cde810340033025361f19b566e10b663a905a3: Status 404 returned error can't find the container with id d545d0709c8b170b47ab67a8a8cde810340033025361f19b566e10b663a905a3
May 21 17:00:00.587184 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.587165    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 21 17:00:00.723697 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:00.723612    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29656380-m85lk" event={"ID":"f6b73fb2-2003-4a7d-b21d-4183a777c770","Type":"ContainerStarted","Data":"d545d0709c8b170b47ab67a8a8cde810340033025361f19b566e10b663a905a3"}
May 21 17:00:01.728762 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:01.728726    2562 generic.go:358] "Generic (PLEG): container finished" podID="f6b73fb2-2003-4a7d-b21d-4183a777c770" containerID="3ceb6050d47b87f7ed2cc9fe8e20dc4628908a4b03c399b5d96ee3a51ec5bbd1" exitCode=0
May 21 17:00:01.729175 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:01.728777    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29656380-m85lk" event={"ID":"f6b73fb2-2003-4a7d-b21d-4183a777c770","Type":"ContainerDied","Data":"3ceb6050d47b87f7ed2cc9fe8e20dc4628908a4b03c399b5d96ee3a51ec5bbd1"}
May 21 17:00:02.862396 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:02.862375    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29656380-m85lk"
May 21 17:00:02.990103 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:02.989983    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x45xr\" (UniqueName: \"kubernetes.io/projected/f6b73fb2-2003-4a7d-b21d-4183a777c770-kube-api-access-x45xr\") pod \"f6b73fb2-2003-4a7d-b21d-4183a777c770\" (UID: \"f6b73fb2-2003-4a7d-b21d-4183a777c770\") "
May 21 17:00:02.992229 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:02.992202    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6b73fb2-2003-4a7d-b21d-4183a777c770-kube-api-access-x45xr" (OuterVolumeSpecName: "kube-api-access-x45xr") pod "f6b73fb2-2003-4a7d-b21d-4183a777c770" (UID: "f6b73fb2-2003-4a7d-b21d-4183a777c770"). InnerVolumeSpecName "kube-api-access-x45xr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 17:00:03.090810 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:03.090774    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x45xr\" (UniqueName: \"kubernetes.io/projected/f6b73fb2-2003-4a7d-b21d-4183a777c770-kube-api-access-x45xr\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\""
May 21 17:00:03.736307 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:03.736272    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29656380-m85lk" event={"ID":"f6b73fb2-2003-4a7d-b21d-4183a777c770","Type":"ContainerDied","Data":"d545d0709c8b170b47ab67a8a8cde810340033025361f19b566e10b663a905a3"}
May 21 17:00:03.736307 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:03.736308    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d545d0709c8b170b47ab67a8a8cde810340033025361f19b566e10b663a905a3"
May 21 17:00:03.736515 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:00:03.736335    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29656380-m85lk"
May 21 17:06:46.125573 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:46.125543    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29656365-9q2nr_dd2fad01-ba6c-48cc-8001-f9c0f60e6292/cleanup/0.log"
May 21 17:06:46.239568 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:46.239538    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29656380-m85lk_f6b73fb2-2003-4a7d-b21d-4183a777c770/cleanup/0.log"
May 21 17:06:46.726127 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:46.726088    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb_31a41f19-a348-4bff-b005-0b0de9bac330/manager/0.log"
May 21 17:06:46.963387 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:46.963358    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-9q4qf_e2acde12-a04d-4635-abde-9cd950fa97f9/postgres/0.log"
May 21 17:06:48.532614 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:48.532585    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-z6jdq_e80afd22-59f9-4646-bd74-ecf27ecd3b4d/kuadrant-console-plugin/0.log"
May 21 17:06:49.555945 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:49.555914    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5df99877bf-pqrbw_24be0444-c553-410d-89b6-731303d5f2fc/kube-auth-proxy/0.log"
May 21 17:06:49.997158 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:49.997076    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d97cc9d7b-6vfk9_7ab80794-ebdf-4d85-bcb2-748a4da45ae6/router/0.log"
May 21 17:06:50.335973 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:50.335936    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5_985fe487-2eba-4d43-a8f8-17142cb541f9/storage-initializer/0.log"
May 21 17:06:50.343029 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:50.343008    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-zjwk5_985fe487-2eba-4d43-a8f8-17142cb541f9/main/0.log"
May 21 17:06:50.684597 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:50.684494    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj_f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be/storage-initializer/0.log"
May 21 17:06:50.691803 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:50.691781    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccw5mbj_f5f8c0f6-d2be-4ba8-8ab0-dc546ea5a0be/main/0.log"
May 21 17:06:50.797869 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:50.797839    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f_48187a78-76fb-42b9-bdb7-4e2bd1ae6305/storage-initializer/0.log"
May 21 17:06:50.804462 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:50.804439    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-jtf9f_48187a78-76fb-42b9-bdb7-4e2bd1ae6305/main/0.log"
May 21 17:06:57.356089 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:57.356046    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9tf8m_336f84d9-849e-405a-8c9d-a8ae4c452a9d/global-pull-secret-syncer/0.log"
May 21 17:06:57.485307 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:57.485276    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ffwpm_5d277af9-bea0-4ffb-adbe-e5c7930e51f4/konnectivity-agent/0.log"
May 21 17:06:57.591674 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:06:57.591649    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-21.ec2.internal_02780303cf6bfe2b07d5904f0becc31e/haproxy/0.log"
May 21 17:07:02.146039 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:02.146006    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-z6jdq_e80afd22-59f9-4646-bd74-ecf27ecd3b4d/kuadrant-console-plugin/0.log"
May 21 17:07:04.049934 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.049904    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6c75959455-hswx2_39cbf8cb-d27b-4e6c-8abc-603723d275fc/cluster-monitoring-operator/0.log"
May 21 17:07:04.268837 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.268813    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lmhtg_b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9/node-exporter/0.log"
May 21 17:07:04.288264 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.288227    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lmhtg_b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9/kube-rbac-proxy/0.log"
May 21 17:07:04.308097 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.308076    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lmhtg_b0cc0f3e-7e26-4759-a3f0-9001acc6c2a9/init-textfile/0.log"
May 21 17:07:04.429075 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.429021    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-hjzqz_d9139e24-9adb-4c84-a365-6a80796df35b/kube-rbac-proxy-main/0.log"
May 21 17:07:04.448189 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.448148    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-hjzqz_d9139e24-9adb-4c84-a365-6a80796df35b/kube-rbac-proxy-self/0.log"
May 21 17:07:04.471715 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.471694    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-hjzqz_d9139e24-9adb-4c84-a365-6a80796df35b/openshift-state-metrics/0.log"
May 21 17:07:04.507140 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.507112    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7ff357c-5f9a-41fa-9c30-a71accf31969/prometheus/0.log"
May 21 17:07:04.525759 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.525685    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7ff357c-5f9a-41fa-9c30-a71accf31969/config-reloader/0.log"
May 21 17:07:04.555380 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.555351    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7ff357c-5f9a-41fa-9c30-a71accf31969/thanos-sidecar/0.log"
May 21 17:07:04.577831 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.577807    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7ff357c-5f9a-41fa-9c30-a71accf31969/kube-rbac-proxy-web/0.log"
May 21 17:07:04.596347 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.596312    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7ff357c-5f9a-41fa-9c30-a71accf31969/kube-rbac-proxy/0.log"
May 21 17:07:04.615708 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.615685    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7ff357c-5f9a-41fa-9c30-a71accf31969/kube-rbac-proxy-thanos/0.log"
May 21 17:07:04.635816 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.635792    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7ff357c-5f9a-41fa-9c30-a71accf31969/init-config-reloader/0.log"
May 21 17:07:04.661002 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.660975    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-d8qpk_3967222d-b640-482f-ae29-8bc9102f7067/prometheus-operator/0.log"
May 21 17:07:04.679293 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.679268    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-d8qpk_3967222d-b640-482f-ae29-8bc9102f7067/kube-rbac-proxy/0.log"
May 21 17:07:04.807785 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.807713    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77b996bc84-rtsb6_1f0704d0-1996-4674-a74c-d0c3cb8710d0/thanos-query/0.log"
May 21 17:07:04.825884 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.825859    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77b996bc84-rtsb6_1f0704d0-1996-4674-a74c-d0c3cb8710d0/kube-rbac-proxy-web/0.log"
May 21 17:07:04.844587 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.844560    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77b996bc84-rtsb6_1f0704d0-1996-4674-a74c-d0c3cb8710d0/kube-rbac-proxy/0.log"
May 21 17:07:04.868678 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.868645    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77b996bc84-rtsb6_1f0704d0-1996-4674-a74c-d0c3cb8710d0/prom-label-proxy/0.log"
May 21 17:07:04.883277 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.883245    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77b996bc84-rtsb6_1f0704d0-1996-4674-a74c-d0c3cb8710d0/kube-rbac-proxy-rules/0.log"
May 21 17:07:04.901507 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:04.901481    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77b996bc84-rtsb6_1f0704d0-1996-4674-a74c-d0c3cb8710d0/kube-rbac-proxy-metrics/0.log"
May 21 17:07:06.135403 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.135368    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"]
May 21 17:07:06.135806 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.135723    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6b73fb2-2003-4a7d-b21d-4183a777c770" containerName="cleanup"
May 21 17:07:06.135806 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.135734    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6b73fb2-2003-4a7d-b21d-4183a777c770" containerName="cleanup"
May 21 17:07:06.135806 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.135784    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="f6b73fb2-2003-4a7d-b21d-4183a777c770" containerName="cleanup"
May 21 17:07:06.139164 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.139139    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.141621 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.141595    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2sr2x\"/\"openshift-service-ca.crt\""
May 21 17:07:06.141761 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.141699    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2sr2x\"/\"default-dockercfg-6xm2l\""
May 21 17:07:06.142937 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.142912    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2sr2x\"/\"kube-root-ca.crt\""
May 21 17:07:06.148312 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.148279    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"]
May 21 17:07:06.184446 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.184411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-lib-modules\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.184637 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.184465    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-podres\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.184637 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.184495    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-proc\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.184637 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.184581    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-sys\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.184637 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.184622    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcgc8\" (UniqueName: \"kubernetes.io/projected/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-kube-api-access-qcgc8\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.285486 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.285449    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-sys\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.285651 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.285499    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qcgc8\" (UniqueName: \"kubernetes.io/projected/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-kube-api-access-qcgc8\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.285651 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.285546    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-lib-modules\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.285651 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.285576    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-sys\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.285651 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.285580    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-podres\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.285859 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.285657    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-proc\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.285859 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.285711    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-lib-modules\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.285859 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.285738    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-podres\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.285859 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.285767    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-proc\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.294043 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.294013    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcgc8\" (UniqueName: \"kubernetes.io/projected/ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8-kube-api-access-qcgc8\") pod \"perf-node-gather-daemonset-6xmt4\" (UID: \"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8\") " pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.450074 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.449949    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:06.583877 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.583852    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"]
May 21 17:07:06.586346 ip-10-0-136-21 kubenswrapper[2562]: W0521 17:07:06.586310    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podea814b3a_cd12_4a0d_9d54_d6a06ca0dbe8.slice/crio-8b77ce604841b59ac1a301e67a0a30c976686074ffaad7cb2c98985699a3fe1f WatchSource:0}: Error finding container 8b77ce604841b59ac1a301e67a0a30c976686074ffaad7cb2c98985699a3fe1f: Status 404 returned error can't find the container with id 8b77ce604841b59ac1a301e67a0a30c976686074ffaad7cb2c98985699a3fe1f
May 21 17:07:06.588187 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:06.588163    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 21 17:07:07.171831 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:07.171798    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4" event={"ID":"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8","Type":"ContainerStarted","Data":"a09f83c4544645dee4cc080e53f7d6de7282615eb086e376a7362cec208419db"}
May 21 17:07:07.171831 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:07.171836    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4" event={"ID":"ea814b3a-cd12-4a0d-9d54-d6a06ca0dbe8","Type":"ContainerStarted","Data":"8b77ce604841b59ac1a301e67a0a30c976686074ffaad7cb2c98985699a3fe1f"}
May 21 17:07:07.172296 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:07.171886    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:07.187591 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:07.187540    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4" podStartSLOduration=1.187525205 podStartE2EDuration="1.187525205s" podCreationTimestamp="2026-05-21 17:07:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 17:07:07.186000518 +0000 UTC m=+2117.657121498" watchObservedRunningTime="2026-05-21 17:07:07.187525205 +0000 UTC m=+2117.658646184"
May 21 17:07:08.207144 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:08.207111    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dblrj_52b56561-b24c-4152-ba4d-c89944871373/dns/0.log"
May 21 17:07:08.226333 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:08.226307    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dblrj_52b56561-b24c-4152-ba4d-c89944871373/kube-rbac-proxy/0.log"
May 21 17:07:08.309642 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:08.309615    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pmgp7_118ef813-488d-4e5f-a9a5-9e6579c03825/dns-node-resolver/0.log"
May 21 17:07:08.815315 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:08.815286    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jslnj_24b5a0be-656b-4d0f-8b47-7f43220f88a8/node-ca/0.log"
May 21 17:07:09.727165 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:09.727133    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5df99877bf-pqrbw_24be0444-c553-410d-89b6-731303d5f2fc/kube-auth-proxy/0.log"
May 21 17:07:09.879281 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:09.879248    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d97cc9d7b-6vfk9_7ab80794-ebdf-4d85-bcb2-748a4da45ae6/router/0.log"
May 21 17:07:10.399476 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:10.399448    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7kpdj_96db6d8f-de2e-4b26-8601-d4994f37e209/serve-healthcheck-canary/0.log"
May 21 17:07:10.958456 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:10.958432    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5nr7k_e24c6d4b-077c-49ee-b96c-47d0ffec0f61/kube-rbac-proxy/0.log"
May 21 17:07:10.979034 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:10.979009    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5nr7k_e24c6d4b-077c-49ee-b96c-47d0ffec0f61/exporter/0.log"
May 21 17:07:11.005445 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:11.005422    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5nr7k_e24c6d4b-077c-49ee-b96c-47d0ffec0f61/extractor/0.log"
May 21 17:07:13.097605 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:13.097574    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29656365-9q2nr_dd2fad01-ba6c-48cc-8001-f9c0f60e6292/cleanup/0.log"
May 21 17:07:13.118164 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:13.118132    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29656380-m85lk_f6b73fb2-2003-4a7d-b21d-4183a777c770/cleanup/0.log"
May 21 17:07:13.186568 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:13.186541    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2sr2x/perf-node-gather-daemonset-6xmt4"
May 21 17:07:13.280246 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:13.280210    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f6f66c9f4-bg2mb_31a41f19-a348-4bff-b005-0b0de9bac330/manager/0.log"
May 21 17:07:13.324045 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:13.324015    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-9q4qf_e2acde12-a04d-4635-abde-9cd950fa97f9/postgres/0.log"
May 21 17:07:14.420015 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:14.419985    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-67b4fd8c8c-ld9cp_0578f9da-02bf-4346-bdf9-69a1284f912f/manager/0.log"
May 21 17:07:20.102851 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:20.102821    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6pj29_b8141c2f-6c4c-4246-b69f-2e04049cd7e6/kube-multus/0.log"
May 21 17:07:20.144292 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:20.144260    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xqm_efbc6e4e-c86f-40cd-bab6-db67d363b367/kube-multus-additional-cni-plugins/0.log"
May 21 17:07:20.181242 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:20.181211    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xqm_efbc6e4e-c86f-40cd-bab6-db67d363b367/egress-router-binary-copy/0.log"
May 21 17:07:20.223281 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:20.223249    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xqm_efbc6e4e-c86f-40cd-bab6-db67d363b367/cni-plugins/0.log"
May 21 17:07:20.255275 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:20.255247    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xqm_efbc6e4e-c86f-40cd-bab6-db67d363b367/bond-cni-plugin/0.log"
May 21 17:07:20.292071 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:20.292022    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xqm_efbc6e4e-c86f-40cd-bab6-db67d363b367/routeoverride-cni/0.log"
May 21 17:07:20.327413 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:20.327388    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xqm_efbc6e4e-c86f-40cd-bab6-db67d363b367/whereabouts-cni-bincopy/0.log"
May 21 17:07:20.362310 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:20.362233    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xqm_efbc6e4e-c86f-40cd-bab6-db67d363b367/whereabouts-cni/0.log"
May 21 17:07:21.018936 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:21.018906    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hrrn5_bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f/network-metrics-daemon/0.log"
May 21 17:07:21.036283 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:21.036256    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hrrn5_bd3e235e-1b4f-47b0-ad4f-527c6d3b2f9f/kube-rbac-proxy/0.log"
May 21 17:07:21.940348 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:21.940316    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9th77_8279711d-7202-4a89-8ec3-cb7cdc2024ba/ovn-controller/0.log"
May 21 17:07:21.967097 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:21.967065    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9th77_8279711d-7202-4a89-8ec3-cb7cdc2024ba/ovn-acl-logging/0.log"
May 21 17:07:21.984456 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:21.984437    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9th77_8279711d-7202-4a89-8ec3-cb7cdc2024ba/kube-rbac-proxy-node/0.log"
May 21 17:07:22.003424 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:22.003398    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9th77_8279711d-7202-4a89-8ec3-cb7cdc2024ba/kube-rbac-proxy-ovn-metrics/0.log"
May 21 17:07:22.020610 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:22.020586    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9th77_8279711d-7202-4a89-8ec3-cb7cdc2024ba/northd/0.log"
May 21 17:07:22.038738 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:22.038714    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9th77_8279711d-7202-4a89-8ec3-cb7cdc2024ba/nbdb/0.log"
May 21 17:07:22.060797 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:22.060770    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9th77_8279711d-7202-4a89-8ec3-cb7cdc2024ba/sbdb/0.log"
May 21 17:07:22.151696 ip-10-0-136-21 kubenswrapper[2562]: I0521 17:07:22.151664    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9th77_8279711d-7202-4a89-8ec3-cb7cdc2024ba/ovnkube-controller/0.log"