Apr 28 19:16:53.028653 ip-10-0-131-5 systemd[1]: Starting Kubernetes Kubelet...
Apr 28 19:16:53.617545 ip-10-0-131-5 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Apr 28 19:16:53.617545 ip-10-0-131-5 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Apr 28 19:16:53.617545 ip-10-0-131-5 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Apr 28 19:16:53.617545 ip-10-0-131-5 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Apr 28 19:16:53.617545 ip-10-0-131-5 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Apr 28 19:16:53.622696 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.622576    2569 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Apr 28 19:16:53.629347 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629319    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 28 19:16:53.629347 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629340    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 28 19:16:53.629347 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629344    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 28 19:16:53.629347 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629347    2569 feature_gate.go:328] unrecognized feature gate: Example2
Apr 28 19:16:53.629347 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629350    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 28 19:16:53.629347 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629353    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 28 19:16:53.629347 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629356    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629359    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629362    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629364    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629367    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629370    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629372    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629375    2569 feature_gate.go:328] unrecognized feature gate: Example
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629379    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629381    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629384    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629386    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629389    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629391    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629394    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629408    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629416    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629418    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629421    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629424    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 28 19:16:53.629616 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629426    2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629429    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629432    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629435    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629438    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629441    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629443    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629446    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629449    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629451    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629454    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629457    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629460    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629463    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629466    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629468    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629471    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629473    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629476    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 28 19:16:53.630096 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629479    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629484    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629487    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629490    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629493    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629496    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629498    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629501    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629504    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629506    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629509    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629512    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629514    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629517    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629519    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629522    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629526    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629530    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629533    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 28 19:16:53.630642 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629536    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629538    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629541    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629544    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629547    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629549    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629552    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629556    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629559    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629561    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629564    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629567    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629569    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629571    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629580    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629583    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629586    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629588    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629591    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629594    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 28 19:16:53.631107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629596    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.629599    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630003    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630008    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630011    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630013    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630016    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630019    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630022    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630025    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630028    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630030    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630033    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630036    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630038    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630044    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630047    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630050    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630052    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 28 19:16:53.631609 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630055    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630058    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630060    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630063    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630065    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630068    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630070    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630078    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630081    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630084    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630086    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630089    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630091    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630094    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630096    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630100    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630104    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630107    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630109    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 28 19:16:53.632061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630112    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630115    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630118    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630120    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630123    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630126    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630128    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630130    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630133    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630135    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630138    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630141    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630143    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630146    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630149    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630152    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630155    2569 feature_gate.go:328] unrecognized feature gate: Example2
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630157    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630159    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630162    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 28 19:16:53.632541 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630164    2569 feature_gate.go:328] unrecognized feature gate: Example
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630173    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630175    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630177    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630180    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630183    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630186    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630188    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630191    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630193    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630196    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630198    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630201    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630203    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630206    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630209    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630212    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630214    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630217    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630219    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 28 19:16:53.633023 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630221    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630226    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630229    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630233    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630235    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630238    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630241    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630244    2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630248    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.630251    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632628    2569 flags.go:64] FLAG: --address="0.0.0.0"
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632642    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632651    2569 flags.go:64] FLAG: --anonymous-auth="true"
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632656    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632666    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632669    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632673    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632678    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632681    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632684    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632688    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Apr 28 19:16:53.633520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632691    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632694    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632697    2569 flags.go:64] FLAG: --cgroup-root=""
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632700    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632703    2569 flags.go:64] FLAG: --client-ca-file=""
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632706    2569 flags.go:64] FLAG: --cloud-config=""
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632709    2569 flags.go:64] FLAG: --cloud-provider="external"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632712    2569 flags.go:64] FLAG: --cluster-dns="[]"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632718    2569 flags.go:64] FLAG: --cluster-domain=""
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632721    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632724    2569 flags.go:64] FLAG: --config-dir=""
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632727    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632730    2569 flags.go:64] FLAG: --container-log-max-files="5"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632735    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632737    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632740    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632744    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632747    2569 flags.go:64] FLAG: --contention-profiling="false"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632750    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632753    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632756    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632759    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632763    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632766    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632769    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Apr 28 19:16:53.634026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632772    2569 flags.go:64] FLAG: --enable-load-reader="false"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632780    2569 flags.go:64] FLAG: --enable-server="true"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632783    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632790    2569 flags.go:64] FLAG: --event-burst="100"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632794    2569 flags.go:64] FLAG: --event-qps="50"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632796    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632799    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632808    2569 flags.go:64] FLAG: --eviction-hard=""
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632812    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632814    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632817    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632821    2569 flags.go:64] FLAG: --eviction-soft=""
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632824    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632827    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632830    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632833    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632835    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632838    2569 flags.go:64] FLAG: --fail-swap-on="true"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632841    2569 flags.go:64] FLAG: --feature-gates=""
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632845    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632848    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632851    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632854    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632857    2569 flags.go:64] FLAG: --healthz-port="10248"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632860    2569 flags.go:64] FLAG: --help="false"
Apr 28 19:16:53.634654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632863    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-131-5.ec2.internal"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632866    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632869    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632872    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632875    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632878    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632882    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632884    2569 flags.go:64] FLAG: --image-service-endpoint=""
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632887    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632899    2569 flags.go:64] FLAG: --kube-api-burst="100"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632902    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632906    2569 flags.go:64] FLAG: --kube-api-qps="50"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632909    2569 flags.go:64] FLAG: --kube-reserved=""
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632912    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632915    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632918    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632921    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632923    2569 flags.go:64] FLAG: --lock-file=""
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632926    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632929    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632932    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632937    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632940    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632943    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Apr 28 19:16:53.635282 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632945    2569 flags.go:64] FLAG: --logging-format="text"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632948    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632951    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632954    2569 flags.go:64] FLAG: --manifest-url=""
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632957    2569 flags.go:64] FLAG: --manifest-url-header=""
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632961    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632964    2569 flags.go:64] FLAG: --max-open-files="1000000"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632968    2569 flags.go:64] FLAG: --max-pods="110"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632972    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632975    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632978    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632981    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632984    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632987    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632989    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.632997    2569 flags.go:64] FLAG: --node-status-max-images="50"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633000    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633003    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633011    2569 flags.go:64] FLAG: --pod-cidr=""
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633014    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c8cfe89231412ff3ee8cb6207fa0be33cad0f08e88c9c0f1e9f7e8c6f14d6715"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633019    2569 flags.go:64] FLAG: --pod-manifest-path=""
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633022    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633025    2569 flags.go:64] FLAG: --pods-per-core="0"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633028    2569 flags.go:64] FLAG: --port="10250"
Apr 28 19:16:53.635928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633031    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633034    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0936ad4c1bcc1c9f5"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633038    2569 flags.go:64] FLAG: --qos-reserved=""
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633040    2569 flags.go:64] FLAG: --read-only-port="10255"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633043    2569 flags.go:64] FLAG: --register-node="true"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633046    2569 flags.go:64] FLAG: --register-schedulable="true"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633049    2569 flags.go:64] FLAG: --register-with-taints=""
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633053    2569 flags.go:64] FLAG: --registry-burst="10"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633055    2569 flags.go:64] FLAG: --registry-qps="5"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633058    2569 flags.go:64] FLAG: --reserved-cpus=""
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633060    2569 flags.go:64] FLAG: --reserved-memory=""
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633064    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633067    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633070    2569 flags.go:64] FLAG: --rotate-certificates="false"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633072    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633075    2569 flags.go:64] FLAG: --runonce="false"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633078    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633082    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633084    2569 flags.go:64] FLAG: --seccomp-default="false"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633087    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633090    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633093    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633095    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633098    2569 flags.go:64] FLAG: --storage-driver-password="root"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633101    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633104    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Apr 28 19:16:53.636512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633108    2569 flags.go:64] FLAG: --storage-driver-user="root"
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633117    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633120    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633123    2569 flags.go:64] FLAG: --system-cgroups=""
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633126    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633131    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633133    2569 flags.go:64] FLAG: --tls-cert-file=""
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633136    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633142    2569 flags.go:64] FLAG: --tls-min-version=""
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633145    2569 flags.go:64] FLAG: --tls-private-key-file=""
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633147    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633150    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633153    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633156    2569 flags.go:64] FLAG: --v="2"
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633160    2569 flags.go:64] FLAG: --version="false"
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633164    2569 flags.go:64] FLAG: --vmodule=""
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633169    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.633172    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633283    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633286    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633289    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633291    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633294    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633297    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 28 19:16:53.637128 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633299    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633302    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633304    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633307    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633309    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633312    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633315    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633317    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633321    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633324    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633332    2569 feature_gate.go:328] unrecognized feature gate: Example
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633335    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633337    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633339    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633342    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633344    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633347    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633349    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633352    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633354    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 28 19:16:53.637755 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633357    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633359    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633362    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633365    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633367    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633370    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633372    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633375    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633377    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633380    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633382    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633385    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633387    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633390    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633392    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633395    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633413    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633416    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633420    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 28 19:16:53.638252 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633424    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633432    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633434    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633437    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633445    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633448    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633450    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633453    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633455    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633458    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633460    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633463    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633465    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633468    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633470    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633473    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633475    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633478    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633480    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633483    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 28 19:16:53.638743 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633485    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633488    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633490    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633494    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633498    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633501    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633504    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633507    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633509    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633512    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633514    2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633516    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633519    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633523    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633526    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633529    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633531    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633539    2569 feature_gate.go:328] unrecognized feature gate: Example2
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633542    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 28 19:16:53.639278 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633544    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 28 19:16:53.639803 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.633547    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 28 19:16:53.639803 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.634597    2569 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Apr 28 19:16:53.642447 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.642426    2569 server.go:530] "Kubelet version" kubeletVersion="v1.33.9"
Apr 28 19:16:53.642490 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.642448    2569 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Apr 28 19:16:53.642519 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642509    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 28 19:16:53.642519 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642515    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 28 19:16:53.642519 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642519    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642522    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642526    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642528    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642531    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642534    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642536    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642539    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642542    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642544    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642547    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642550    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642552    2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642555    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642557    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642560    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642562    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642566    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642570    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 28 19:16:53.642600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642573    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642576    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642579    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642582    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642586    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642588    2569 feature_gate.go:328] unrecognized feature gate: Example
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642591    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642593    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642596    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642599    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642601    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642604    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642607    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642610    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642613    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642616    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642618    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642621    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642623    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642627    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 28 19:16:53.643061 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642630    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642633    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642636    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642638    2569 feature_gate.go:328] unrecognized feature gate: Example2
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642641    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642643    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642646    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642648    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642651    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642654    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642657    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642659    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642661    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642664    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642668    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642670    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642673    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642677    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642679    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642682    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 28 19:16:53.643600 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642684    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642687    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642689    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642692    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642694    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642697    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642699    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642702    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642704    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642707    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642709    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642712    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642715    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642717    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642720    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642722    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642731    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642734    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642737    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 28 19:16:53.644081 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642739    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642742    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642744    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642747    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642749    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642752    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.642757    2569 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642863    2569 feature_gate.go:328] unrecognized feature gate: Example2
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642867    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642871    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642874    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642880    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642883    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642886    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642889    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642892    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 28 19:16:53.644559 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642895    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642898    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642900    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642903    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642906    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642909    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642911    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642914    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642916    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642919    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642921    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642924    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642926    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642928    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642931    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642934    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642938    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642941    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642943    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 28 19:16:53.644937 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642946    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642948    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642951    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642953    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642955    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642958    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642961    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642963    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642965    2569 feature_gate.go:328] unrecognized feature gate: Example
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642969    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642971    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642974    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642976    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642978    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642981    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642983    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642986    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642988    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642991    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642993    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 28 19:16:53.645424 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642995    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.642998    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643000    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643003    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643005    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643008    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643010    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643013    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643015    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643018    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643020    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643022    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643025    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643027    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643030    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643032    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643035    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643038    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643042    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 28 19:16:53.645921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643045    2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643048    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643051    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643054    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643057    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643059    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643062    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643065    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643067    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643070    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643073    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643076    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643078    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643081    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643083    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643085    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643088    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643090    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 28 19:16:53.646382 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:53.643093    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 28 19:16:53.647025 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.643098    2569 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Apr 28 19:16:53.647025 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.644194    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Apr 28 19:16:53.647089 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.647035    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Apr 28 19:16:53.647911 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.647898    2569 server.go:1019] "Starting client certificate rotation"
Apr 28 19:16:53.648014 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.647998    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Apr 28 19:16:53.648054 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.648044    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Apr 28 19:16:53.684705 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.684683    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Apr 28 19:16:53.689481 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.689449    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Apr 28 19:16:53.714093 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.714073    2569 log.go:25] "Validated CRI v1 runtime API"
Apr 28 19:16:53.724936 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.724918    2569 log.go:25] "Validated CRI v1 image API"
Apr 28 19:16:53.726733 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.726714    2569 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Apr 28 19:16:53.729075 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.729060    2569 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Apr 28 19:16:53.730006 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.729988    2569 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d698f72b-774c-4af7-871f-cb138a3e4be8:/dev/nvme0n1p4 e0cf0fae-620c-41ae-aed9-bb4c52f87b35:/dev/nvme0n1p3]
Apr 28 19:16:53.730052 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.730006    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Apr 28 19:16:53.737041 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.736924    2569 manager.go:217] Machine: {Timestamp:2026-04-28 19:16:53.733885907 +0000 UTC m=+0.558226007 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096779 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26844708b546ed78e134def3f11612 SystemUUID:ec268447-08b5-46ed-78e1-34def3f11612 BootID:61c24d2b-152e-4d87-b2b5-f45a8fdc056e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:18:e0:d7:fe:bf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:18:e0:d7:fe:bf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:ec:71:58:45:f9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Apr 28 19:16:53.737041 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.737038    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Apr 28 19:16:53.737170 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.737157    2569 manager.go:233] Version: {KernelVersion:5.14.0-570.107.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260414-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Apr 28 19:16:53.741246 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.741220    2569 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Apr 28 19:16:53.741459 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.741248    2569 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-5.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Apr 28 19:16:53.741503 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.741469    2569 topology_manager.go:138] "Creating topology manager with none policy"
Apr 28 19:16:53.741503 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.741478    2569 container_manager_linux.go:306] "Creating device plugin manager"
Apr 28 19:16:53.741503 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.741491    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Apr 28 19:16:53.741585 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.741507    2569 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Apr 28 19:16:53.744528 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.744516    2569 state_mem.go:36] "Initialized new in-memory state store"
Apr 28 19:16:53.744644 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.744635    2569 server.go:1267] "Using root directory" path="/var/lib/kubelet"
Apr 28 19:16:53.748978 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.748968    2569 kubelet.go:491] "Attempting to sync node with API server"
Apr 28 19:16:53.749017 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.748984    2569 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
Apr 28 19:16:53.749017 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.748998    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Apr 28 19:16:53.749017 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.749008    2569 kubelet.go:397] "Adding apiserver pod source"
Apr 28 19:16:53.749131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.749019    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Apr 28 19:16:53.750641 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.750629    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Apr 28 19:16:53.750698 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.750648    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Apr 28 19:16:53.754652 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.754635    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bbjn2"
Apr 28 19:16:53.755315 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.755302    2569 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.10-2.rhaos4.20.gita4d0894.el9" apiVersion="v1"
Apr 28 19:16:53.758127 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.758110    2569 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Apr 28 19:16:53.762148 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:53.762124    2569 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-5.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Apr 28 19:16:53.762195 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:53.762136    2569 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Apr 28 19:16:53.762195 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.762175    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Apr 28 19:16:53.762195 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.762189    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Apr 28 19:16:53.762195 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.762196    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Apr 28 19:16:53.762297 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.762201    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Apr 28 19:16:53.762297 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.762208    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Apr 28 19:16:53.762297 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.762214    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Apr 28 19:16:53.762297 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.762219    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Apr 28 19:16:53.762297 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.762241    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Apr 28 19:16:53.762297 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.762248    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Apr 28 19:16:53.762297 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.762253    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Apr 28 19:16:53.762297 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.762265    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Apr 28 19:16:53.762297 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.762274    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Apr 28 19:16:53.763866 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.763853    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bbjn2"
Apr 28 19:16:53.763900 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.763870    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Apr 28 19:16:53.763900 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.763879    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
Apr 28 19:16:53.767905 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.767890    2569 watchdog_linux.go:99] "Systemd watchdog is not enabled"
Apr 28 19:16:53.767952 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.767945    2569 server.go:1295] "Started kubelet"
Apr 28 19:16:53.768055 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.768026    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Apr 28 19:16:53.768146 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.768100    2569 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Apr 28 19:16:53.768187 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.768172    2569 server_v1.go:47] "podresources" method="list" useActivePods=true
Apr 28 19:16:53.768913 ip-10-0-131-5 systemd[1]: Started Kubernetes Kubelet.
Apr 28 19:16:53.770627 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.770612    2569 server.go:317] "Adding debug handlers to kubelet server"
Apr 28 19:16:53.776799 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.776769    2569 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Apr 28 19:16:53.785800 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.785781    2569 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-131-5.ec2.internal" not found
Apr 28 19:16:53.798556 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:53.798538    2569 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Apr 28 19:16:53.799079 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.799052    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Apr 28 19:16:53.800276 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.800261    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Apr 28 19:16:53.801719 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.801700    2569 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
Apr 28 19:16:53.801719 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.801704    2569 volume_manager.go:295] "The desired_state_of_world populator starts"
Apr 28 19:16:53.801855 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.801731    2569 volume_manager.go:297] "Starting Kubelet Volume Manager"
Apr 28 19:16:53.801855 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.801833    2569 reconstruct.go:97] "Volume reconstruction finished"
Apr 28 19:16:53.801855 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.801842    2569 reconciler.go:26] "Reconciler: start to sync state"
Apr 28 19:16:53.801958 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:53.801884    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:53.802891 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.802872    2569 factory.go:55] Registering systemd factory
Apr 28 19:16:53.802966 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.802895    2569 factory.go:223] Registration of the systemd container factory successfully
Apr 28 19:16:53.803096 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.803084    2569 factory.go:153] Registering CRI-O factory
Apr 28 19:16:53.803133 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.803098    2569 factory.go:223] Registration of the crio container factory successfully
Apr 28 19:16:53.803166 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.803145    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Apr 28 19:16:53.803211 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.803166    2569 factory.go:103] Registering Raw factory
Apr 28 19:16:53.803211 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.803176    2569 manager.go:1196] Started watching for new ooms in manager
Apr 28 19:16:53.803321 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.803304    2569 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 28 19:16:53.803662 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.803647    2569 manager.go:319] Starting recovery of all containers
Apr 28 19:16:53.804299 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.804281    2569 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-131-5.ec2.internal" not found
Apr 28 19:16:53.806902 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:53.806872    2569 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-5.ec2.internal\" not found" node="ip-10-0-131-5.ec2.internal"
Apr 28 19:16:53.811230 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.811212    2569 manager.go:324] Recovery completed
Apr 28 19:16:53.815163 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.815150    2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 28 19:16:53.818532 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.818518    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory"
Apr 28 19:16:53.818600 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.818543    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure"
Apr 28 19:16:53.818600 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.818554    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID"
Apr 28 19:16:53.819001 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.818989    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Apr 28 19:16:53.819044 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.819002    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Apr 28 19:16:53.819044 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.819039    2569 state_mem.go:36] "Initialized new in-memory state store"
Apr 28 19:16:53.824699 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.824688    2569 policy_none.go:49] "None policy: Start"
Apr 28 19:16:53.824742 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.824708    2569 memory_manager.go:186] "Starting memorymanager" policy="None"
Apr 28 19:16:53.824742 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.824717    2569 state_mem.go:35] "Initializing new in-memory state store"
Apr 28 19:16:53.864806 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.864791    2569 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-131-5.ec2.internal" not found
Apr 28 19:16:53.879097 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.867996    2569 manager.go:341] "Starting Device Plugin manager"
Apr 28 19:16:53.879097 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:53.868028    2569 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Apr 28 19:16:53.879097 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.868037    2569 server.go:85] "Starting device plugin registration server"
Apr 28 19:16:53.879097 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.868253    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Apr 28 19:16:53.879097 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.868262    2569 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Apr 28 19:16:53.879097 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.868391    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Apr 28 19:16:53.879097 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.868473    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Apr 28 19:16:53.879097 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.868482    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Apr 28 19:16:53.879097 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:53.868970    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Apr 28 19:16:53.879097 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:53.869009    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:53.936603 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.936574    2569 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
Apr 28 19:16:53.937748 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.937728    2569 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
Apr 28 19:16:53.937845 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.937756    2569 status_manager.go:230] "Starting to sync pod status with apiserver"
Apr 28 19:16:53.937845 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.937777    2569 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Apr 28 19:16:53.937845 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.937786    2569 kubelet.go:2451] "Starting kubelet main sync loop"
Apr 28 19:16:53.937992 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:53.937888    2569 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Apr 28 19:16:53.940445 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.940385    2569 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 28 19:16:53.968482 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.968469    2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 28 19:16:53.969267 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.969249    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory"
Apr 28 19:16:53.969358 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.969282    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure"
Apr 28 19:16:53.969358 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.969293    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID"
Apr 28 19:16:53.969358 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.969316    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-5.ec2.internal"
Apr 28 19:16:53.977318 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:53.977301    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-5.ec2.internal"
Apr 28 19:16:53.977395 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:53.977325    2569 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-5.ec2.internal\": node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:53.989829 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:53.989812    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:54.038518 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.038486    2569 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal"]
Apr 28 19:16:54.038596 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.038556    2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 28 19:16:54.040009 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.039997    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory"
Apr 28 19:16:54.040068 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.040023    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure"
Apr 28 19:16:54.040068 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.040035    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID"
Apr 28 19:16:54.041233 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.041222    2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 28 19:16:54.041381 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.041367    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.041434 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.041395    2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 28 19:16:54.042112 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.042090    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory"
Apr 28 19:16:54.042170 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.042112    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory"
Apr 28 19:16:54.042170 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.042120    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure"
Apr 28 19:16:54.042170 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.042131    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure"
Apr 28 19:16:54.042170 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.042134    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID"
Apr 28 19:16:54.042170 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.042141    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID"
Apr 28 19:16:54.043296 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.043282    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.043336 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.043306    2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 28 19:16:54.049323 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.049308    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory"
Apr 28 19:16:54.049413 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.049333    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure"
Apr 28 19:16:54.049413 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.049343    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID"
Apr 28 19:16:54.059941 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:54.059923    2569 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-5.ec2.internal\" not found" node="ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.065642 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:54.065627    2569 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-5.ec2.internal\" not found" node="ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.090811 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:54.090792    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:54.103144 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.103126    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b50e7600b47f76668e274e014c99f3ac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal\" (UID: \"b50e7600b47f76668e274e014c99f3ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.103213 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.103149    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b50e7600b47f76668e274e014c99f3ac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal\" (UID: \"b50e7600b47f76668e274e014c99f3ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.103213 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.103166    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55d83b29c984d704f2c407ca2173be08-config\") pod \"kube-apiserver-proxy-ip-10-0-131-5.ec2.internal\" (UID: \"55d83b29c984d704f2c407ca2173be08\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.191812 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:54.191741    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:54.204190 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.204170    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b50e7600b47f76668e274e014c99f3ac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal\" (UID: \"b50e7600b47f76668e274e014c99f3ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.204260 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.204195    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b50e7600b47f76668e274e014c99f3ac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal\" (UID: \"b50e7600b47f76668e274e014c99f3ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.204260 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.204213    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55d83b29c984d704f2c407ca2173be08-config\") pod \"kube-apiserver-proxy-ip-10-0-131-5.ec2.internal\" (UID: \"55d83b29c984d704f2c407ca2173be08\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.204324 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.204279    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b50e7600b47f76668e274e014c99f3ac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal\" (UID: \"b50e7600b47f76668e274e014c99f3ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.204324 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.204291    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55d83b29c984d704f2c407ca2173be08-config\") pod \"kube-apiserver-proxy-ip-10-0-131-5.ec2.internal\" (UID: \"55d83b29c984d704f2c407ca2173be08\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.204324 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.204284    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b50e7600b47f76668e274e014c99f3ac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal\" (UID: \"b50e7600b47f76668e274e014c99f3ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.291986 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:54.291953    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:54.361525 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.361491    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.368134 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.368115    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:54.392881 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:54.392854    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:54.493460 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:54.493368    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:54.593948 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:54.593914    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:54.647416 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.647369    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Apr 28 19:16:54.648035 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.647537    2569 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Apr 28 19:16:54.648035 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.647571    2569 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Apr 28 19:16:54.686607 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.686579    2569 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 28 19:16:54.694952 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:54.694929    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:54.768031 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.767780    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-04-27 19:11:53 +0000 UTC" deadline="2028-02-05 09:50:25.77441125 +0000 UTC"
Apr 28 19:16:54.768031 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.768028    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15542h33m31.006387068s"
Apr 28 19:16:54.795925 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:54.795899    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:54.800064 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.800043    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Apr 28 19:16:54.812524 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.812504    2569 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Apr 28 19:16:54.843702 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.843673    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xbtlc"
Apr 28 19:16:54.844680 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:54.844639    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb50e7600b47f76668e274e014c99f3ac.slice/crio-8b42f05209fef8f481c5c3808e60e6b126c58513c902b48085d003430851f165 WatchSource:0}: Error finding container 8b42f05209fef8f481c5c3808e60e6b126c58513c902b48085d003430851f165: Status 404 returned error can't find the container with id 8b42f05209fef8f481c5c3808e60e6b126c58513c902b48085d003430851f165
Apr 28 19:16:54.845132 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:54.845111    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55d83b29c984d704f2c407ca2173be08.slice/crio-6c5eed3154b7521dfc6a90199fea86d608695236e1a8637db95ef278c8af3a09 WatchSource:0}: Error finding container 6c5eed3154b7521dfc6a90199fea86d608695236e1a8637db95ef278c8af3a09: Status 404 returned error can't find the container with id 6c5eed3154b7521dfc6a90199fea86d608695236e1a8637db95ef278c8af3a09
Apr 28 19:16:54.849199 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.849184    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 28 19:16:54.853357 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.853342    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xbtlc"
Apr 28 19:16:54.896516 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:54.896496    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:54.940802 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.940754    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" event={"ID":"b50e7600b47f76668e274e014c99f3ac","Type":"ContainerStarted","Data":"8b42f05209fef8f481c5c3808e60e6b126c58513c902b48085d003430851f165"}
Apr 28 19:16:54.941727 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:54.941702    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal" event={"ID":"55d83b29c984d704f2c407ca2173be08","Type":"ContainerStarted","Data":"6c5eed3154b7521dfc6a90199fea86d608695236e1a8637db95ef278c8af3a09"}
Apr 28 19:16:54.996942 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:54.996869    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:55.097456 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:55.097428    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:55.198134 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:55.198084    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:55.298825 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:55.298747    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found"
Apr 28 19:16:55.315469 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.315443    2569 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 28 19:16:55.400737 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.400535    2569 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:55.413876 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.413847    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Apr 28 19:16:55.415558 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.415533    2569 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal"
Apr 28 19:16:55.422857 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.422838    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Apr 28 19:16:55.750433 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.750340    2569 apiserver.go:52] "Watching apiserver"
Apr 28 19:16:55.761211 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.761185    2569 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Apr 28 19:16:55.762849 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.762823    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-fbwbg","kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal","openshift-cluster-node-tuning-operator/tuned-xkhgf","openshift-dns/node-resolver-qpmnk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal","openshift-multus/multus-bp2ht","openshift-network-diagnostics/network-check-target-khqq9","openshift-ovn-kubernetes/ovnkube-node-ptjf5","kube-system/konnectivity-agent-jxpx9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw","openshift-image-registry/node-ca-tzfn9","openshift-multus/multus-additional-cni-plugins-mvd4r","openshift-multus/network-metrics-daemon-hgm2f"]
Apr 28 19:16:55.766877 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.766770    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.766877 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.766842    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fbwbg"
Apr 28 19:16:55.770112 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.770094    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Apr 28 19:16:55.770457 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.770390    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Apr 28 19:16:55.770554 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.770455    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Apr 28 19:16:55.770554 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.770488    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rdqkb\""
Apr 28 19:16:55.770554 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.770392    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Apr 28 19:16:55.770554 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.770536    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bn9qj\""
Apr 28 19:16:55.770734 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.770720    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:16:55.770820 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.770803    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qpmnk"
Apr 28 19:16:55.770820 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:55.770807    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:16:55.770952 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.770851    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Apr 28 19:16:55.773421 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.773162    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.774384 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.774212    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2t6hn\""
Apr 28 19:16:55.774384 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.774373    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Apr 28 19:16:55.774749 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.774670    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Apr 28 19:16:55.776231 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.775640    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Apr 28 19:16:55.776231 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.775718    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.776231 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.775981    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Apr 28 19:16:55.776231 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.776165    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Apr 28 19:16:55.776783 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.776689    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-knvt4\""
Apr 28 19:16:55.776783 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.776775    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Apr 28 19:16:55.777771 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.777753    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jxpx9"
Apr 28 19:16:55.778425 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.778382    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Apr 28 19:16:55.778514 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.778426    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zzcp7\""
Apr 28 19:16:55.778588 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.778575    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Apr 28 19:16:55.778642 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.778576    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Apr 28 19:16:55.779969 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.779717    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Apr 28 19:16:55.779969 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.779738    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Apr 28 19:16:55.779969 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.779756    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Apr 28 19:16:55.780797 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.780779    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Apr 28 19:16:55.780887 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.780811    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Apr 28 19:16:55.780887 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.780818    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ffmqm\""
Apr 28 19:16:55.782126 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.782108    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.782209 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.782194    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tzfn9"
Apr 28 19:16:55.784450 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.784429    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Apr 28 19:16:55.784650 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.784438    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.785100 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.784969    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Apr 28 19:16:55.785100 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.784995    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Apr 28 19:16:55.785100 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.785049    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ftc49\""
Apr 28 19:16:55.785295 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.785119    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Apr 28 19:16:55.785554 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.785537    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lfw5v\""
Apr 28 19:16:55.785654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.785580    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Apr 28 19:16:55.785722 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.785665    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Apr 28 19:16:55.786681 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.786663    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:16:55.786777 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:55.786731    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:16:55.787095 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.787079    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vjt94\""
Apr 28 19:16:55.787095 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.787090    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Apr 28 19:16:55.787244 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.787078    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Apr 28 19:16:55.802909 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.802893    2569 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
Apr 28 19:16:55.813294 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-run-netns\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.813294 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813301    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-run-systemd\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.813469 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813325    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-systemd\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.813469 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813382    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-var-lib-kubelet\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.813469 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813443    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-cnibin\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.813620 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813483    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-registration-dir\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.813620 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813512    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a07090c-27b9-4995-8133-9976a9919809-agent-certs\") pod \"konnectivity-agent-jxpx9\" (UID: \"2a07090c-27b9-4995-8133-9976a9919809\") " pod="kube-system/konnectivity-agent-jxpx9"
Apr 28 19:16:55.813620 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813534    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-kubernetes\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.813620 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813556    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-tuned\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.813620 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813580    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-run-multus-certs\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.813620 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813603    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fd16e46-3b64-4b7f-9d06-6ed637847678-cnibin\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.813851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813639    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fd16e46-3b64-4b7f-9d06-6ed637847678-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.813851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813664    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1fd16e46-3b64-4b7f-9d06-6ed637847678-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.813851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813681    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/25d9e6b9-b643-471e-ae03-5d308920bbbb-multus-daemon-config\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.813851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813703    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-etc-kubernetes\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.813851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813729    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-var-lib-openvswitch\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.813851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813745    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crtg8\" (UniqueName: \"kubernetes.io/projected/578ad6cd-a560-4a18-8806-846b76fc9338-kube-api-access-crtg8\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.813851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813793    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bmf4\" (UniqueName: \"kubernetes.io/projected/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-kube-api-access-7bmf4\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.813851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813828    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-sysconfig\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.814206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813856    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/88298985-0c2c-4787-9fa7-539c47c429e2-iptables-alerter-script\") pod \"iptables-alerter-fbwbg\" (UID: \"88298985-0c2c-4787-9fa7-539c47c429e2\") " pod="openshift-network-operator/iptables-alerter-fbwbg"
Apr 28 19:16:55.814206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813880    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88298985-0c2c-4787-9fa7-539c47c429e2-host-slash\") pod \"iptables-alerter-fbwbg\" (UID: \"88298985-0c2c-4787-9fa7-539c47c429e2\") " pod="openshift-network-operator/iptables-alerter-fbwbg"
Apr 28 19:16:55.814206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813903    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-cni-netd\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.814206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813930    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1fd16e46-3b64-4b7f-9d06-6ed637847678-os-release\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.814206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813953    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4hmm\" (UniqueName: \"kubernetes.io/projected/1fd16e46-3b64-4b7f-9d06-6ed637847678-kube-api-access-g4hmm\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.814206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813976    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-sysctl-d\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.814206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.813999    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-var-lib-cni-bin\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.814206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814046    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-etc-openvswitch\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.814206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814077    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-run-ovn\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.814206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814102    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-node-log\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.814206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814124    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/989021b3-c3c6-43a9-8d1e-c2a8660c0d89-tmp-dir\") pod \"node-resolver-qpmnk\" (UID: \"989021b3-c3c6-43a9-8d1e-c2a8660c0d89\") " pod="openshift-dns/node-resolver-qpmnk"
Apr 28 19:16:55.814206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814158    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-socket-dir\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.814206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814184    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fd16e46-3b64-4b7f-9d06-6ed637847678-system-cni-dir\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814217    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwngk\" (UniqueName: \"kubernetes.io/projected/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-kube-api-access-hwngk\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814243    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-systemd-units\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814267    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/578ad6cd-a560-4a18-8806-846b76fc9338-env-overrides\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814290    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/578ad6cd-a560-4a18-8806-846b76fc9338-ovnkube-script-lib\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814324    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmb8n\" (UniqueName: \"kubernetes.io/projected/88298985-0c2c-4787-9fa7-539c47c429e2-kube-api-access-jmb8n\") pod \"iptables-alerter-fbwbg\" (UID: \"88298985-0c2c-4787-9fa7-539c47c429e2\") " pod="openshift-network-operator/iptables-alerter-fbwbg"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814362    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-multus-conf-dir\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814392    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-slash\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814433    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-device-dir\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814467    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-os-release\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814492    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-kubelet\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814515    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-run-netns\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814536    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-cni-bin\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814581    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frc48\" (UniqueName: \"kubernetes.io/projected/8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad-kube-api-access-frc48\") pod \"node-ca-tzfn9\" (UID: \"8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad\") " pod="openshift-image-registry/node-ca-tzfn9"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814622    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-run\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814655    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-run-openvswitch\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.814787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814683    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/989021b3-c3c6-43a9-8d1e-c2a8660c0d89-hosts-file\") pod \"node-resolver-qpmnk\" (UID: \"989021b3-c3c6-43a9-8d1e-c2a8660c0d89\") " pod="openshift-dns/node-resolver-qpmnk"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814715    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814740    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-sysctl-conf\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814764    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzqcd\" (UniqueName: \"kubernetes.io/projected/25d9e6b9-b643-471e-ae03-5d308920bbbb-kube-api-access-hzqcd\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814786    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad-serviceca\") pod \"node-ca-tzfn9\" (UID: \"8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad\") " pod="openshift-image-registry/node-ca-tzfn9"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814817    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-modprobe-d\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814835    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-lib-modules\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814853    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-run-k8s-cni-cncf-io\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814898    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-run-ovn-kubernetes\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814928    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1fd16e46-3b64-4b7f-9d06-6ed637847678-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814956    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-sys\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.814981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-host\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815022    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-system-cni-dir\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815063    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-multus-cni-dir\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815098    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/25d9e6b9-b643-471e-ae03-5d308920bbbb-cni-binary-copy\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815131    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-multus-socket-dir-parent\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815163    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-hostroot\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.815494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cn6j\" (UniqueName: \"kubernetes.io/projected/989021b3-c3c6-43a9-8d1e-c2a8660c0d89-kube-api-access-9cn6j\") pod \"node-resolver-qpmnk\" (UID: \"989021b3-c3c6-43a9-8d1e-c2a8660c0d89\") " pod="openshift-dns/node-resolver-qpmnk"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815246    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-sys-fs\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815273    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-log-socket\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815305    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815353    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnzgm\" (UniqueName: \"kubernetes.io/projected/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-kube-api-access-wnzgm\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815383    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6gc8\" (UniqueName: \"kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8\") pod \"network-check-target-khqq9\" (UID: \"ad86af47-f5e9-4e6f-8640-b3771e91074e\") " pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815448    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a07090c-27b9-4995-8133-9976a9919809-konnectivity-ca\") pod \"konnectivity-agent-jxpx9\" (UID: \"2a07090c-27b9-4995-8133-9976a9919809\") " pod="kube-system/konnectivity-agent-jxpx9"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815489    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/578ad6cd-a560-4a18-8806-846b76fc9338-ovn-node-metrics-cert\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815532    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815567    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fd16e46-3b64-4b7f-9d06-6ed637847678-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815640    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-var-lib-cni-multus\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815668    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-var-lib-kubelet\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815695    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/578ad6cd-a560-4a18-8806-846b76fc9338-ovnkube-config\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815713    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815734    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad-host\") pod \"node-ca-tzfn9\" (UID: \"8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad\") " pod="openshift-image-registry/node-ca-tzfn9"
Apr 28 19:16:55.816157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.815767    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-tmp\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.847000 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.846968    2569 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 28 19:16:55.854978 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.854946    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-27 19:11:54 +0000 UTC" deadline="2027-11-27 07:06:40.25198184 +0000 UTC"
Apr 28 19:16:55.854978 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.854978    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13859h49m44.397007761s"
Apr 28 19:16:55.916453 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916422    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad-serviceca\") pod \"node-ca-tzfn9\" (UID: \"8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad\") " pod="openshift-image-registry/node-ca-tzfn9"
Apr 28 19:16:55.916616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916463    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-modprobe-d\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.916616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916489    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-lib-modules\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.916616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916568    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-run-k8s-cni-cncf-io\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.916616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916603    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-lib-modules\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.916616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916609    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-run-ovn-kubernetes\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916637    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1fd16e46-3b64-4b7f-9d06-6ed637847678-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916644    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-run-k8s-cni-cncf-io\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916655    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-run-ovn-kubernetes\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916610    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-modprobe-d\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916667    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-sys\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916692    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-host\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916710    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-system-cni-dir\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916727    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-multus-cni-dir\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916727    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-sys\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916750    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/25d9e6b9-b643-471e-ae03-5d308920bbbb-cni-binary-copy\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916765    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-host\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916774    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-multus-socket-dir-parent\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916809    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-multus-cni-dir\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916814    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-hostroot\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916821    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-system-cni-dir\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916851    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-hostroot\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.916868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916829    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-multus-socket-dir-parent\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916888    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9cn6j\" (UniqueName: \"kubernetes.io/projected/989021b3-c3c6-43a9-8d1e-c2a8660c0d89-kube-api-access-9cn6j\") pod \"node-resolver-qpmnk\" (UID: \"989021b3-c3c6-43a9-8d1e-c2a8660c0d89\") " pod="openshift-dns/node-resolver-qpmnk"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916929    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-sys-fs\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916937    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad-serviceca\") pod \"node-ca-tzfn9\" (UID: \"8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad\") " pod="openshift-image-registry/node-ca-tzfn9"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916952    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-sys-fs\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916955    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-log-socket\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916985    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-log-socket\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.916989    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917010    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wnzgm\" (UniqueName: \"kubernetes.io/projected/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-kube-api-access-wnzgm\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917032    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j6gc8\" (UniqueName: \"kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8\") pod \"network-check-target-khqq9\" (UID: \"ad86af47-f5e9-4e6f-8640-b3771e91074e\") " pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917060    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a07090c-27b9-4995-8133-9976a9919809-konnectivity-ca\") pod \"konnectivity-agent-jxpx9\" (UID: \"2a07090c-27b9-4995-8133-9976a9919809\") " pod="kube-system/konnectivity-agent-jxpx9"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917086    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/578ad6cd-a560-4a18-8806-846b76fc9338-ovn-node-metrics-cert\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917104    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917120    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fd16e46-3b64-4b7f-9d06-6ed637847678-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917136    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-var-lib-cni-multus\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917157    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-var-lib-kubelet\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917173    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/578ad6cd-a560-4a18-8806-846b76fc9338-ovnkube-config\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.917618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917188    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917203    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad-host\") pod \"node-ca-tzfn9\" (UID: \"8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad\") " pod="openshift-image-registry/node-ca-tzfn9"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917220    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-tmp\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917222    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1fd16e46-3b64-4b7f-9d06-6ed637847678-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917232    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-var-lib-cni-multus\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917032    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917238    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-run-netns\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917262    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-run-netns\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917286    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-run-systemd\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917309    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917314    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-systemd\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917333    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-var-lib-kubelet\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917340    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-var-lib-kubelet\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917359    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad-host\") pod \"node-ca-tzfn9\" (UID: \"8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad\") " pod="openshift-image-registry/node-ca-tzfn9"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917369    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-cnibin\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917393    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/25d9e6b9-b643-471e-ae03-5d308920bbbb-cni-binary-copy\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917415    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-registration-dir\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917457    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-run-systemd\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.918308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917467    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a07090c-27b9-4995-8133-9976a9919809-agent-certs\") pod \"konnectivity-agent-jxpx9\" (UID: \"2a07090c-27b9-4995-8133-9976a9919809\") " pod="kube-system/konnectivity-agent-jxpx9"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:55.917471    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917493    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-kubernetes\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917503    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-cnibin\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917369    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fd16e46-3b64-4b7f-9d06-6ed637847678-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917517    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-tuned\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:55.917557    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs podName:e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33 nodeName:}" failed. No retries permitted until 2026-04-28 19:16:56.417513707 +0000 UTC m=+3.241853812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs") pod "network-metrics-daemon-hgm2f" (UID: "e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917480    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-var-lib-kubelet\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917586    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917623    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-run-multus-certs\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917653    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fd16e46-3b64-4b7f-9d06-6ed637847678-cnibin\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917665    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-systemd\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917680    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fd16e46-3b64-4b7f-9d06-6ed637847678-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917623    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-registration-dir\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917742    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1fd16e46-3b64-4b7f-9d06-6ed637847678-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917754    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-run-multus-certs\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917785    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/25d9e6b9-b643-471e-ae03-5d308920bbbb-multus-daemon-config\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.919171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917814    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-etc-kubernetes\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917831    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/578ad6cd-a560-4a18-8806-846b76fc9338-ovnkube-config\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917843    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-var-lib-openvswitch\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917874    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-crtg8\" (UniqueName: \"kubernetes.io/projected/578ad6cd-a560-4a18-8806-846b76fc9338-kube-api-access-crtg8\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917898    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fd16e46-3b64-4b7f-9d06-6ed637847678-cnibin\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917905    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7bmf4\" (UniqueName: \"kubernetes.io/projected/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-kube-api-access-7bmf4\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917931    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-sysconfig\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917944    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-etc-kubernetes\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917957    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/88298985-0c2c-4787-9fa7-539c47c429e2-iptables-alerter-script\") pod \"iptables-alerter-fbwbg\" (UID: \"88298985-0c2c-4787-9fa7-539c47c429e2\") " pod="openshift-network-operator/iptables-alerter-fbwbg"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.917982    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88298985-0c2c-4787-9fa7-539c47c429e2-host-slash\") pod \"iptables-alerter-fbwbg\" (UID: \"88298985-0c2c-4787-9fa7-539c47c429e2\") " pod="openshift-network-operator/iptables-alerter-fbwbg"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918005    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-cni-netd\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918033    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1fd16e46-3b64-4b7f-9d06-6ed637847678-os-release\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918081    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g4hmm\" (UniqueName: \"kubernetes.io/projected/1fd16e46-3b64-4b7f-9d06-6ed637847678-kube-api-access-g4hmm\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918115    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-kubernetes\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918124    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-sysctl-d\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918153    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-var-lib-cni-bin\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918160    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-var-lib-openvswitch\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.919922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918163    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a07090c-27b9-4995-8133-9976a9919809-konnectivity-ca\") pod \"konnectivity-agent-jxpx9\" (UID: \"2a07090c-27b9-4995-8133-9976a9919809\") " pod="kube-system/konnectivity-agent-jxpx9"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918177    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-etc-openvswitch\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918202    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-run-ovn\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918310    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-sysctl-d\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918350    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88298985-0c2c-4787-9fa7-539c47c429e2-host-slash\") pod \"iptables-alerter-fbwbg\" (UID: \"88298985-0c2c-4787-9fa7-539c47c429e2\") " pod="openshift-network-operator/iptables-alerter-fbwbg"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918440    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-cni-netd\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918498    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1fd16e46-3b64-4b7f-9d06-6ed637847678-os-release\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918528    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-node-log\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918536    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-run-ovn\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918555    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/989021b3-c3c6-43a9-8d1e-c2a8660c0d89-tmp-dir\") pod \"node-resolver-qpmnk\" (UID: \"989021b3-c3c6-43a9-8d1e-c2a8660c0d89\") " pod="openshift-dns/node-resolver-qpmnk"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918580    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-socket-dir\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918600    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-sysconfig\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918633    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fd16e46-3b64-4b7f-9d06-6ed637847678-system-cni-dir\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918603    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fd16e46-3b64-4b7f-9d06-6ed637847678-system-cni-dir\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918672    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-node-log\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918711    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hwngk\" (UniqueName: \"kubernetes.io/projected/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-kube-api-access-hwngk\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918741    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-systemd-units\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.920497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918768    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/578ad6cd-a560-4a18-8806-846b76fc9338-env-overrides\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918810    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/578ad6cd-a560-4a18-8806-846b76fc9338-ovnkube-script-lib\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918835    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jmb8n\" (UniqueName: \"kubernetes.io/projected/88298985-0c2c-4787-9fa7-539c47c429e2-kube-api-access-jmb8n\") pod \"iptables-alerter-fbwbg\" (UID: \"88298985-0c2c-4787-9fa7-539c47c429e2\") " pod="openshift-network-operator/iptables-alerter-fbwbg"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918844    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fd16e46-3b64-4b7f-9d06-6ed637847678-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918879    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-multus-conf-dir\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918919    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-slash\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918957    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/25d9e6b9-b643-471e-ae03-5d308920bbbb-multus-daemon-config\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918967    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-device-dir\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918993    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-os-release\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919048    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-kubelet\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919075    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-run-netns\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919087    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-socket-dir\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919104    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-etc-openvswitch\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919114    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-cni-bin\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919148    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-frc48\" (UniqueName: \"kubernetes.io/projected/8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad-kube-api-access-frc48\") pod \"node-ca-tzfn9\" (UID: \"8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad\") " pod="openshift-image-registry/node-ca-tzfn9"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919154    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-os-release\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919151    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-slash\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.918968    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/989021b3-c3c6-43a9-8d1e-c2a8660c0d89-tmp-dir\") pod \"node-resolver-qpmnk\" (UID: \"989021b3-c3c6-43a9-8d1e-c2a8660c0d89\") " pod="openshift-dns/node-resolver-qpmnk"
Apr 28 19:16:55.921215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919180    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-systemd-units\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919186    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-run\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919225    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-host-var-lib-cni-bin\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919234    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-run-netns\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919227    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-run-openvswitch\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919270    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-cni-bin\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919256    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-run-openvswitch\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919286    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/989021b3-c3c6-43a9-8d1e-c2a8660c0d89-hosts-file\") pod \"node-resolver-qpmnk\" (UID: \"989021b3-c3c6-43a9-8d1e-c2a8660c0d89\") " pod="openshift-dns/node-resolver-qpmnk"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919292    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-run\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919154    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-device-dir\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919319    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919323    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/88298985-0c2c-4787-9fa7-539c47c429e2-iptables-alerter-script\") pod \"iptables-alerter-fbwbg\" (UID: \"88298985-0c2c-4787-9fa7-539c47c429e2\") " pod="openshift-network-operator/iptables-alerter-fbwbg"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919329    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/989021b3-c3c6-43a9-8d1e-c2a8660c0d89-hosts-file\") pod \"node-resolver-qpmnk\" (UID: \"989021b3-c3c6-43a9-8d1e-c2a8660c0d89\") " pod="openshift-dns/node-resolver-qpmnk"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919105    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/25d9e6b9-b643-471e-ae03-5d308920bbbb-multus-conf-dir\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919347    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-sysctl-conf\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919374    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hzqcd\" (UniqueName: \"kubernetes.io/projected/25d9e6b9-b643-471e-ae03-5d308920bbbb-kube-api-access-hzqcd\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919385    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919383    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/578ad6cd-a560-4a18-8806-846b76fc9338-host-kubelet\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.921946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919568    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-sysctl-conf\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.922847 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919785    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/578ad6cd-a560-4a18-8806-846b76fc9338-ovnkube-script-lib\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.922847 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919896    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/578ad6cd-a560-4a18-8806-846b76fc9338-env-overrides\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.922847 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.919929    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1fd16e46-3b64-4b7f-9d06-6ed637847678-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:55.922847 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.921394    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/578ad6cd-a560-4a18-8806-846b76fc9338-ovn-node-metrics-cert\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.922847 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.921570    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-etc-tuned\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.922847 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.921758    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-tmp\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.922847 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.921920    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a07090c-27b9-4995-8133-9976a9919809-agent-certs\") pod \"konnectivity-agent-jxpx9\" (UID: \"2a07090c-27b9-4995-8133-9976a9919809\") " pod="kube-system/konnectivity-agent-jxpx9"
Apr 28 19:16:55.927135 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.927119    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cn6j\" (UniqueName: \"kubernetes.io/projected/989021b3-c3c6-43a9-8d1e-c2a8660c0d89-kube-api-access-9cn6j\") pod \"node-resolver-qpmnk\" (UID: \"989021b3-c3c6-43a9-8d1e-c2a8660c0d89\") " pod="openshift-dns/node-resolver-qpmnk"
Apr 28 19:16:55.934192 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:55.934169    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 28 19:16:55.934192 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:55.934194    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 28 19:16:55.934351 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:55.934208    2569 projected.go:194] Error preparing data for projected volume kube-api-access-j6gc8 for pod openshift-network-diagnostics/network-check-target-khqq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 28 19:16:55.934351 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:55.934276    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8 podName:ad86af47-f5e9-4e6f-8640-b3771e91074e nodeName:}" failed. No retries permitted until 2026-04-28 19:16:56.434261773 +0000 UTC m=+3.258601862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-j6gc8" (UniqueName: "kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8") pod "network-check-target-khqq9" (UID: "ad86af47-f5e9-4e6f-8640-b3771e91074e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 28 19:16:55.936303 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.936281    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwngk\" (UniqueName: \"kubernetes.io/projected/6a7302f5-8d7a-4665-af2a-81b3fe58a0c4-kube-api-access-hwngk\") pod \"tuned-xkhgf\" (UID: \"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4\") " pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:55.966362 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.966325    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crtg8\" (UniqueName: \"kubernetes.io/projected/578ad6cd-a560-4a18-8806-846b76fc9338-kube-api-access-crtg8\") pod \"ovnkube-node-ptjf5\" (UID: \"578ad6cd-a560-4a18-8806-846b76fc9338\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:55.967369 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.967343    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmb8n\" (UniqueName: \"kubernetes.io/projected/88298985-0c2c-4787-9fa7-539c47c429e2-kube-api-access-jmb8n\") pod \"iptables-alerter-fbwbg\" (UID: \"88298985-0c2c-4787-9fa7-539c47c429e2\") " pod="openshift-network-operator/iptables-alerter-fbwbg"
Apr 28 19:16:55.967565 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.967544    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bmf4\" (UniqueName: \"kubernetes.io/projected/e66fe8ab-55e6-4b17-a723-bfd159f94c5d-kube-api-access-7bmf4\") pod \"aws-ebs-csi-driver-node-4pqhw\" (UID: \"e66fe8ab-55e6-4b17-a723-bfd159f94c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:55.968141 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.968118    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzqcd\" (UniqueName: \"kubernetes.io/projected/25d9e6b9-b643-471e-ae03-5d308920bbbb-kube-api-access-hzqcd\") pod \"multus-bp2ht\" (UID: \"25d9e6b9-b643-471e-ae03-5d308920bbbb\") " pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:55.969841 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.969823    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnzgm\" (UniqueName: \"kubernetes.io/projected/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-kube-api-access-wnzgm\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:16:55.972778 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.972760    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frc48\" (UniqueName: \"kubernetes.io/projected/8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad-kube-api-access-frc48\") pod \"node-ca-tzfn9\" (UID: \"8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad\") " pod="openshift-image-registry/node-ca-tzfn9"
Apr 28 19:16:55.987829 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:55.987811    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4hmm\" (UniqueName: \"kubernetes.io/projected/1fd16e46-3b64-4b7f-9d06-6ed637847678-kube-api-access-g4hmm\") pod \"multus-additional-cni-plugins-mvd4r\" (UID: \"1fd16e46-3b64-4b7f-9d06-6ed637847678\") " pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:56.047936 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.047868    2569 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 28 19:16:56.080165 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.080138    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xkhgf"
Apr 28 19:16:56.089885 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.089860    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fbwbg"
Apr 28 19:16:56.096471 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.096452    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qpmnk"
Apr 28 19:16:56.102105 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.102088    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bp2ht"
Apr 28 19:16:56.108871 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.108853    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:16:56.115295 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.115275    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jxpx9"
Apr 28 19:16:56.121927 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.121902    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw"
Apr 28 19:16:56.130443 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.130418    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tzfn9"
Apr 28 19:16:56.134918 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.134900    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mvd4r"
Apr 28 19:16:56.375862 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.375787    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-rw72g"]
Apr 28 19:16:56.380451 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.380423    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:16:56.380562 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:56.380507    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:16:56.422502 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.422477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-dbus\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:16:56.422655 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.422527    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:16:56.422655 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.422617    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:16:56.422757 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:56.422686    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 28 19:16:56.422757 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.422734    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-kubelet-config\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:16:56.422845 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:56.422794    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs podName:e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33 nodeName:}" failed. No retries permitted until 2026-04-28 19:16:57.422776225 +0000 UTC m=+4.247116315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs") pod "network-metrics-daemon-hgm2f" (UID: "e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 28 19:16:56.472053 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:56.472026    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25d9e6b9_b643_471e_ae03_5d308920bbbb.slice/crio-ce4d96c5d84f2f2d95bdb974a55bf1e035b28801cb1b7356539cf5a5f45ba827 WatchSource:0}: Error finding container ce4d96c5d84f2f2d95bdb974a55bf1e035b28801cb1b7356539cf5a5f45ba827: Status 404 returned error can't find the container with id ce4d96c5d84f2f2d95bdb974a55bf1e035b28801cb1b7356539cf5a5f45ba827
Apr 28 19:16:56.474850 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:56.474823    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88298985_0c2c_4787_9fa7_539c47c429e2.slice/crio-e29ba9c4f383c6272b237af7b0a1e5d20c419e244a2834045909c70360cb2196 WatchSource:0}: Error finding container e29ba9c4f383c6272b237af7b0a1e5d20c419e244a2834045909c70360cb2196: Status 404 returned error can't find the container with id e29ba9c4f383c6272b237af7b0a1e5d20c419e244a2834045909c70360cb2196
Apr 28 19:16:56.477721 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:56.477698    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod989021b3_c3c6_43a9_8d1e_c2a8660c0d89.slice/crio-4689d0a26c72994dea233f0a74a12ebf169b85ac5363ec2c510c8315f6a51f33 WatchSource:0}: Error finding container 4689d0a26c72994dea233f0a74a12ebf169b85ac5363ec2c510c8315f6a51f33: Status 404 returned error can't find the container with id 4689d0a26c72994dea233f0a74a12ebf169b85ac5363ec2c510c8315f6a51f33
Apr 28 19:16:56.478608 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:56.478506    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a7302f5_8d7a_4665_af2a_81b3fe58a0c4.slice/crio-02719480ced6a27326d0fff50d034a1239b5c6542194ee516aec838871a01a2e WatchSource:0}: Error finding container 02719480ced6a27326d0fff50d034a1239b5c6542194ee516aec838871a01a2e: Status 404 returned error can't find the container with id 02719480ced6a27326d0fff50d034a1239b5c6542194ee516aec838871a01a2e
Apr 28 19:16:56.479274 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:56.479203    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fd16e46_3b64_4b7f_9d06_6ed637847678.slice/crio-1b9678438f63c91cb12cb02f38ea8800fabb397955b0bff228da50fea522c774 WatchSource:0}: Error finding container 1b9678438f63c91cb12cb02f38ea8800fabb397955b0bff228da50fea522c774: Status 404 returned error can't find the container with id 1b9678438f63c91cb12cb02f38ea8800fabb397955b0bff228da50fea522c774
Apr 28 19:16:56.499162 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:56.499141    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a07090c_27b9_4995_8133_9976a9919809.slice/crio-e93b7f5fd442945e000e2820cbf637be5234343c0bbce1e75f2eb242a73a1931 WatchSource:0}: Error finding container e93b7f5fd442945e000e2820cbf637be5234343c0bbce1e75f2eb242a73a1931: Status 404 returned error can't find the container with id e93b7f5fd442945e000e2820cbf637be5234343c0bbce1e75f2eb242a73a1931
Apr 28 19:16:56.503192 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:56.503165    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod578ad6cd_a560_4a18_8806_846b76fc9338.slice/crio-b1c9bbd8abc58399e946f4a4b930f9fe278183328fa0096969408340a5f0068c WatchSource:0}: Error finding container b1c9bbd8abc58399e946f4a4b930f9fe278183328fa0096969408340a5f0068c: Status 404 returned error can't find the container with id b1c9bbd8abc58399e946f4a4b930f9fe278183328fa0096969408340a5f0068c
Apr 28 19:16:56.504180 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:16:56.504158    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8595e0_ea1f_4107_b1b4_f9f55cdaf8ad.slice/crio-af8cadd814e8b028b0a0ed31c3b3fdd6b6bc6e5e685e0ef1717b10dc4c86293f WatchSource:0}: Error finding container af8cadd814e8b028b0a0ed31c3b3fdd6b6bc6e5e685e0ef1717b10dc4c86293f: Status 404 returned error can't find the container with id af8cadd814e8b028b0a0ed31c3b3fdd6b6bc6e5e685e0ef1717b10dc4c86293f
Apr 28 19:16:56.523720 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.523700    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-dbus\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:16:56.523786 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.523746    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j6gc8\" (UniqueName: \"kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8\") pod \"network-check-target-khqq9\" (UID: \"ad86af47-f5e9-4e6f-8640-b3771e91074e\") " pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:16:56.523845 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.523785    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:16:56.523845 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.523780    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-dbus\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:16:56.523845 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.523810    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-kubelet-config\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:16:56.523963 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.523865    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-kubelet-config\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:16:56.523963 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:56.523873    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 28 19:16:56.523963 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:56.523883    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Apr 28 19:16:56.523963 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:56.523893    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 28 19:16:56.523963 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:56.523908    2569 projected.go:194] Error preparing data for projected volume kube-api-access-j6gc8 for pod openshift-network-diagnostics/network-check-target-khqq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 28 19:16:56.523963 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:56.523928    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret podName:a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e nodeName:}" failed. No retries permitted until 2026-04-28 19:16:57.023914452 +0000 UTC m=+3.848254543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret") pod "global-pull-secret-syncer-rw72g" (UID: "a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e") : object "kube-system"/"original-pull-secret" not registered
Apr 28 19:16:56.523963 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:56.523946    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8 podName:ad86af47-f5e9-4e6f-8640-b3771e91074e nodeName:}" failed. No retries permitted until 2026-04-28 19:16:57.523934442 +0000 UTC m=+4.348274527 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-j6gc8" (UniqueName: "kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8") pod "network-check-target-khqq9" (UID: "ad86af47-f5e9-4e6f-8640-b3771e91074e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 28 19:16:56.855728 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.855608    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-27 19:11:54 +0000 UTC" deadline="2027-10-07 13:45:05.877573396 +0000 UTC"
Apr 28 19:16:56.855728 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.855645    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12642h28m9.021932445s"
Apr 28 19:16:56.958327 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.958241    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal" event={"ID":"55d83b29c984d704f2c407ca2173be08","Type":"ContainerStarted","Data":"e614fda05ef26a4e34cfb729c4a096c9f6e546da877bf7b936708c66a561adab"}
Apr 28 19:16:56.961318 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.961229    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tzfn9" event={"ID":"8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad","Type":"ContainerStarted","Data":"af8cadd814e8b028b0a0ed31c3b3fdd6b6bc6e5e685e0ef1717b10dc4c86293f"}
Apr 28 19:16:56.963850 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.963511    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5" event={"ID":"578ad6cd-a560-4a18-8806-846b76fc9338","Type":"ContainerStarted","Data":"b1c9bbd8abc58399e946f4a4b930f9fe278183328fa0096969408340a5f0068c"}
Apr 28 19:16:56.970463 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.970378    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw" event={"ID":"e66fe8ab-55e6-4b17-a723-bfd159f94c5d","Type":"ContainerStarted","Data":"199d04a5c6919087db3c4dcd36be418fb772d05ad0fdc071680aecc70a3a24ae"}
Apr 28 19:16:56.977071 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.976904    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fbwbg" event={"ID":"88298985-0c2c-4787-9fa7-539c47c429e2","Type":"ContainerStarted","Data":"e29ba9c4f383c6272b237af7b0a1e5d20c419e244a2834045909c70360cb2196"}
Apr 28 19:16:56.986709 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.986666    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bp2ht" event={"ID":"25d9e6b9-b643-471e-ae03-5d308920bbbb","Type":"ContainerStarted","Data":"ce4d96c5d84f2f2d95bdb974a55bf1e035b28801cb1b7356539cf5a5f45ba827"}
Apr 28 19:16:56.997593 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:56.997540    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jxpx9" event={"ID":"2a07090c-27b9-4995-8133-9976a9919809","Type":"ContainerStarted","Data":"e93b7f5fd442945e000e2820cbf637be5234343c0bbce1e75f2eb242a73a1931"}
Apr 28 19:16:57.000287 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:57.000232    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvd4r" event={"ID":"1fd16e46-3b64-4b7f-9d06-6ed637847678","Type":"ContainerStarted","Data":"1b9678438f63c91cb12cb02f38ea8800fabb397955b0bff228da50fea522c774"}
Apr 28 19:16:57.002879 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:57.002818    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xkhgf" event={"ID":"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4","Type":"ContainerStarted","Data":"02719480ced6a27326d0fff50d034a1239b5c6542194ee516aec838871a01a2e"}
Apr 28 19:16:57.008288 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:57.008265    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qpmnk" event={"ID":"989021b3-c3c6-43a9-8d1e-c2a8660c0d89","Type":"ContainerStarted","Data":"4689d0a26c72994dea233f0a74a12ebf169b85ac5363ec2c510c8315f6a51f33"}
Apr 28 19:16:57.028899 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:57.028063    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:16:57.028899 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:57.028277    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Apr 28 19:16:57.028899 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:57.028367    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret podName:a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e nodeName:}" failed. No retries permitted until 2026-04-28 19:16:58.028347676 +0000 UTC m=+4.852687767 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret") pod "global-pull-secret-syncer-rw72g" (UID: "a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e") : object "kube-system"/"original-pull-secret" not registered
Apr 28 19:16:57.442418 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:57.434210    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:16:57.442418 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:57.441718    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 28 19:16:57.442418 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:57.441806    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs podName:e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33 nodeName:}" failed. No retries permitted until 2026-04-28 19:16:59.441785026 +0000 UTC m=+6.266125126 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs") pod "network-metrics-daemon-hgm2f" (UID: "e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 28 19:16:57.535307 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:57.535262    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j6gc8\" (UniqueName: \"kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8\") pod \"network-check-target-khqq9\" (UID: \"ad86af47-f5e9-4e6f-8640-b3771e91074e\") " pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:16:57.535577 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:57.535558    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 28 19:16:57.535664 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:57.535584    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 28 19:16:57.535664 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:57.535597    2569 projected.go:194] Error preparing data for projected volume kube-api-access-j6gc8 for pod openshift-network-diagnostics/network-check-target-khqq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 28 19:16:57.535664 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:57.535656    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8 podName:ad86af47-f5e9-4e6f-8640-b3771e91074e nodeName:}" failed. No retries permitted until 2026-04-28 19:16:59.535637115 +0000 UTC m=+6.359977216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-j6gc8" (UniqueName: "kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8") pod "network-check-target-khqq9" (UID: "ad86af47-f5e9-4e6f-8640-b3771e91074e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 28 19:16:57.938966 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:57.938676    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:16:57.939393 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:57.939041    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:16:57.939476 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:57.939465    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:16:57.939583 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:57.939559    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:16:57.939732 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:57.939717    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:16:57.939814 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:57.939796    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:16:58.024143 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:58.023018    2569 generic.go:358] "Generic (PLEG): container finished" podID="b50e7600b47f76668e274e014c99f3ac" containerID="13ed225f9148601e3e6e79f605cc4467b9f57265e38a4bc4a7e5979f42b7f93b" exitCode=0
Apr 28 19:16:58.024143 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:58.023930    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" event={"ID":"b50e7600b47f76668e274e014c99f3ac","Type":"ContainerDied","Data":"13ed225f9148601e3e6e79f605cc4467b9f57265e38a4bc4a7e5979f42b7f93b"}
Apr 28 19:16:58.040037 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:58.040003    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:16:58.040145 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:58.040131    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Apr 28 19:16:58.040228 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:58.040186    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret podName:a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e nodeName:}" failed. No retries permitted until 2026-04-28 19:17:00.040169276 +0000 UTC m=+6.864509376 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret") pod "global-pull-secret-syncer-rw72g" (UID: "a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e") : object "kube-system"/"original-pull-secret" not registered
Apr 28 19:16:58.048190 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:58.048139    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal" podStartSLOduration=3.048123569 podStartE2EDuration="3.048123569s" podCreationTimestamp="2026-04-28 19:16:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:16:56.977033366 +0000 UTC m=+3.801373475" watchObservedRunningTime="2026-04-28 19:16:58.048123569 +0000 UTC m=+4.872463701"
Apr 28 19:16:59.032183 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:59.031765    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" event={"ID":"b50e7600b47f76668e274e014c99f3ac","Type":"ContainerStarted","Data":"a05851c19346728c6042b03e040704dd9e9ed9fdfb7cb691af3bc0e75e18eb19"}
Apr 28 19:16:59.453914 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:59.453816    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:16:59.454092 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:59.453985    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 28 19:16:59.454092 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:59.454048    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs podName:e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:03.454027725 +0000 UTC m=+10.278367816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs") pod "network-metrics-daemon-hgm2f" (UID: "e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 28 19:16:59.554900 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:59.554607    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j6gc8\" (UniqueName: \"kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8\") pod \"network-check-target-khqq9\" (UID: \"ad86af47-f5e9-4e6f-8640-b3771e91074e\") " pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:16:59.554900 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:59.554818    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 28 19:16:59.554900 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:59.554842    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 28 19:16:59.554900 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:59.554855    2569 projected.go:194] Error preparing data for projected volume kube-api-access-j6gc8 for pod openshift-network-diagnostics/network-check-target-khqq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 28 19:16:59.555232 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:59.554919    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8 podName:ad86af47-f5e9-4e6f-8640-b3771e91074e nodeName:}" failed. No retries permitted until 2026-04-28 19:17:03.554900282 +0000 UTC m=+10.379240390 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-j6gc8" (UniqueName: "kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8") pod "network-check-target-khqq9" (UID: "ad86af47-f5e9-4e6f-8640-b3771e91074e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 28 19:16:59.939622 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:59.939530    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:16:59.939787 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:59.939669    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:16:59.940437 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:59.940069    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:16:59.940437 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:59.940170    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:16:59.940437 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:16:59.940249    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:16:59.940437 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:16:59.940332    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:00.059052 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:00.059019    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:00.059497 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:00.059193    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Apr 28 19:17:00.059497 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:00.059256    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret podName:a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e nodeName:}" failed. No retries permitted until 2026-04-28 19:17:04.059237866 +0000 UTC m=+10.883577959 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret") pod "global-pull-secret-syncer-rw72g" (UID: "a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e") : object "kube-system"/"original-pull-secret" not registered
Apr 28 19:17:01.938598 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:01.938559    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:01.939048 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:01.938567    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:01.939048 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:01.938667    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:17:01.939048 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:01.938682    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:01.939048 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:01.938818    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:17:01.939048 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:01.938881    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:03.487306 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:03.487050    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:03.487306 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:03.487186    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 28 19:17:03.487977 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:03.487953    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs podName:e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:11.48792272 +0000 UTC m=+18.312262820 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs") pod "network-metrics-daemon-hgm2f" (UID: "e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 28 19:17:03.588226 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:03.587913    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j6gc8\" (UniqueName: \"kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8\") pod \"network-check-target-khqq9\" (UID: \"ad86af47-f5e9-4e6f-8640-b3771e91074e\") " pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:03.588226 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:03.588114    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 28 19:17:03.588226 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:03.588134    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 28 19:17:03.588226 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:03.588146    2569 projected.go:194] Error preparing data for projected volume kube-api-access-j6gc8 for pod openshift-network-diagnostics/network-check-target-khqq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 28 19:17:03.588226 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:03.588205    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8 podName:ad86af47-f5e9-4e6f-8640-b3771e91074e nodeName:}" failed. No retries permitted until 2026-04-28 19:17:11.588185847 +0000 UTC m=+18.412525956 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-j6gc8" (UniqueName: "kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8") pod "network-check-target-khqq9" (UID: "ad86af47-f5e9-4e6f-8640-b3771e91074e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 28 19:17:03.939777 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:03.939271    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:03.939777 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:03.939387    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:17:03.939777 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:03.939533    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:03.939777 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:03.939615    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:03.939777 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:03.939656    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:03.939777 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:03.939743    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:17:04.092793 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:04.092190    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:04.092793 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:04.092368    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Apr 28 19:17:04.092793 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:04.092450    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret podName:a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e nodeName:}" failed. No retries permitted until 2026-04-28 19:17:12.092431771 +0000 UTC m=+18.916771878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret") pod "global-pull-secret-syncer-rw72g" (UID: "a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e") : object "kube-system"/"original-pull-secret" not registered
Apr 28 19:17:05.938489 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:05.938436    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:05.938489 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:05.938470    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:05.939070 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:05.938436    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:05.939070 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:05.938584    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:17:05.939070 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:05.938721    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:17:05.939070 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:05.938814    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:07.938331 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:07.938296    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:07.938331 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:07.938334    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:07.938829 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:07.938296    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:07.938829 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:07.938454    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:17:07.938829 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:07.938585    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:17:07.938829 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:07.938669    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:09.938473 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:09.938442    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:09.938924 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:09.938442    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:09.938924 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:09.938568    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:17:09.938924 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:09.938443    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:09.938924 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:09.938680    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:17:09.938924 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:09.938794    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:11.549710 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:11.549668    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:11.550197 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:11.549819    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 28 19:17:11.550197 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:11.549876    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs podName:e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:27.549860929 +0000 UTC m=+34.374201028 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs") pod "network-metrics-daemon-hgm2f" (UID: "e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 28 19:17:11.650444 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:11.650390    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j6gc8\" (UniqueName: \"kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8\") pod \"network-check-target-khqq9\" (UID: \"ad86af47-f5e9-4e6f-8640-b3771e91074e\") " pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:11.650604 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:11.650537    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 28 19:17:11.650604 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:11.650562    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 28 19:17:11.650604 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:11.650576    2569 projected.go:194] Error preparing data for projected volume kube-api-access-j6gc8 for pod openshift-network-diagnostics/network-check-target-khqq9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 28 19:17:11.650735 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:11.650642    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8 podName:ad86af47-f5e9-4e6f-8640-b3771e91074e nodeName:}" failed. No retries permitted until 2026-04-28 19:17:27.650623043 +0000 UTC m=+34.474963134 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-j6gc8" (UniqueName: "kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8") pod "network-check-target-khqq9" (UID: "ad86af47-f5e9-4e6f-8640-b3771e91074e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 28 19:17:11.938230 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:11.938160    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:11.938230 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:11.938215    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:11.938390 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:11.938306    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:11.938390 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:11.938306    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:11.938467 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:11.938429    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:17:11.938533 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:11.938505    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:17:12.153518 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:12.153475    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:12.153680 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:12.153659    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Apr 28 19:17:12.153749 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:12.153738    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret podName:a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e nodeName:}" failed. No retries permitted until 2026-04-28 19:17:28.153718397 +0000 UTC m=+34.978058486 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret") pod "global-pull-secret-syncer-rw72g" (UID: "a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e") : object "kube-system"/"original-pull-secret" not registered
Apr 28 19:17:13.940763 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:13.940667    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:13.941533 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:13.940788    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:13.941533 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:13.940866    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:17:13.941533 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:13.940936    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:13.941533 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:13.940997    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:17:13.941533 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:13.940782    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:14.061597 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:14.061023    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tzfn9" event={"ID":"8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad","Type":"ContainerStarted","Data":"8ad7e8e0e02e5f773286a9c54e4244f67be6ebf0afa8bfa97b777c526d3bff27"}
Apr 28 19:17:14.066649 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:14.066609    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5" event={"ID":"578ad6cd-a560-4a18-8806-846b76fc9338","Type":"ContainerStarted","Data":"8bb746f9d8988af6f5b893ae9fe81d8fcd6fc98f0bfabf5faedafa4560fff500"}
Apr 28 19:17:14.068655 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:14.068610    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw" event={"ID":"e66fe8ab-55e6-4b17-a723-bfd159f94c5d","Type":"ContainerStarted","Data":"22aa64ec04af47ea3dc9dc13ec8e994223a7d9ccd143ce034abf245737845a24"}
Apr 28 19:17:14.070048 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:14.069936    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bp2ht" event={"ID":"25d9e6b9-b643-471e-ae03-5d308920bbbb","Type":"ContainerStarted","Data":"db7a087f103cbcce1f41047d25d6654cd6847d71ffcdbfb105c1ca3fa3f32a2d"}
Apr 28 19:17:14.072327 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:14.072292    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xkhgf" event={"ID":"6a7302f5-8d7a-4665-af2a-81b3fe58a0c4","Type":"ContainerStarted","Data":"5fef8e90fdbecf2a10460a66357942cd87e1dd02e9a8391e16a384aceb71c350"}
Apr 28 19:17:14.081089 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:14.080947    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tzfn9" podStartSLOduration=7.67137038 podStartE2EDuration="20.08093735s" podCreationTimestamp="2026-04-28 19:16:54 +0000 UTC" firstStartedPulling="2026-04-28 19:16:56.506794928 +0000 UTC m=+3.331135029" lastFinishedPulling="2026-04-28 19:17:08.916361912 +0000 UTC m=+15.740701999" observedRunningTime="2026-04-28 19:17:14.080684433 +0000 UTC m=+20.905024562" watchObservedRunningTime="2026-04-28 19:17:14.08093735 +0000 UTC m=+20.905277457"
Apr 28 19:17:14.081281 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:14.081252    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" podStartSLOduration=19.081246524 podStartE2EDuration="19.081246524s" podCreationTimestamp="2026-04-28 19:16:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:16:59.066553214 +0000 UTC m=+5.890893334" watchObservedRunningTime="2026-04-28 19:17:14.081246524 +0000 UTC m=+20.905586632"
Apr 28 19:17:14.127664 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:14.127627    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jxpx9" podStartSLOduration=2.852952996 podStartE2EDuration="20.127611072s" podCreationTimestamp="2026-04-28 19:16:54 +0000 UTC" firstStartedPulling="2026-04-28 19:16:56.503067436 +0000 UTC m=+3.327407536" lastFinishedPulling="2026-04-28 19:17:13.777725519 +0000 UTC m=+20.602065612" observedRunningTime="2026-04-28 19:17:14.105748098 +0000 UTC m=+20.930088203" watchObservedRunningTime="2026-04-28 19:17:14.127611072 +0000 UTC m=+20.951951179"
Apr 28 19:17:14.127974 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:14.127951    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xkhgf" podStartSLOduration=3.846558086 podStartE2EDuration="21.127945249s" podCreationTimestamp="2026-04-28 19:16:53 +0000 UTC" firstStartedPulling="2026-04-28 19:16:56.497998902 +0000 UTC m=+3.322339003" lastFinishedPulling="2026-04-28 19:17:13.779386066 +0000 UTC m=+20.603726166" observedRunningTime="2026-04-28 19:17:14.127269611 +0000 UTC m=+20.951609720" watchObservedRunningTime="2026-04-28 19:17:14.127945249 +0000 UTC m=+20.952285353"
Apr 28 19:17:14.152713 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:14.152667    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-bp2ht" podStartSLOduration=2.767040364 podStartE2EDuration="20.152651186s" podCreationTimestamp="2026-04-28 19:16:54 +0000 UTC" firstStartedPulling="2026-04-28 19:16:56.473895113 +0000 UTC m=+3.298235199" lastFinishedPulling="2026-04-28 19:17:13.859505921 +0000 UTC m=+20.683846021" observedRunningTime="2026-04-28 19:17:14.152468913 +0000 UTC m=+20.976809021" watchObservedRunningTime="2026-04-28 19:17:14.152651186 +0000 UTC m=+20.976991294"
Apr 28 19:17:15.075786 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.075563    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jxpx9" event={"ID":"2a07090c-27b9-4995-8133-9976a9919809","Type":"ContainerStarted","Data":"e73dd62a7aa8030235f0605073faf81244ad5a8fc65b665672263526e8b2720b"}
Apr 28 19:17:15.077244 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.077221    2569 generic.go:358] "Generic (PLEG): container finished" podID="1fd16e46-3b64-4b7f-9d06-6ed637847678" containerID="c13857638b99de189cda0198f165be77e3106ed794dd2999348c8f77c11142f5" exitCode=0
Apr 28 19:17:15.077344 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.077301    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvd4r" event={"ID":"1fd16e46-3b64-4b7f-9d06-6ed637847678","Type":"ContainerDied","Data":"c13857638b99de189cda0198f165be77e3106ed794dd2999348c8f77c11142f5"}
Apr 28 19:17:15.077569 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.077547    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Apr 28 19:17:15.078762 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.078733    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qpmnk" event={"ID":"989021b3-c3c6-43a9-8d1e-c2a8660c0d89","Type":"ContainerStarted","Data":"2fcc6b4c33117f29b5217cc320c34327d8b87eaa023411ad9b0800ae0c7589b8"}
Apr 28 19:17:15.081318 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.081300    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:17:15.081610 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.081590    2569 generic.go:358] "Generic (PLEG): container finished" podID="578ad6cd-a560-4a18-8806-846b76fc9338" containerID="c10ac7f146441e10d9bdc1112737ad5975a6f445a89742d7e36e4aebb8c38cf0" exitCode=1
Apr 28 19:17:15.081706 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.081649    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5" event={"ID":"578ad6cd-a560-4a18-8806-846b76fc9338","Type":"ContainerDied","Data":"c10ac7f146441e10d9bdc1112737ad5975a6f445a89742d7e36e4aebb8c38cf0"}
Apr 28 19:17:15.081706 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.081665    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5" event={"ID":"578ad6cd-a560-4a18-8806-846b76fc9338","Type":"ContainerStarted","Data":"05c2fc836e66735be29cc5d04ab0cdd82be61409884284d21b36cf75ebde7bdc"}
Apr 28 19:17:15.081706 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.081674    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5" event={"ID":"578ad6cd-a560-4a18-8806-846b76fc9338","Type":"ContainerStarted","Data":"74ea79facbd54cd726ced687e1ed126c9fa741f55241263eb904adfdc4f2cdab"}
Apr 28 19:17:15.081706 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.081682    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5" event={"ID":"578ad6cd-a560-4a18-8806-846b76fc9338","Type":"ContainerStarted","Data":"b959ad0b42a9ffd034b49fbac711fa354f748223bd5f393b4d3059ae5adb78ab"}
Apr 28 19:17:15.081706 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.081695    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5" event={"ID":"578ad6cd-a560-4a18-8806-846b76fc9338","Type":"ContainerStarted","Data":"e068ecdbb75eac8fb78d5a2165c0728ef5ad62d954480fc8e1b6f780842fc307"}
Apr 28 19:17:15.083262 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.083229    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw" event={"ID":"e66fe8ab-55e6-4b17-a723-bfd159f94c5d","Type":"ContainerStarted","Data":"42d129a21cef43fb626ff127bfe465893fb6c6298080dacf287cb4ed429a443a"}
Apr 28 19:17:15.129246 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.129206    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-qpmnk" podStartSLOduration=3.86595595 podStartE2EDuration="21.129193772s" podCreationTimestamp="2026-04-28 19:16:54 +0000 UTC" firstStartedPulling="2026-04-28 19:16:56.497995839 +0000 UTC m=+3.322335926" lastFinishedPulling="2026-04-28 19:17:13.761233656 +0000 UTC m=+20.585573748" observedRunningTime="2026-04-28 19:17:15.129120019 +0000 UTC m=+21.953460126" watchObservedRunningTime="2026-04-28 19:17:15.129193772 +0000 UTC m=+21.953533881"
Apr 28 19:17:15.881614 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.881514    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-04-28T19:17:15.077564911Z","UUID":"2d3ce650-48e9-46cf-82aa-4f9abb3cee00","Handler":null,"Name":"","Endpoint":""}
Apr 28 19:17:15.883293 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.883266    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Apr 28 19:17:15.883293 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.883293    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Apr 28 19:17:15.938840 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.938811    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:15.938967 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:15.938939    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:15.939133 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.939114    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:15.939247 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:15.939222    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:17:15.939414 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:15.939377    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:15.939515 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:15.939492    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:17:16.087458 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:16.087429    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw" event={"ID":"e66fe8ab-55e6-4b17-a723-bfd159f94c5d","Type":"ContainerStarted","Data":"1c9f7fc7975964dbf8345eb094d25f4b0f36068014f9c02d1f8b20a03a2f401e"}
Apr 28 19:17:16.091382 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:16.091237    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fbwbg" event={"ID":"88298985-0c2c-4787-9fa7-539c47c429e2","Type":"ContainerStarted","Data":"1afadd38749e6f5d8af00811e62d46e8c859117ceda7c9912083ea52d4302321"}
Apr 28 19:17:16.131205 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:16.131160    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pqhw" podStartSLOduration=2.7132204140000002 podStartE2EDuration="22.131147021s" podCreationTimestamp="2026-04-28 19:16:54 +0000 UTC" firstStartedPulling="2026-04-28 19:16:56.50337352 +0000 UTC m=+3.327713616" lastFinishedPulling="2026-04-28 19:17:15.921300131 +0000 UTC m=+22.745640223" observedRunningTime="2026-04-28 19:17:16.114523997 +0000 UTC m=+22.938864132" watchObservedRunningTime="2026-04-28 19:17:16.131147021 +0000 UTC m=+22.955487131"
Apr 28 19:17:17.095918 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:17.095894    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:17:17.096436 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:17.096309    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5" event={"ID":"578ad6cd-a560-4a18-8806-846b76fc9338","Type":"ContainerStarted","Data":"23df0831795a58320f7f6e9441d7934a84b4b77cb520555688121778ab83ea12"}
Apr 28 19:17:17.938808 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:17.938771    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:17.938982 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:17.938913    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:17:17.939356 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:17.939340    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:17.939465 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:17.939447    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:17:17.939630 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:17.939604    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:17.939732 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:17.939692    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:18.434227 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:18.434202    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jxpx9"
Apr 28 19:17:18.435051 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:18.435030    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jxpx9"
Apr 28 19:17:18.460684 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:18.460632    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fbwbg" podStartSLOduration=8.159852025 podStartE2EDuration="25.46062056s" podCreationTimestamp="2026-04-28 19:16:53 +0000 UTC" firstStartedPulling="2026-04-28 19:16:56.477013201 +0000 UTC m=+3.301353288" lastFinishedPulling="2026-04-28 19:17:13.777781734 +0000 UTC m=+20.602121823" observedRunningTime="2026-04-28 19:17:16.13083295 +0000 UTC m=+22.955173059" watchObservedRunningTime="2026-04-28 19:17:18.46062056 +0000 UTC m=+25.284960668"
Apr 28 19:17:19.103416 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:19.103168    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:17:19.104017 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:19.103778    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5" event={"ID":"578ad6cd-a560-4a18-8806-846b76fc9338","Type":"ContainerStarted","Data":"ac57efcdcc786037f6da562e95f8e38a891f144e32964d605f67668444bf99c4"}
Apr 28 19:17:19.104017 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:19.103975    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jxpx9"
Apr 28 19:17:19.104883 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:19.104865    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jxpx9"
Apr 28 19:17:19.938731 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:19.938700    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:19.938731 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:19.938719    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:19.939450 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:19.938719    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:19.939450 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:19.938824    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:17:19.939450 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:19.938894    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:17:19.939450 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:19.938975    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:20.107035 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:20.107008    2569 generic.go:358] "Generic (PLEG): container finished" podID="1fd16e46-3b64-4b7f-9d06-6ed637847678" containerID="eae62a912baf582bd2d736e778db327e355f44a4875d5c749bed70ef703dd61c" exitCode=0
Apr 28 19:17:20.107184 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:20.107099    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvd4r" event={"ID":"1fd16e46-3b64-4b7f-9d06-6ed637847678","Type":"ContainerDied","Data":"eae62a912baf582bd2d736e778db327e355f44a4875d5c749bed70ef703dd61c"}
Apr 28 19:17:20.107533 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:20.107516    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:17:20.107648 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:20.107631    2569 scope.go:117] "RemoveContainer" containerID="c10ac7f146441e10d9bdc1112737ad5975a6f445a89742d7e36e4aebb8c38cf0"
Apr 28 19:17:20.122547 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:20.122529    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:17:21.012192 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.011962    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-khqq9"]
Apr 28 19:17:21.012687 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.012282    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:21.012687 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:21.012412    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:21.015086 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.015061    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rw72g"]
Apr 28 19:17:21.015171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.015146    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:21.015257 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:21.015237    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:17:21.015775 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.015738    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hgm2f"]
Apr 28 19:17:21.015864 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.015813    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:21.015925 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:21.015890    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:17:21.112265 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.112210    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:17:21.112616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.112593    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5" event={"ID":"578ad6cd-a560-4a18-8806-846b76fc9338","Type":"ContainerStarted","Data":"b28d18e69c06c1a328b76b6377fc35066f155fa01a96481d0bc186a658c1f0ec"}
Apr 28 19:17:21.112824 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.112801    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:17:21.112932 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.112831    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:17:21.114533 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.114507    2569 generic.go:358] "Generic (PLEG): container finished" podID="1fd16e46-3b64-4b7f-9d06-6ed637847678" containerID="e6b115137f4e302f18998fea0dcc005b599d98ec870fdd81267417a2aef90497" exitCode=0
Apr 28 19:17:21.114623 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.114535    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvd4r" event={"ID":"1fd16e46-3b64-4b7f-9d06-6ed637847678","Type":"ContainerDied","Data":"e6b115137f4e302f18998fea0dcc005b599d98ec870fdd81267417a2aef90497"}
Apr 28 19:17:21.128196 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.128169    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:17:21.188140 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:21.188095    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5" podStartSLOduration=9.873211784 podStartE2EDuration="27.188082116s" podCreationTimestamp="2026-04-28 19:16:54 +0000 UTC" firstStartedPulling="2026-04-28 19:16:56.506882695 +0000 UTC m=+3.331222785" lastFinishedPulling="2026-04-28 19:17:13.821753017 +0000 UTC m=+20.646093117" observedRunningTime="2026-04-28 19:17:21.158171434 +0000 UTC m=+27.982511542" watchObservedRunningTime="2026-04-28 19:17:21.188082116 +0000 UTC m=+28.012422218"
Apr 28 19:17:22.118524 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:22.118492    2569 generic.go:358] "Generic (PLEG): container finished" podID="1fd16e46-3b64-4b7f-9d06-6ed637847678" containerID="5f3a5ccb9776258a85cec829be5b08687f482e7c3ca05faf5240f6e0d02a5e75" exitCode=0
Apr 28 19:17:22.118984 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:22.118568    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvd4r" event={"ID":"1fd16e46-3b64-4b7f-9d06-6ed637847678","Type":"ContainerDied","Data":"5f3a5ccb9776258a85cec829be5b08687f482e7c3ca05faf5240f6e0d02a5e75"}
Apr 28 19:17:22.939074 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:22.938998    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:22.939209 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:22.938998    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:22.939209 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:22.939099    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:22.939209 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:22.939170    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:17:22.939209 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:22.939010    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:22.939364 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:22.939257    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:17:24.938939 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:24.938762    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:24.939429 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:24.938762    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:24.939429 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:24.939008    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:17:24.939429 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:24.938762    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:24.939429 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:24.939109    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-khqq9" podUID="ad86af47-f5e9-4e6f-8640-b3771e91074e"
Apr 28 19:17:24.939429 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:24.939161    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rw72g" podUID="a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e"
Apr 28 19:17:25.960644 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:25.960615    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeReady"
Apr 28 19:17:25.961077 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:25.960735    2569 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
Apr 28 19:17:26.016862 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.016835    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lt6tx"]
Apr 28 19:17:26.046685 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.046641    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g4hd4"]
Apr 28 19:17:26.046854 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.046689    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:26.050510 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.050199    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Apr 28 19:17:26.050510 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.050266    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jtx2j\""
Apr 28 19:17:26.050510 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.050306    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Apr 28 19:17:26.064349 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.064324    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lt6tx"]
Apr 28 19:17:26.064490 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.064433    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g4hd4"]
Apr 28 19:17:26.064490 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.064483    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:17:26.067309 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.067285    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Apr 28 19:17:26.067435 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.067374    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Apr 28 19:17:26.068810 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.068779    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-524nx\""
Apr 28 19:17:26.068959 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.068940    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Apr 28 19:17:26.159184 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.159157    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/35caf455-585a-430d-be29-50d6ad53fcd7-config-volume\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:26.159311 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.159198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:17:26.159311 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.159228    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x268p\" (UniqueName: \"kubernetes.io/projected/35caf455-585a-430d-be29-50d6ad53fcd7-kube-api-access-x268p\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:26.159311 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.159268    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35caf455-585a-430d-be29-50d6ad53fcd7-tmp-dir\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:26.159484 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.159317    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:26.159484 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.159340    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dm26\" (UniqueName: \"kubernetes.io/projected/f38d24be-b797-499f-8593-7b9ec113f103-kube-api-access-5dm26\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:17:26.259911 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.259836    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35caf455-585a-430d-be29-50d6ad53fcd7-tmp-dir\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:26.259911 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.259902    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:26.260120 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.259933    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5dm26\" (UniqueName: \"kubernetes.io/projected/f38d24be-b797-499f-8593-7b9ec113f103-kube-api-access-5dm26\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:17:26.260120 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.259989    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/35caf455-585a-430d-be29-50d6ad53fcd7-config-volume\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:26.260120 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.260032    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:17:26.260120 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.260064    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x268p\" (UniqueName: \"kubernetes.io/projected/35caf455-585a-430d-be29-50d6ad53fcd7-kube-api-access-x268p\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:26.260120 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:26.260077    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 28 19:17:26.260344 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:26.260162    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls podName:35caf455-585a-430d-be29-50d6ad53fcd7 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:26.760140028 +0000 UTC m=+33.584480127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls") pod "dns-default-lt6tx" (UID: "35caf455-585a-430d-be29-50d6ad53fcd7") : secret "dns-default-metrics-tls" not found
Apr 28 19:17:26.260344 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:26.260179    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 28 19:17:26.260344 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.260205    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35caf455-585a-430d-be29-50d6ad53fcd7-tmp-dir\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:26.260344 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:26.260238    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert podName:f38d24be-b797-499f-8593-7b9ec113f103 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:26.760222861 +0000 UTC m=+33.584562965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert") pod "ingress-canary-g4hd4" (UID: "f38d24be-b797-499f-8593-7b9ec113f103") : secret "canary-serving-cert" not found
Apr 28 19:17:26.260551 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.260540    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/35caf455-585a-430d-be29-50d6ad53fcd7-config-volume\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:26.273164 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.273140    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x268p\" (UniqueName: \"kubernetes.io/projected/35caf455-585a-430d-be29-50d6ad53fcd7-kube-api-access-x268p\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:26.273371 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.273352    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dm26\" (UniqueName: \"kubernetes.io/projected/f38d24be-b797-499f-8593-7b9ec113f103-kube-api-access-5dm26\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:17:26.763899 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.763848    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:17:26.764088 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.763931    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:26.764088 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:26.764031    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 28 19:17:26.764088 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:26.764032    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 28 19:17:26.764088 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:26.764081    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls podName:35caf455-585a-430d-be29-50d6ad53fcd7 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:27.764064242 +0000 UTC m=+34.588404328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls") pod "dns-default-lt6tx" (UID: "35caf455-585a-430d-be29-50d6ad53fcd7") : secret "dns-default-metrics-tls" not found
Apr 28 19:17:26.764298 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:26.764108    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert podName:f38d24be-b797-499f-8593-7b9ec113f103 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:27.76409071 +0000 UTC m=+34.588430799 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert") pod "ingress-canary-g4hd4" (UID: "f38d24be-b797-499f-8593-7b9ec113f103") : secret "canary-serving-cert" not found
Apr 28 19:17:26.939035 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.938995    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:26.939189 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.939072    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:26.939189 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.939171    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:26.943630 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.943372    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Apr 28 19:17:26.943630 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.943480    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Apr 28 19:17:26.943630 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.943482    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cnh2p\""
Apr 28 19:17:26.943630 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.943533    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Apr 28 19:17:26.943630 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.943482    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Apr 28 19:17:26.943630 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:26.943544    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9tnjq\""
Apr 28 19:17:27.569853 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:27.569813    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:27.570238 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:27.569963    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Apr 28 19:17:27.570238 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:27.570037    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs podName:e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:59.570021539 +0000 UTC m=+66.394361628 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs") pod "network-metrics-daemon-hgm2f" (UID: "e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33") : secret "metrics-daemon-secret" not found
Apr 28 19:17:27.670252 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:27.670211    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j6gc8\" (UniqueName: \"kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8\") pod \"network-check-target-khqq9\" (UID: \"ad86af47-f5e9-4e6f-8640-b3771e91074e\") " pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:27.673901 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:27.673871    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6gc8\" (UniqueName: \"kubernetes.io/projected/ad86af47-f5e9-4e6f-8640-b3771e91074e-kube-api-access-j6gc8\") pod \"network-check-target-khqq9\" (UID: \"ad86af47-f5e9-4e6f-8640-b3771e91074e\") " pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:27.771200 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:27.771133    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:27.771386 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:27.771242    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:17:27.771386 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:27.771306    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 28 19:17:27.771386 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:27.771349    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 28 19:17:27.771386 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:27.771387    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls podName:35caf455-585a-430d-be29-50d6ad53fcd7 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:29.771366662 +0000 UTC m=+36.595706749 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls") pod "dns-default-lt6tx" (UID: "35caf455-585a-430d-be29-50d6ad53fcd7") : secret "dns-default-metrics-tls" not found
Apr 28 19:17:27.771627 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:27.771419    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert podName:f38d24be-b797-499f-8593-7b9ec113f103 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:29.771411199 +0000 UTC m=+36.595751285 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert") pod "ingress-canary-g4hd4" (UID: "f38d24be-b797-499f-8593-7b9ec113f103") : secret "canary-serving-cert" not found
Apr 28 19:17:27.859959 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:27.859874    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:28.174969 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:28.174888    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:28.177559 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:28.177534    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e-original-pull-secret\") pod \"global-pull-secret-syncer-rw72g\" (UID: \"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e\") " pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:28.460669 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:28.460632    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rw72g"
Apr 28 19:17:29.063320 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:29.063276    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-khqq9"]
Apr 28 19:17:29.069547 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:29.069520    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rw72g"]
Apr 28 19:17:29.151925 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:17:29.151849    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad86af47_f5e9_4e6f_8640_b3771e91074e.slice/crio-479c70fc9dc873ddac6796f08a05f971e6a9b7cb39c9c224636be90e7033ca29 WatchSource:0}: Error finding container 479c70fc9dc873ddac6796f08a05f971e6a9b7cb39c9c224636be90e7033ca29: Status 404 returned error can't find the container with id 479c70fc9dc873ddac6796f08a05f971e6a9b7cb39c9c224636be90e7033ca29
Apr 28 19:17:29.152793 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:17:29.152761    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda444a3ec_874d_4d0f_a9f8_4cc5a9c3846e.slice/crio-9d7fbefb6871d03fe0c6bd4749e34175571bb914a1414e3eb315c3a4f4948ab7 WatchSource:0}: Error finding container 9d7fbefb6871d03fe0c6bd4749e34175571bb914a1414e3eb315c3a4f4948ab7: Status 404 returned error can't find the container with id 9d7fbefb6871d03fe0c6bd4749e34175571bb914a1414e3eb315c3a4f4948ab7
Apr 28 19:17:29.787843 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:29.787367    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:17:29.787843 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:29.787701    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:29.787843 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:29.787555    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 28 19:17:29.788090 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:29.787878    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert podName:f38d24be-b797-499f-8593-7b9ec113f103 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:33.787860558 +0000 UTC m=+40.612200658 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert") pod "ingress-canary-g4hd4" (UID: "f38d24be-b797-499f-8593-7b9ec113f103") : secret "canary-serving-cert" not found
Apr 28 19:17:29.788221 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:29.787817    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 28 19:17:29.788316 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:29.788270    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls podName:35caf455-585a-430d-be29-50d6ad53fcd7 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:33.788251227 +0000 UTC m=+40.612591313 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls") pod "dns-default-lt6tx" (UID: "35caf455-585a-430d-be29-50d6ad53fcd7") : secret "dns-default-metrics-tls" not found
Apr 28 19:17:30.136082 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:30.135986    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-khqq9" event={"ID":"ad86af47-f5e9-4e6f-8640-b3771e91074e","Type":"ContainerStarted","Data":"479c70fc9dc873ddac6796f08a05f971e6a9b7cb39c9c224636be90e7033ca29"}
Apr 28 19:17:30.139542 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:30.139508    2569 generic.go:358] "Generic (PLEG): container finished" podID="1fd16e46-3b64-4b7f-9d06-6ed637847678" containerID="51e93db9af4f64408ad86560ad5cefff5c5fef956ed8c30b43e5c63fa5d193af" exitCode=0
Apr 28 19:17:30.139682 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:30.139587    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvd4r" event={"ID":"1fd16e46-3b64-4b7f-9d06-6ed637847678","Type":"ContainerDied","Data":"51e93db9af4f64408ad86560ad5cefff5c5fef956ed8c30b43e5c63fa5d193af"}
Apr 28 19:17:30.140846 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:30.140808    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rw72g" event={"ID":"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e","Type":"ContainerStarted","Data":"9d7fbefb6871d03fe0c6bd4749e34175571bb914a1414e3eb315c3a4f4948ab7"}
Apr 28 19:17:31.146530 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:31.146493    2569 generic.go:358] "Generic (PLEG): container finished" podID="1fd16e46-3b64-4b7f-9d06-6ed637847678" containerID="5b283b7ad75c7a8cad5fec9c8e92cd6c1e9fdab2a949035d02c05f9df560a471" exitCode=0
Apr 28 19:17:31.147015 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:31.146545    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvd4r" event={"ID":"1fd16e46-3b64-4b7f-9d06-6ed637847678","Type":"ContainerDied","Data":"5b283b7ad75c7a8cad5fec9c8e92cd6c1e9fdab2a949035d02c05f9df560a471"}
Apr 28 19:17:33.821277 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:33.821251    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:17:33.821754 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:33.821301    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:33.821754 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:33.821417    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 28 19:17:33.821754 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:33.821475    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls podName:35caf455-585a-430d-be29-50d6ad53fcd7 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:41.821456486 +0000 UTC m=+48.645796575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls") pod "dns-default-lt6tx" (UID: "35caf455-585a-430d-be29-50d6ad53fcd7") : secret "dns-default-metrics-tls" not found
Apr 28 19:17:33.821754 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:33.821414    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 28 19:17:33.821754 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:33.821557    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert podName:f38d24be-b797-499f-8593-7b9ec113f103 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:41.821539172 +0000 UTC m=+48.645879262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert") pod "ingress-canary-g4hd4" (UID: "f38d24be-b797-499f-8593-7b9ec113f103") : secret "canary-serving-cert" not found
Apr 28 19:17:34.153762 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:34.153684    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-khqq9" event={"ID":"ad86af47-f5e9-4e6f-8640-b3771e91074e","Type":"ContainerStarted","Data":"46bf852725abd53decfd2d94b84779b3231df04ef8d746e96efb7976050a3a8c"}
Apr 28 19:17:34.153896 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:34.153817    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:17:34.156674 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:34.156655    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvd4r" event={"ID":"1fd16e46-3b64-4b7f-9d06-6ed637847678","Type":"ContainerStarted","Data":"bc887336b51032d2d293e2b1024c818432d3de42f32a60a2a2d9c2754bc18b2e"}
Apr 28 19:17:34.157954 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:34.157936    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rw72g" event={"ID":"a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e","Type":"ContainerStarted","Data":"5f26270f69cb2c8f242b92e3ae644ef20860df100146c9c90e179eab25fad164"}
Apr 28 19:17:34.170800 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:34.170765    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-khqq9" podStartSLOduration=35.53686085 podStartE2EDuration="40.170753052s" podCreationTimestamp="2026-04-28 19:16:54 +0000 UTC" firstStartedPulling="2026-04-28 19:17:29.163503275 +0000 UTC m=+35.987843362" lastFinishedPulling="2026-04-28 19:17:33.797395465 +0000 UTC m=+40.621735564" observedRunningTime="2026-04-28 19:17:34.170286164 +0000 UTC m=+40.994626273" watchObservedRunningTime="2026-04-28 19:17:34.170753052 +0000 UTC m=+40.995093216"
Apr 28 19:17:34.185685 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:34.185645    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rw72g" podStartSLOduration=33.543309552 podStartE2EDuration="38.185633833s" podCreationTimestamp="2026-04-28 19:16:56 +0000 UTC" firstStartedPulling="2026-04-28 19:17:29.163578639 +0000 UTC m=+35.987918724" lastFinishedPulling="2026-04-28 19:17:33.805902916 +0000 UTC m=+40.630243005" observedRunningTime="2026-04-28 19:17:34.185082136 +0000 UTC m=+41.009422243" watchObservedRunningTime="2026-04-28 19:17:34.185633833 +0000 UTC m=+41.009973918"
Apr 28 19:17:34.209820 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:34.209777    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mvd4r" podStartSLOduration=7.520568697 podStartE2EDuration="40.209766467s" podCreationTimestamp="2026-04-28 19:16:54 +0000 UTC" firstStartedPulling="2026-04-28 19:16:56.497997925 +0000 UTC m=+3.322338013" lastFinishedPulling="2026-04-28 19:17:29.187195693 +0000 UTC m=+36.011535783" observedRunningTime="2026-04-28 19:17:34.208169676 +0000 UTC m=+41.032509784" watchObservedRunningTime="2026-04-28 19:17:34.209766467 +0000 UTC m=+41.034106624"
Apr 28 19:17:41.870820 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:41.870778    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:17:41.870820 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:41.870832    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:41.871334 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:41.870917    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 28 19:17:41.871334 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:41.870921    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 28 19:17:41.871334 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:41.870977    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls podName:35caf455-585a-430d-be29-50d6ad53fcd7 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:57.870963852 +0000 UTC m=+64.695303939 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls") pod "dns-default-lt6tx" (UID: "35caf455-585a-430d-be29-50d6ad53fcd7") : secret "dns-default-metrics-tls" not found
Apr 28 19:17:41.871334 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:41.870992    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert podName:f38d24be-b797-499f-8593-7b9ec113f103 nodeName:}" failed. No retries permitted until 2026-04-28 19:17:57.870984485 +0000 UTC m=+64.695324571 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert") pod "ingress-canary-g4hd4" (UID: "f38d24be-b797-499f-8593-7b9ec113f103") : secret "canary-serving-cert" not found
Apr 28 19:17:53.131059 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:53.131030    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ptjf5"
Apr 28 19:17:57.879177 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:57.879126    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:17:57.879177 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:57.879191    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:17:57.879633 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:57.879272    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 28 19:17:57.879633 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:57.879350    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert podName:f38d24be-b797-499f-8593-7b9ec113f103 nodeName:}" failed. No retries permitted until 2026-04-28 19:18:29.879333496 +0000 UTC m=+96.703673581 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert") pod "ingress-canary-g4hd4" (UID: "f38d24be-b797-499f-8593-7b9ec113f103") : secret "canary-serving-cert" not found
Apr 28 19:17:57.879633 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:57.879275    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 28 19:17:57.879633 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:57.879440    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls podName:35caf455-585a-430d-be29-50d6ad53fcd7 nodeName:}" failed. No retries permitted until 2026-04-28 19:18:29.879425687 +0000 UTC m=+96.703765774 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls") pod "dns-default-lt6tx" (UID: "35caf455-585a-430d-be29-50d6ad53fcd7") : secret "dns-default-metrics-tls" not found
Apr 28 19:17:59.589028 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:17:59.588989    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:17:59.589423 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:59.589109    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Apr 28 19:17:59.589423 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:17:59.589168    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs podName:e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33 nodeName:}" failed. No retries permitted until 2026-04-28 19:19:03.589154891 +0000 UTC m=+130.413494978 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs") pod "network-metrics-daemon-hgm2f" (UID: "e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33") : secret "metrics-daemon-secret" not found
Apr 28 19:18:05.161821 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:18:05.161790    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-khqq9"
Apr 28 19:18:29.893910 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:18:29.893873    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:18:29.894375 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:18:29.893921    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:18:29.894375 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:18:29.894005    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 28 19:18:29.894375 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:18:29.894008    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 28 19:18:29.894375 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:18:29.894064    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert podName:f38d24be-b797-499f-8593-7b9ec113f103 nodeName:}" failed. No retries permitted until 2026-04-28 19:19:33.894050107 +0000 UTC m=+160.718390193 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert") pod "ingress-canary-g4hd4" (UID: "f38d24be-b797-499f-8593-7b9ec113f103") : secret "canary-serving-cert" not found
Apr 28 19:18:29.894375 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:18:29.894078    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls podName:35caf455-585a-430d-be29-50d6ad53fcd7 nodeName:}" failed. No retries permitted until 2026-04-28 19:19:33.894071605 +0000 UTC m=+160.718411690 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls") pod "dns-default-lt6tx" (UID: "35caf455-585a-430d-be29-50d6ad53fcd7") : secret "dns-default-metrics-tls" not found
Apr 28 19:19:03.627116 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:03.627069    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:19:03.627617 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:03.627238    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Apr 28 19:19:03.627617 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:03.627314    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs podName:e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33 nodeName:}" failed. No retries permitted until 2026-04-28 19:21:05.627297794 +0000 UTC m=+252.451637880 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs") pod "network-metrics-daemon-hgm2f" (UID: "e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33") : secret "metrics-daemon-secret" not found
Apr 28 19:19:11.173203 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.173169    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"]
Apr 28 19:19:11.174910 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.174893    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-6svf6"]
Apr 28 19:19:11.175047 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.175030    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"
Apr 28 19:19:11.177015 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.176994    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.185949 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.185917    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Apr 28 19:19:11.185949 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.185931    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Apr 28 19:19:11.186125 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.185951    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Apr 28 19:19:11.186125 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.186042    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-g9wdw\""
Apr 28 19:19:11.186302 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.186276    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Apr 28 19:19:11.186302 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.186297    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Apr 28 19:19:11.187044 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.187023    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Apr 28 19:19:11.187133 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.187121    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Apr 28 19:19:11.187393 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.187376    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Apr 28 19:19:11.187494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.187376    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gfqvn\""
Apr 28 19:19:11.190877 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.190861    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Apr 28 19:19:11.199378 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.199360    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"]
Apr 28 19:19:11.206135 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.206116    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-6svf6"]
Apr 28 19:19:11.277507 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.277472    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sgbc\" (UniqueName: \"kubernetes.io/projected/034e2b42-803b-45fd-a6df-4371bec965ff-kube-api-access-9sgbc\") pod \"console-operator-9d4b6777b-6svf6\" (UID: \"034e2b42-803b-45fd-a6df-4371bec965ff\") " pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.277666 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.277586    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/034e2b42-803b-45fd-a6df-4371bec965ff-serving-cert\") pod \"console-operator-9d4b6777b-6svf6\" (UID: \"034e2b42-803b-45fd-a6df-4371bec965ff\") " pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.277666 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.277616    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/abc28c3c-a5be-45cd-8095-f9d59dd936dc-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-x8wk8\" (UID: \"abc28c3c-a5be-45cd-8095-f9d59dd936dc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"
Apr 28 19:19:11.277768 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.277690    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/034e2b42-803b-45fd-a6df-4371bec965ff-config\") pod \"console-operator-9d4b6777b-6svf6\" (UID: \"034e2b42-803b-45fd-a6df-4371bec965ff\") " pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.277768 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.277735    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/abc28c3c-a5be-45cd-8095-f9d59dd936dc-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-x8wk8\" (UID: \"abc28c3c-a5be-45cd-8095-f9d59dd936dc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"
Apr 28 19:19:11.277768 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.277762    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/034e2b42-803b-45fd-a6df-4371bec965ff-trusted-ca\") pod \"console-operator-9d4b6777b-6svf6\" (UID: \"034e2b42-803b-45fd-a6df-4371bec965ff\") " pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.277895 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.277808    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx5j4\" (UniqueName: \"kubernetes.io/projected/abc28c3c-a5be-45cd-8095-f9d59dd936dc-kube-api-access-tx5j4\") pod \"kube-storage-version-migrator-operator-6769c5d45-x8wk8\" (UID: \"abc28c3c-a5be-45cd-8095-f9d59dd936dc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"
Apr 28 19:19:11.278709 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.278690    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6789dcc546-g9gsl"]
Apr 28 19:19:11.280474 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.280460    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.285754 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.285736    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-m25ps\""
Apr 28 19:19:11.285854 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.285779    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Apr 28 19:19:11.286029 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.286015    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Apr 28 19:19:11.290342 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.290324    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Apr 28 19:19:11.292981 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.292965    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Apr 28 19:19:11.313525 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.313502    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6789dcc546-g9gsl"]
Apr 28 19:19:11.368373 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.368351    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-585dfdc468-x7kx7"]
Apr 28 19:19:11.370102 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.370087    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.373344 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.373327    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-69j57\""
Apr 28 19:19:11.373661 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.373637    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Apr 28 19:19:11.373770 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.373752    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Apr 28 19:19:11.374013 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.374000    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Apr 28 19:19:11.374280 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.374268    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Apr 28 19:19:11.378474 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.378449    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/af47071b-d8de-49e7-b564-1ab0414ff0e9-ca-trust-extracted\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.378575 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.378526    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af47071b-d8de-49e7-b564-1ab0414ff0e9-trusted-ca\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.378642 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.378604    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.378696 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.378640    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/af47071b-d8de-49e7-b564-1ab0414ff0e9-installation-pull-secrets\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.378752 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.378733    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/034e2b42-803b-45fd-a6df-4371bec965ff-serving-cert\") pod \"console-operator-9d4b6777b-6svf6\" (UID: \"034e2b42-803b-45fd-a6df-4371bec965ff\") " pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.378813 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.378780    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/abc28c3c-a5be-45cd-8095-f9d59dd936dc-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-x8wk8\" (UID: \"abc28c3c-a5be-45cd-8095-f9d59dd936dc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"
Apr 28 19:19:11.378868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.378830    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/af47071b-d8de-49e7-b564-1ab0414ff0e9-image-registry-private-configuration\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.378921 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.378880    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-bound-sa-token\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.378971 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.378917    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6tks\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-kube-api-access-c6tks\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.379023 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.378966    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/034e2b42-803b-45fd-a6df-4371bec965ff-config\") pod \"console-operator-9d4b6777b-6svf6\" (UID: \"034e2b42-803b-45fd-a6df-4371bec965ff\") " pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.379023 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.379004    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-certificates\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.379124 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.379069    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/abc28c3c-a5be-45cd-8095-f9d59dd936dc-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-x8wk8\" (UID: \"abc28c3c-a5be-45cd-8095-f9d59dd936dc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"
Apr 28 19:19:11.379124 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.379103    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/034e2b42-803b-45fd-a6df-4371bec965ff-trusted-ca\") pod \"console-operator-9d4b6777b-6svf6\" (UID: \"034e2b42-803b-45fd-a6df-4371bec965ff\") " pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.379222 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.379144    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tx5j4\" (UniqueName: \"kubernetes.io/projected/abc28c3c-a5be-45cd-8095-f9d59dd936dc-kube-api-access-tx5j4\") pod \"kube-storage-version-migrator-operator-6769c5d45-x8wk8\" (UID: \"abc28c3c-a5be-45cd-8095-f9d59dd936dc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"
Apr 28 19:19:11.379222 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.379184    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9sgbc\" (UniqueName: \"kubernetes.io/projected/034e2b42-803b-45fd-a6df-4371bec965ff-kube-api-access-9sgbc\") pod \"console-operator-9d4b6777b-6svf6\" (UID: \"034e2b42-803b-45fd-a6df-4371bec965ff\") " pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.382126 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.382105    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/034e2b42-803b-45fd-a6df-4371bec965ff-config\") pod \"console-operator-9d4b6777b-6svf6\" (UID: \"034e2b42-803b-45fd-a6df-4371bec965ff\") " pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.382946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.382922    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/abc28c3c-a5be-45cd-8095-f9d59dd936dc-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-x8wk8\" (UID: \"abc28c3c-a5be-45cd-8095-f9d59dd936dc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"
Apr 28 19:19:11.383063 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.382949    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Apr 28 19:19:11.383538 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.383521    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/034e2b42-803b-45fd-a6df-4371bec965ff-trusted-ca\") pod \"console-operator-9d4b6777b-6svf6\" (UID: \"034e2b42-803b-45fd-a6df-4371bec965ff\") " pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.383595 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.383542    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/034e2b42-803b-45fd-a6df-4371bec965ff-serving-cert\") pod \"console-operator-9d4b6777b-6svf6\" (UID: \"034e2b42-803b-45fd-a6df-4371bec965ff\") " pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.384093 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.384077    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/abc28c3c-a5be-45cd-8095-f9d59dd936dc-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-x8wk8\" (UID: \"abc28c3c-a5be-45cd-8095-f9d59dd936dc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"
Apr 28 19:19:11.387665 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.387643    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-585dfdc468-x7kx7"]
Apr 28 19:19:11.391735 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.391715    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx5j4\" (UniqueName: \"kubernetes.io/projected/abc28c3c-a5be-45cd-8095-f9d59dd936dc-kube-api-access-tx5j4\") pod \"kube-storage-version-migrator-operator-6769c5d45-x8wk8\" (UID: \"abc28c3c-a5be-45cd-8095-f9d59dd936dc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"
Apr 28 19:19:11.391950 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.391933    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sgbc\" (UniqueName: \"kubernetes.io/projected/034e2b42-803b-45fd-a6df-4371bec965ff-kube-api-access-9sgbc\") pod \"console-operator-9d4b6777b-6svf6\" (UID: \"034e2b42-803b-45fd-a6df-4371bec965ff\") " pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.480049 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480029    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-bound-sa-token\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.480199 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480059    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/10e853da-a692-43bb-b339-3bf2059fd151-snapshots\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.480199 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480080    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndhd2\" (UniqueName: \"kubernetes.io/projected/10e853da-a692-43bb-b339-3bf2059fd151-kube-api-access-ndhd2\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.480199 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480101    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/10e853da-a692-43bb-b339-3bf2059fd151-serving-cert\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.480199 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480121    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/af47071b-d8de-49e7-b564-1ab0414ff0e9-ca-trust-extracted\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.480347 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480217    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e853da-a692-43bb-b339-3bf2059fd151-service-ca-bundle\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.480347 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480272    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/af47071b-d8de-49e7-b564-1ab0414ff0e9-image-registry-private-configuration\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.480347 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480315    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c6tks\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-kube-api-access-c6tks\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.480473 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480354    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-certificates\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.480473 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480364    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/af47071b-d8de-49e7-b564-1ab0414ff0e9-ca-trust-extracted\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.480473 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480374    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10e853da-a692-43bb-b339-3bf2059fd151-tmp\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.480473 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480436    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af47071b-d8de-49e7-b564-1ab0414ff0e9-trusted-ca\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.480473 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480464    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/af47071b-d8de-49e7-b564-1ab0414ff0e9-installation-pull-secrets\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.480680 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480507    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.480680 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:11.480587    2569 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Apr 28 19:19:11.480680 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:11.480600    2569 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6789dcc546-g9gsl: secret "image-registry-tls" not found
Apr 28 19:19:11.480680 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480620    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e853da-a692-43bb-b339-3bf2059fd151-trusted-ca-bundle\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.480680 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:11.480672    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls podName:af47071b-d8de-49e7-b564-1ab0414ff0e9 nodeName:}" failed. No retries permitted until 2026-04-28 19:19:11.980654964 +0000 UTC m=+138.804995064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls") pod "image-registry-6789dcc546-g9gsl" (UID: "af47071b-d8de-49e7-b564-1ab0414ff0e9") : secret "image-registry-tls" not found
Apr 28 19:19:11.480917 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.480854    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-certificates\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.481150 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.481131    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af47071b-d8de-49e7-b564-1ab0414ff0e9-trusted-ca\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.482688 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.482670    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/af47071b-d8de-49e7-b564-1ab0414ff0e9-installation-pull-secrets\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.482745 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.482718    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/af47071b-d8de-49e7-b564-1ab0414ff0e9-image-registry-private-configuration\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.484533 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.484518    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"
Apr 28 19:19:11.489279 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.489260    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:11.500525 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.500506    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-bound-sa-token\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.501332 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.501307    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6tks\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-kube-api-access-c6tks\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.582096 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.581922    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e853da-a692-43bb-b339-3bf2059fd151-trusted-ca-bundle\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.582096 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.581977    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/10e853da-a692-43bb-b339-3bf2059fd151-snapshots\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.582096 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.582007    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ndhd2\" (UniqueName: \"kubernetes.io/projected/10e853da-a692-43bb-b339-3bf2059fd151-kube-api-access-ndhd2\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.582096 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.582050    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/10e853da-a692-43bb-b339-3bf2059fd151-serving-cert\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.582096 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.582082    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e853da-a692-43bb-b339-3bf2059fd151-service-ca-bundle\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.582599 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.582147    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10e853da-a692-43bb-b339-3bf2059fd151-tmp\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.582599 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.582522    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10e853da-a692-43bb-b339-3bf2059fd151-tmp\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.583089 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.583063    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/10e853da-a692-43bb-b339-3bf2059fd151-snapshots\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.583248 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.583225    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e853da-a692-43bb-b339-3bf2059fd151-service-ca-bundle\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.583389 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.583369    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e853da-a692-43bb-b339-3bf2059fd151-trusted-ca-bundle\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.585086 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.585060    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/10e853da-a692-43bb-b339-3bf2059fd151-serving-cert\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.591195 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.591155    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndhd2\" (UniqueName: \"kubernetes.io/projected/10e853da-a692-43bb-b339-3bf2059fd151-kube-api-access-ndhd2\") pod \"insights-operator-585dfdc468-x7kx7\" (UID: \"10e853da-a692-43bb-b339-3bf2059fd151\") " pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.603937 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.603914    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8"]
Apr 28 19:19:11.606777 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:19:11.606744    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabc28c3c_a5be_45cd_8095_f9d59dd936dc.slice/crio-c86efb006086472bbd04492deb33e00f7d31e1f4a44ca49a4b84f0031957c9b2 WatchSource:0}: Error finding container c86efb006086472bbd04492deb33e00f7d31e1f4a44ca49a4b84f0031957c9b2: Status 404 returned error can't find the container with id c86efb006086472bbd04492deb33e00f7d31e1f4a44ca49a4b84f0031957c9b2
Apr 28 19:19:11.620230 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.620206    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-6svf6"]
Apr 28 19:19:11.622659 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:19:11.622639    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034e2b42_803b_45fd_a6df_4371bec965ff.slice/crio-6d2a23f726b6cf9c97c1356a968df2aa8e76f29fe4efd2ae6af13579c865bf4a WatchSource:0}: Error finding container 6d2a23f726b6cf9c97c1356a968df2aa8e76f29fe4efd2ae6af13579c865bf4a: Status 404 returned error can't find the container with id 6d2a23f726b6cf9c97c1356a968df2aa8e76f29fe4efd2ae6af13579c865bf4a
Apr 28 19:19:11.679475 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.679454    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-585dfdc468-x7kx7"
Apr 28 19:19:11.792056 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.792019    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-585dfdc468-x7kx7"]
Apr 28 19:19:11.795546 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:19:11.795518    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10e853da_a692_43bb_b339_3bf2059fd151.slice/crio-33cc1559782756f4bb69b93bdd2d6abd0d7a33943a3d3cbaef7a4e6dcfb177ec WatchSource:0}: Error finding container 33cc1559782756f4bb69b93bdd2d6abd0d7a33943a3d3cbaef7a4e6dcfb177ec: Status 404 returned error can't find the container with id 33cc1559782756f4bb69b93bdd2d6abd0d7a33943a3d3cbaef7a4e6dcfb177ec
Apr 28 19:19:11.985864 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:11.985828    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:11.986053 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:11.985947    2569 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Apr 28 19:19:11.986053 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:11.985962    2569 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6789dcc546-g9gsl: secret "image-registry-tls" not found
Apr 28 19:19:11.986053 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:11.986023    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls podName:af47071b-d8de-49e7-b564-1ab0414ff0e9 nodeName:}" failed. No retries permitted until 2026-04-28 19:19:12.986005088 +0000 UTC m=+139.810345173 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls") pod "image-registry-6789dcc546-g9gsl" (UID: "af47071b-d8de-49e7-b564-1ab0414ff0e9") : secret "image-registry-tls" not found
Apr 28 19:19:12.347301 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:12.347263    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8" event={"ID":"abc28c3c-a5be-45cd-8095-f9d59dd936dc","Type":"ContainerStarted","Data":"c86efb006086472bbd04492deb33e00f7d31e1f4a44ca49a4b84f0031957c9b2"}
Apr 28 19:19:12.348558 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:12.348516    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-x7kx7" event={"ID":"10e853da-a692-43bb-b339-3bf2059fd151","Type":"ContainerStarted","Data":"33cc1559782756f4bb69b93bdd2d6abd0d7a33943a3d3cbaef7a4e6dcfb177ec"}
Apr 28 19:19:12.349786 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:12.349758    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6" event={"ID":"034e2b42-803b-45fd-a6df-4371bec965ff","Type":"ContainerStarted","Data":"6d2a23f726b6cf9c97c1356a968df2aa8e76f29fe4efd2ae6af13579c865bf4a"}
Apr 28 19:19:12.995149 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:12.995111    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:12.995313 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:12.995270    2569 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Apr 28 19:19:12.995313 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:12.995286    2569 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6789dcc546-g9gsl: secret "image-registry-tls" not found
Apr 28 19:19:12.995435 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:12.995350    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls podName:af47071b-d8de-49e7-b564-1ab0414ff0e9 nodeName:}" failed. No retries permitted until 2026-04-28 19:19:14.995330549 +0000 UTC m=+141.819670642 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls") pod "image-registry-6789dcc546-g9gsl" (UID: "af47071b-d8de-49e7-b564-1ab0414ff0e9") : secret "image-registry-tls" not found
Apr 28 19:19:15.009429 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:15.009373    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:15.009779 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:15.009523    2569 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Apr 28 19:19:15.009779 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:15.009543    2569 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6789dcc546-g9gsl: secret "image-registry-tls" not found
Apr 28 19:19:15.009779 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:15.009595    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls podName:af47071b-d8de-49e7-b564-1ab0414ff0e9 nodeName:}" failed. No retries permitted until 2026-04-28 19:19:19.009581363 +0000 UTC m=+145.833921449 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls") pod "image-registry-6789dcc546-g9gsl" (UID: "af47071b-d8de-49e7-b564-1ab0414ff0e9") : secret "image-registry-tls" not found
Apr 28 19:19:15.358562 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:15.358486    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-x7kx7" event={"ID":"10e853da-a692-43bb-b339-3bf2059fd151","Type":"ContainerStarted","Data":"e475b538e0b5217482754d9a0b908e5e25d207147f1f0f69b5fa2f1306563d4a"}
Apr 28 19:19:15.359862 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:15.359840    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/0.log"
Apr 28 19:19:15.359964 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:15.359878    2569 generic.go:358] "Generic (PLEG): container finished" podID="034e2b42-803b-45fd-a6df-4371bec965ff" containerID="952c71bb03b87589fb2354c781efc8eae3d9339348038aec43014120c43074c6" exitCode=255
Apr 28 19:19:15.359964 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:15.359951    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6" event={"ID":"034e2b42-803b-45fd-a6df-4371bec965ff","Type":"ContainerDied","Data":"952c71bb03b87589fb2354c781efc8eae3d9339348038aec43014120c43074c6"}
Apr 28 19:19:15.360199 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:15.360182    2569 scope.go:117] "RemoveContainer" containerID="952c71bb03b87589fb2354c781efc8eae3d9339348038aec43014120c43074c6"
Apr 28 19:19:15.361278 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:15.361246    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8" event={"ID":"abc28c3c-a5be-45cd-8095-f9d59dd936dc","Type":"ContainerStarted","Data":"9cd9b7cc241ac855faff2dd1e8bf7c8bc70835f3d2c8b1aece1ae3d722c99661"}
Apr 28 19:19:15.530571 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:15.530524    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8" podStartSLOduration=1.740900754 podStartE2EDuration="4.530508762s" podCreationTimestamp="2026-04-28 19:19:11 +0000 UTC" firstStartedPulling="2026-04-28 19:19:11.608614651 +0000 UTC m=+138.432954737" lastFinishedPulling="2026-04-28 19:19:14.39822265 +0000 UTC m=+141.222562745" observedRunningTime="2026-04-28 19:19:15.529426571 +0000 UTC m=+142.353766680" watchObservedRunningTime="2026-04-28 19:19:15.530508762 +0000 UTC m=+142.354848869"
Apr 28 19:19:15.530703 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:15.530655    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-585dfdc468-x7kx7" podStartSLOduration=1.929081321 podStartE2EDuration="4.530650798s" podCreationTimestamp="2026-04-28 19:19:11 +0000 UTC" firstStartedPulling="2026-04-28 19:19:11.797290706 +0000 UTC m=+138.621630791" lastFinishedPulling="2026-04-28 19:19:14.398860181 +0000 UTC m=+141.223200268" observedRunningTime="2026-04-28 19:19:15.433571501 +0000 UTC m=+142.257911609" watchObservedRunningTime="2026-04-28 19:19:15.530650798 +0000 UTC m=+142.354990906"
Apr 28 19:19:16.365372 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:16.365342    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:19:16.365774 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:16.365720    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/0.log"
Apr 28 19:19:16.365774 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:16.365752    2569 generic.go:358] "Generic (PLEG): container finished" podID="034e2b42-803b-45fd-a6df-4371bec965ff" containerID="762d27b7a2ec660ce6b5995c6000b94c0592743745c2ca8e80f736a8a88a472f" exitCode=255
Apr 28 19:19:16.365846 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:16.365830    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6" event={"ID":"034e2b42-803b-45fd-a6df-4371bec965ff","Type":"ContainerDied","Data":"762d27b7a2ec660ce6b5995c6000b94c0592743745c2ca8e80f736a8a88a472f"}
Apr 28 19:19:16.365880 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:16.365865    2569 scope.go:117] "RemoveContainer" containerID="952c71bb03b87589fb2354c781efc8eae3d9339348038aec43014120c43074c6"
Apr 28 19:19:16.366082 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:16.366062    2569 scope.go:117] "RemoveContainer" containerID="762d27b7a2ec660ce6b5995c6000b94c0592743745c2ca8e80f736a8a88a472f"
Apr 28 19:19:16.366281 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:16.366262    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-6svf6_openshift-console-operator(034e2b42-803b-45fd-a6df-4371bec965ff)\"" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6" podUID="034e2b42-803b-45fd-a6df-4371bec965ff"
Apr 28 19:19:17.369159 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:17.369131    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:19:17.369665 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:17.369566    2569 scope.go:117] "RemoveContainer" containerID="762d27b7a2ec660ce6b5995c6000b94c0592743745c2ca8e80f736a8a88a472f"
Apr 28 19:19:17.369796 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:17.369775    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-6svf6_openshift-console-operator(034e2b42-803b-45fd-a6df-4371bec965ff)\"" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6" podUID="034e2b42-803b-45fd-a6df-4371bec965ff"
Apr 28 19:19:18.613713 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.613684    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qpmnk_989021b3-c3c6-43a9-8d1e-c2a8660c0d89/dns-node-resolver/0.log"
Apr 28 19:19:18.685724 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.685697    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-865cb79987-snhtr"]
Apr 28 19:19:18.687863 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.687848    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-865cb79987-snhtr"
Apr 28 19:19:18.690792 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.690763    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Apr 28 19:19:18.690899 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.690767    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Apr 28 19:19:18.692100 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.692076    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-2jz25\""
Apr 28 19:19:18.692199 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.692141    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Apr 28 19:19:18.692263 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.692242    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Apr 28 19:19:18.697825 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.697803    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-865cb79987-snhtr"]
Apr 28 19:19:18.739628 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.739596    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/be0cfd01-5749-4bbc-9b15-76d63a71524b-signing-key\") pod \"service-ca-865cb79987-snhtr\" (UID: \"be0cfd01-5749-4bbc-9b15-76d63a71524b\") " pod="openshift-service-ca/service-ca-865cb79987-snhtr"
Apr 28 19:19:18.739754 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.739643    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/be0cfd01-5749-4bbc-9b15-76d63a71524b-signing-cabundle\") pod \"service-ca-865cb79987-snhtr\" (UID: \"be0cfd01-5749-4bbc-9b15-76d63a71524b\") " pod="openshift-service-ca/service-ca-865cb79987-snhtr"
Apr 28 19:19:18.739754 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.739718    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxttx\" (UniqueName: \"kubernetes.io/projected/be0cfd01-5749-4bbc-9b15-76d63a71524b-kube-api-access-kxttx\") pod \"service-ca-865cb79987-snhtr\" (UID: \"be0cfd01-5749-4bbc-9b15-76d63a71524b\") " pod="openshift-service-ca/service-ca-865cb79987-snhtr"
Apr 28 19:19:18.840238 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.840208    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/be0cfd01-5749-4bbc-9b15-76d63a71524b-signing-cabundle\") pod \"service-ca-865cb79987-snhtr\" (UID: \"be0cfd01-5749-4bbc-9b15-76d63a71524b\") " pod="openshift-service-ca/service-ca-865cb79987-snhtr"
Apr 28 19:19:18.840369 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.840257    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kxttx\" (UniqueName: \"kubernetes.io/projected/be0cfd01-5749-4bbc-9b15-76d63a71524b-kube-api-access-kxttx\") pod \"service-ca-865cb79987-snhtr\" (UID: \"be0cfd01-5749-4bbc-9b15-76d63a71524b\") " pod="openshift-service-ca/service-ca-865cb79987-snhtr"
Apr 28 19:19:18.840369 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.840348    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/be0cfd01-5749-4bbc-9b15-76d63a71524b-signing-key\") pod \"service-ca-865cb79987-snhtr\" (UID: \"be0cfd01-5749-4bbc-9b15-76d63a71524b\") " pod="openshift-service-ca/service-ca-865cb79987-snhtr"
Apr 28 19:19:18.840983 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.840963    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/be0cfd01-5749-4bbc-9b15-76d63a71524b-signing-cabundle\") pod \"service-ca-865cb79987-snhtr\" (UID: \"be0cfd01-5749-4bbc-9b15-76d63a71524b\") " pod="openshift-service-ca/service-ca-865cb79987-snhtr"
Apr 28 19:19:18.842658 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.842641    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/be0cfd01-5749-4bbc-9b15-76d63a71524b-signing-key\") pod \"service-ca-865cb79987-snhtr\" (UID: \"be0cfd01-5749-4bbc-9b15-76d63a71524b\") " pod="openshift-service-ca/service-ca-865cb79987-snhtr"
Apr 28 19:19:18.849448 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.849427    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxttx\" (UniqueName: \"kubernetes.io/projected/be0cfd01-5749-4bbc-9b15-76d63a71524b-kube-api-access-kxttx\") pod \"service-ca-865cb79987-snhtr\" (UID: \"be0cfd01-5749-4bbc-9b15-76d63a71524b\") " pod="openshift-service-ca/service-ca-865cb79987-snhtr"
Apr 28 19:19:18.996308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:18.996276    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-865cb79987-snhtr"
Apr 28 19:19:19.042014 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:19.041985    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:19.042141 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:19.042127    2569 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Apr 28 19:19:19.042141 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:19.042139    2569 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6789dcc546-g9gsl: secret "image-registry-tls" not found
Apr 28 19:19:19.042253 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:19.042208    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls podName:af47071b-d8de-49e7-b564-1ab0414ff0e9 nodeName:}" failed. No retries permitted until 2026-04-28 19:19:27.042193841 +0000 UTC m=+153.866533926 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls") pod "image-registry-6789dcc546-g9gsl" (UID: "af47071b-d8de-49e7-b564-1ab0414ff0e9") : secret "image-registry-tls" not found
Apr 28 19:19:19.111112 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:19.111086    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-865cb79987-snhtr"]
Apr 28 19:19:19.114041 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:19:19.114008    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe0cfd01_5749_4bbc_9b15_76d63a71524b.slice/crio-3c107b37cf2ed02815cedc53196557e46355d91c91505ed9654e8ae1f2b2fb76 WatchSource:0}: Error finding container 3c107b37cf2ed02815cedc53196557e46355d91c91505ed9654e8ae1f2b2fb76: Status 404 returned error can't find the container with id 3c107b37cf2ed02815cedc53196557e46355d91c91505ed9654e8ae1f2b2fb76
Apr 28 19:19:19.375534 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:19.375451    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-865cb79987-snhtr" event={"ID":"be0cfd01-5749-4bbc-9b15-76d63a71524b","Type":"ContainerStarted","Data":"3c107b37cf2ed02815cedc53196557e46355d91c91505ed9654e8ae1f2b2fb76"}
Apr 28 19:19:19.619704 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:19.619674    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tzfn9_8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad/node-ca/0.log"
Apr 28 19:19:21.014142 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:21.014109    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6769c5d45-x8wk8_abc28c3c-a5be-45cd-8095-f9d59dd936dc/kube-storage-version-migrator-operator/0.log"
Apr 28 19:19:21.382090 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:21.382008    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-865cb79987-snhtr" event={"ID":"be0cfd01-5749-4bbc-9b15-76d63a71524b","Type":"ContainerStarted","Data":"00d2cae7d7dbeb0f7c6f97a5cef57521e3e2504f12cf2e34559e3b85445c09ca"}
Apr 28 19:19:21.489425 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:21.489379    2569 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:21.489425 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:21.489429    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:21.489736 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:21.489725    2569 scope.go:117] "RemoveContainer" containerID="762d27b7a2ec660ce6b5995c6000b94c0592743745c2ca8e80f736a8a88a472f"
Apr 28 19:19:21.489886 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:21.489870    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-6svf6_openshift-console-operator(034e2b42-803b-45fd-a6df-4371bec965ff)\"" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6" podUID="034e2b42-803b-45fd-a6df-4371bec965ff"
Apr 28 19:19:27.109709 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:27.109673    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:27.112098 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:27.112075    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls\") pod \"image-registry-6789dcc546-g9gsl\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") " pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:27.189901 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:27.189875    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:27.325521 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:27.325473    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-865cb79987-snhtr" podStartSLOduration=7.770987217 podStartE2EDuration="9.325458545s" podCreationTimestamp="2026-04-28 19:19:18 +0000 UTC" firstStartedPulling="2026-04-28 19:19:19.115773977 +0000 UTC m=+145.940114063" lastFinishedPulling="2026-04-28 19:19:20.670245301 +0000 UTC m=+147.494585391" observedRunningTime="2026-04-28 19:19:21.447137558 +0000 UTC m=+148.271477665" watchObservedRunningTime="2026-04-28 19:19:27.325458545 +0000 UTC m=+154.149798654"
Apr 28 19:19:27.326023 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:27.325999    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6789dcc546-g9gsl"]
Apr 28 19:19:27.330266 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:19:27.330230    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf47071b_d8de_49e7_b564_1ab0414ff0e9.slice/crio-c5c9a47dde4fb431bf359d3ae29cc34c956e42b41fe536153fb7d03fe496c1df WatchSource:0}: Error finding container c5c9a47dde4fb431bf359d3ae29cc34c956e42b41fe536153fb7d03fe496c1df: Status 404 returned error can't find the container with id c5c9a47dde4fb431bf359d3ae29cc34c956e42b41fe536153fb7d03fe496c1df
Apr 28 19:19:27.398712 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:27.398688    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6789dcc546-g9gsl" event={"ID":"af47071b-d8de-49e7-b564-1ab0414ff0e9","Type":"ContainerStarted","Data":"d13b326655b342b4c5a3ef8e4c053c33977611a54bd7821ceb041de3c7c96f12"}
Apr 28 19:19:27.398820 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:27.398724    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6789dcc546-g9gsl" event={"ID":"af47071b-d8de-49e7-b564-1ab0414ff0e9","Type":"ContainerStarted","Data":"c5c9a47dde4fb431bf359d3ae29cc34c956e42b41fe536153fb7d03fe496c1df"}
Apr 28 19:19:27.398820 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:27.398810    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:27.423824 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:27.423779    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6789dcc546-g9gsl" podStartSLOduration=16.423762899 podStartE2EDuration="16.423762899s" podCreationTimestamp="2026-04-28 19:19:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:19:27.422938841 +0000 UTC m=+154.247278950" watchObservedRunningTime="2026-04-28 19:19:27.423762899 +0000 UTC m=+154.248103010"
Apr 28 19:19:29.059393 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:29.059349    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-lt6tx" podUID="35caf455-585a-430d-be29-50d6ad53fcd7"
Apr 28 19:19:29.074602 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:29.074576    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-g4hd4" podUID="f38d24be-b797-499f-8593-7b9ec113f103"
Apr 28 19:19:29.405436 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:29.405345    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:19:29.405593 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:29.405344    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:19:29.963090 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:29.963054    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-hgm2f" podUID="e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33"
Apr 28 19:19:33.961669 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:33.961638    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:19:33.962099 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:33.961687    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:19:33.963946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:33.963921    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35caf455-585a-430d-be29-50d6ad53fcd7-metrics-tls\") pod \"dns-default-lt6tx\" (UID: \"35caf455-585a-430d-be29-50d6ad53fcd7\") " pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:19:33.964044 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:33.963972    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f38d24be-b797-499f-8593-7b9ec113f103-cert\") pod \"ingress-canary-g4hd4\" (UID: \"f38d24be-b797-499f-8593-7b9ec113f103\") " pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:19:34.209575 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:34.209552    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jtx2j\""
Apr 28 19:19:34.209728 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:34.209552    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-524nx\""
Apr 28 19:19:34.216803 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:34.216723    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g4hd4"
Apr 28 19:19:34.216803 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:34.216762    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:19:34.380173 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:34.380145    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lt6tx"]
Apr 28 19:19:34.383598 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:19:34.383567    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35caf455_585a_430d_be29_50d6ad53fcd7.slice/crio-8c22c9615a270577f223351b8df8a01720519b77f1e47d1edfc271c8c17e1261 WatchSource:0}: Error finding container 8c22c9615a270577f223351b8df8a01720519b77f1e47d1edfc271c8c17e1261: Status 404 returned error can't find the container with id 8c22c9615a270577f223351b8df8a01720519b77f1e47d1edfc271c8c17e1261
Apr 28 19:19:34.384157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:34.384067    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g4hd4"]
Apr 28 19:19:34.387161 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:19:34.387125    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf38d24be_b797_499f_8593_7b9ec113f103.slice/crio-f74fc67ffcb1614bea42fc447e80691fe630ed85b6288015e60c3c3c409ca69b WatchSource:0}: Error finding container f74fc67ffcb1614bea42fc447e80691fe630ed85b6288015e60c3c3c409ca69b: Status 404 returned error can't find the container with id f74fc67ffcb1614bea42fc447e80691fe630ed85b6288015e60c3c3c409ca69b
Apr 28 19:19:34.417487 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:34.417456    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lt6tx" event={"ID":"35caf455-585a-430d-be29-50d6ad53fcd7","Type":"ContainerStarted","Data":"8c22c9615a270577f223351b8df8a01720519b77f1e47d1edfc271c8c17e1261"}
Apr 28 19:19:34.418252 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:34.418231    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g4hd4" event={"ID":"f38d24be-b797-499f-8593-7b9ec113f103","Type":"ContainerStarted","Data":"f74fc67ffcb1614bea42fc447e80691fe630ed85b6288015e60c3c3c409ca69b"}
Apr 28 19:19:36.425618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.425528    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lt6tx" event={"ID":"35caf455-585a-430d-be29-50d6ad53fcd7","Type":"ContainerStarted","Data":"9479b7553b45be17430cf314a90892c48be48b123d8f354e6994fb036d7559dd"}
Apr 28 19:19:36.425618 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.425573    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lt6tx" event={"ID":"35caf455-585a-430d-be29-50d6ad53fcd7","Type":"ContainerStarted","Data":"0b6be5d0c77e809577b9ffbac277236414afa29b76dc3a7473d20cc456fd3f49"}
Apr 28 19:19:36.426068 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.425747    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:19:36.426828 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.426805    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g4hd4" event={"ID":"f38d24be-b797-499f-8593-7b9ec113f103","Type":"ContainerStarted","Data":"3c4a7f4abec7dfeced775743ad5df42c78cb6155640328d2f8ee71f29f746e07"}
Apr 28 19:19:36.446971 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.446927    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-lt6tx" podStartSLOduration=129.709145134 podStartE2EDuration="2m11.446915918s" podCreationTimestamp="2026-04-28 19:17:25 +0000 UTC" firstStartedPulling="2026-04-28 19:19:34.385580803 +0000 UTC m=+161.209920904" lastFinishedPulling="2026-04-28 19:19:36.123351602 +0000 UTC m=+162.947691688" observedRunningTime="2026-04-28 19:19:36.446298183 +0000 UTC m=+163.270638290" watchObservedRunningTime="2026-04-28 19:19:36.446915918 +0000 UTC m=+163.271256027"
Apr 28 19:19:36.465213 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.465179    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g4hd4" podStartSLOduration=129.72849244 podStartE2EDuration="2m11.465171094s" podCreationTimestamp="2026-04-28 19:17:25 +0000 UTC" firstStartedPulling="2026-04-28 19:19:34.388917414 +0000 UTC m=+161.213257504" lastFinishedPulling="2026-04-28 19:19:36.125596056 +0000 UTC m=+162.949936158" observedRunningTime="2026-04-28 19:19:36.46425645 +0000 UTC m=+163.288596560" watchObservedRunningTime="2026-04-28 19:19:36.465171094 +0000 UTC m=+163.289511201"
Apr 28 19:19:36.866342 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.866070    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fjrcg"]
Apr 28 19:19:36.868338 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.868316    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6789dcc546-g9gsl"]
Apr 28 19:19:36.868496 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.868479    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:36.874694 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.874673    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Apr 28 19:19:36.876025 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.876002    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8vs7f\""
Apr 28 19:19:36.876130 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.876023    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Apr 28 19:19:36.928413 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.928384    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fjrcg"]
Apr 28 19:19:36.937729 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.937706    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-549b6c67db-42h9s"]
Apr 28 19:19:36.939601 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.939582    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:36.939695 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.939669    2569 scope.go:117] "RemoveContainer" containerID="762d27b7a2ec660ce6b5995c6000b94c0592743745c2ca8e80f736a8a88a472f"
Apr 28 19:19:36.989608 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.989585    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc141dec-64d4-4235-bae0-ec7e3e98edc4-installation-pull-secrets\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:36.989720 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.989644    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfk8z\" (UniqueName: \"kubernetes.io/projected/fc141dec-64d4-4235-bae0-ec7e3e98edc4-kube-api-access-kfk8z\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:36.989720 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.989678    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aeee4d80-c824-4f03-a757-515670c34252-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:36.989806 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.989732    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aeee4d80-c824-4f03-a757-515670c34252-crio-socket\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:36.989839 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.989818    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc141dec-64d4-4235-bae0-ec7e3e98edc4-registry-tls\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:36.989872 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.989858    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc141dec-64d4-4235-bae0-ec7e3e98edc4-bound-sa-token\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:36.989941 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.989913    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk6d8\" (UniqueName: \"kubernetes.io/projected/aeee4d80-c824-4f03-a757-515670c34252-kube-api-access-zk6d8\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:36.990340 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.990121    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aeee4d80-c824-4f03-a757-515670c34252-data-volume\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:36.990340 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.990329    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc141dec-64d4-4235-bae0-ec7e3e98edc4-ca-trust-extracted\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:36.990521 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.990365    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc141dec-64d4-4235-bae0-ec7e3e98edc4-trusted-ca\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:36.990521 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.990433    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc141dec-64d4-4235-bae0-ec7e3e98edc4-image-registry-private-configuration\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:36.990521 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.990510    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aeee4d80-c824-4f03-a757-515670c34252-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:36.991177 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:36.991151    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc141dec-64d4-4235-bae0-ec7e3e98edc4-registry-certificates\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.005022 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.004995    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-549b6c67db-42h9s"]
Apr 28 19:19:37.092503 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092461    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc141dec-64d4-4235-bae0-ec7e3e98edc4-installation-pull-secrets\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.092503 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092505    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kfk8z\" (UniqueName: \"kubernetes.io/projected/fc141dec-64d4-4235-bae0-ec7e3e98edc4-kube-api-access-kfk8z\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.092745 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092529    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aeee4d80-c824-4f03-a757-515670c34252-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:37.092745 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092559    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aeee4d80-c824-4f03-a757-515670c34252-crio-socket\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:37.092745 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092580    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc141dec-64d4-4235-bae0-ec7e3e98edc4-registry-tls\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.092745 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092596    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc141dec-64d4-4235-bae0-ec7e3e98edc4-bound-sa-token\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.092745 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092623    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6d8\" (UniqueName: \"kubernetes.io/projected/aeee4d80-c824-4f03-a757-515670c34252-kube-api-access-zk6d8\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:37.092745 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092657    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aeee4d80-c824-4f03-a757-515670c34252-data-volume\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:37.092745 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092680    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc141dec-64d4-4235-bae0-ec7e3e98edc4-ca-trust-extracted\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.092745 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092683    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aeee4d80-c824-4f03-a757-515670c34252-crio-socket\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:37.092745 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092707    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc141dec-64d4-4235-bae0-ec7e3e98edc4-trusted-ca\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.092745 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092742    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc141dec-64d4-4235-bae0-ec7e3e98edc4-image-registry-private-configuration\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.093218 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092771    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aeee4d80-c824-4f03-a757-515670c34252-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:37.093218 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.092816    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc141dec-64d4-4235-bae0-ec7e3e98edc4-registry-certificates\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.093218 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.093003    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aeee4d80-c824-4f03-a757-515670c34252-data-volume\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:37.093366 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.093298    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc141dec-64d4-4235-bae0-ec7e3e98edc4-ca-trust-extracted\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.093773 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.093748    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc141dec-64d4-4235-bae0-ec7e3e98edc4-registry-certificates\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.093874 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.093808    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aeee4d80-c824-4f03-a757-515670c34252-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:37.093874 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.093845    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc141dec-64d4-4235-bae0-ec7e3e98edc4-trusted-ca\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.095195 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.095164    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc141dec-64d4-4235-bae0-ec7e3e98edc4-installation-pull-secrets\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.095347 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.095332    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc141dec-64d4-4235-bae0-ec7e3e98edc4-registry-tls\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.095522 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.095504    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc141dec-64d4-4235-bae0-ec7e3e98edc4-image-registry-private-configuration\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.095913 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.095897    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aeee4d80-c824-4f03-a757-515670c34252-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:37.107690 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.107671    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfk8z\" (UniqueName: \"kubernetes.io/projected/fc141dec-64d4-4235-bae0-ec7e3e98edc4-kube-api-access-kfk8z\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.109628 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.109610    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc141dec-64d4-4235-bae0-ec7e3e98edc4-bound-sa-token\") pod \"image-registry-549b6c67db-42h9s\" (UID: \"fc141dec-64d4-4235-bae0-ec7e3e98edc4\") " pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.112563 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.112547    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk6d8\" (UniqueName: \"kubernetes.io/projected/aeee4d80-c824-4f03-a757-515670c34252-kube-api-access-zk6d8\") pod \"insights-runtime-extractor-fjrcg\" (UID: \"aeee4d80-c824-4f03-a757-515670c34252\") " pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:37.177855 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.177807    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fjrcg"
Apr 28 19:19:37.250437 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.249990    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:37.367364 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.367331    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fjrcg"]
Apr 28 19:19:37.372217 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:19:37.372190    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeee4d80_c824_4f03_a757_515670c34252.slice/crio-c9714cbe95e78782a6069a7fee870863ba6ae0a1aa459fa58e36890905a624f9 WatchSource:0}: Error finding container c9714cbe95e78782a6069a7fee870863ba6ae0a1aa459fa58e36890905a624f9: Status 404 returned error can't find the container with id c9714cbe95e78782a6069a7fee870863ba6ae0a1aa459fa58e36890905a624f9
Apr 28 19:19:37.421100 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.421076    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-549b6c67db-42h9s"]
Apr 28 19:19:37.423768 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:19:37.423740    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc141dec_64d4_4235_bae0_ec7e3e98edc4.slice/crio-a10a5035852c7b045fa0bbf863af08f437a21613f83b41a36b3382654ad84bfc WatchSource:0}: Error finding container a10a5035852c7b045fa0bbf863af08f437a21613f83b41a36b3382654ad84bfc: Status 404 returned error can't find the container with id a10a5035852c7b045fa0bbf863af08f437a21613f83b41a36b3382654ad84bfc
Apr 28 19:19:37.430843 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.430816    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjrcg" event={"ID":"aeee4d80-c824-4f03-a757-515670c34252","Type":"ContainerStarted","Data":"bab6140d25bcacdc4013ae3086376de1aaf2e9a03edcc369c100f42a8e0ad404"}
Apr 28 19:19:37.431185 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.430852    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjrcg" event={"ID":"aeee4d80-c824-4f03-a757-515670c34252","Type":"ContainerStarted","Data":"c9714cbe95e78782a6069a7fee870863ba6ae0a1aa459fa58e36890905a624f9"}
Apr 28 19:19:37.432778 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.432760    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:19:37.432880 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.432834    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6" event={"ID":"034e2b42-803b-45fd-a6df-4371bec965ff","Type":"ContainerStarted","Data":"43bc18387f34ce6e73482e7fc7971e02e48f61a3baf1ab87f99a98928b8f2018"}
Apr 28 19:19:37.433117 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.433088    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:37.434546 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.434503    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-549b6c67db-42h9s" event={"ID":"fc141dec-64d4-4235-bae0-ec7e3e98edc4","Type":"ContainerStarted","Data":"a10a5035852c7b045fa0bbf863af08f437a21613f83b41a36b3382654ad84bfc"}
Apr 28 19:19:37.470849 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.470808    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6" podStartSLOduration=23.70131873 podStartE2EDuration="26.470793268s" podCreationTimestamp="2026-04-28 19:19:11 +0000 UTC" firstStartedPulling="2026-04-28 19:19:11.624329998 +0000 UTC m=+138.448670084" lastFinishedPulling="2026-04-28 19:19:14.393804536 +0000 UTC m=+141.218144622" observedRunningTime="2026-04-28 19:19:37.470644055 +0000 UTC m=+164.294984163" watchObservedRunningTime="2026-04-28 19:19:37.470793268 +0000 UTC m=+164.295133373"
Apr 28 19:19:37.874380 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:37.874353    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-9d4b6777b-6svf6"
Apr 28 19:19:38.438467 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:38.438425    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-549b6c67db-42h9s" event={"ID":"fc141dec-64d4-4235-bae0-ec7e3e98edc4","Type":"ContainerStarted","Data":"ef93c67e2a576338a366b5a5c2a9b411c2c4214d2bbcc075c89f3cbe8da00ba5"}
Apr 28 19:19:38.438939 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:38.438585    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:38.440105 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:38.440081    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjrcg" event={"ID":"aeee4d80-c824-4f03-a757-515670c34252","Type":"ContainerStarted","Data":"7bcd814ccb6963a1f44811ce0132c74ccb4ece9743313e3846afd59ebdce8a61"}
Apr 28 19:19:38.501112 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:38.501059    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-549b6c67db-42h9s" podStartSLOduration=2.501044406 podStartE2EDuration="2.501044406s" podCreationTimestamp="2026-04-28 19:19:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:19:38.499696268 +0000 UTC m=+165.324036378" watchObservedRunningTime="2026-04-28 19:19:38.501044406 +0000 UTC m=+165.325384513"
Apr 28 19:19:39.443857 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:39.443826    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjrcg" event={"ID":"aeee4d80-c824-4f03-a757-515670c34252","Type":"ContainerStarted","Data":"6290736bbba38c39febd3eb884a9850395cb76c80c64920e6c6e8114d434752f"}
Apr 28 19:19:39.468120 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:39.468078    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fjrcg" podStartSLOduration=1.57675999 podStartE2EDuration="3.468064471s" podCreationTimestamp="2026-04-28 19:19:36 +0000 UTC" firstStartedPulling="2026-04-28 19:19:37.4269655 +0000 UTC m=+164.251305586" lastFinishedPulling="2026-04-28 19:19:39.318269981 +0000 UTC m=+166.142610067" observedRunningTime="2026-04-28 19:19:39.466735912 +0000 UTC m=+166.291076042" watchObservedRunningTime="2026-04-28 19:19:39.468064471 +0000 UTC m=+166.292404578"
Apr 28 19:19:42.938068 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:42.937976    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:19:46.318175 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:46.318142    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp"]
Apr 28 19:19:46.373062 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:46.373035    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp"]
Apr 28 19:19:46.373206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:46.373145    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp"
Apr 28 19:19:46.376108 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:46.376088    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Apr 28 19:19:46.376108 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:46.376102    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-qgz56\""
Apr 28 19:19:46.436842 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:46.436819    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lt6tx"
Apr 28 19:19:46.461768 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:46.461739    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ba388df3-0277-4c59-8dda-8c29928a0112-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-jrfsp\" (UID: \"ba388df3-0277-4c59-8dda-8c29928a0112\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp"
Apr 28 19:19:46.562566 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:46.562537    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ba388df3-0277-4c59-8dda-8c29928a0112-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-jrfsp\" (UID: \"ba388df3-0277-4c59-8dda-8c29928a0112\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp"
Apr 28 19:19:46.562730 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:46.562714    2569 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Apr 28 19:19:46.562823 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:19:46.562810    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba388df3-0277-4c59-8dda-8c29928a0112-tls-certificates podName:ba388df3-0277-4c59-8dda-8c29928a0112 nodeName:}" failed. No retries permitted until 2026-04-28 19:19:47.062781392 +0000 UTC m=+173.887121484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/ba388df3-0277-4c59-8dda-8c29928a0112-tls-certificates") pod "prometheus-operator-admission-webhook-57cf98b594-jrfsp" (UID: "ba388df3-0277-4c59-8dda-8c29928a0112") : secret "prometheus-operator-admission-webhook-tls" not found
Apr 28 19:19:46.873911 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:46.873878    2569 patch_prober.go:28] interesting pod/image-registry-6789dcc546-g9gsl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Apr 28 19:19:46.874066 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:46.873939    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6789dcc546-g9gsl" podUID="af47071b-d8de-49e7-b564-1ab0414ff0e9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Apr 28 19:19:47.065701 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:47.065673    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ba388df3-0277-4c59-8dda-8c29928a0112-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-jrfsp\" (UID: \"ba388df3-0277-4c59-8dda-8c29928a0112\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp"
Apr 28 19:19:47.068033 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:47.068015    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ba388df3-0277-4c59-8dda-8c29928a0112-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-jrfsp\" (UID: \"ba388df3-0277-4c59-8dda-8c29928a0112\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp"
Apr 28 19:19:47.281550 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:47.281520    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp"
Apr 28 19:19:47.408243 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:47.408206    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp"]
Apr 28 19:19:47.412671 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:19:47.412643    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba388df3_0277_4c59_8dda_8c29928a0112.slice/crio-bd682cfc31c4ebb9b72b7afa054a603e919bbf2a2195cb916fd1962a0693b3b1 WatchSource:0}: Error finding container bd682cfc31c4ebb9b72b7afa054a603e919bbf2a2195cb916fd1962a0693b3b1: Status 404 returned error can't find the container with id bd682cfc31c4ebb9b72b7afa054a603e919bbf2a2195cb916fd1962a0693b3b1
Apr 28 19:19:47.464102 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:47.464070    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp" event={"ID":"ba388df3-0277-4c59-8dda-8c29928a0112","Type":"ContainerStarted","Data":"bd682cfc31c4ebb9b72b7afa054a603e919bbf2a2195cb916fd1962a0693b3b1"}
Apr 28 19:19:48.467800 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:48.467771    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp" event={"ID":"ba388df3-0277-4c59-8dda-8c29928a0112","Type":"ContainerStarted","Data":"37a2dbd3a207e745e5fdb5e07acee5d72968ee9b1e900c1e870f723c791d6256"}
Apr 28 19:19:48.468095 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:48.467982    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp"
Apr 28 19:19:48.472465 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:48.472446    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp"
Apr 28 19:19:48.485946 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:48.485906    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jrfsp" podStartSLOduration=1.552194557 podStartE2EDuration="2.485894188s" podCreationTimestamp="2026-04-28 19:19:46 +0000 UTC" firstStartedPulling="2026-04-28 19:19:47.414477471 +0000 UTC m=+174.238817561" lastFinishedPulling="2026-04-28 19:19:48.348177106 +0000 UTC m=+175.172517192" observedRunningTime="2026-04-28 19:19:48.484854837 +0000 UTC m=+175.309194945" watchObservedRunningTime="2026-04-28 19:19:48.485894188 +0000 UTC m=+175.310234295"
Apr 28 19:19:49.551046 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.551019    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f6cb8675f-ccr8c"]
Apr 28 19:19:49.552885 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.552868    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.563696 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.563675    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Apr 28 19:19:49.563807 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.563766    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Apr 28 19:19:49.565117 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.565100    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Apr 28 19:19:49.565209 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.565144    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Apr 28 19:19:49.565209 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.565155    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Apr 28 19:19:49.565478 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.565462    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5nxrd\""
Apr 28 19:19:49.565557 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.565500    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Apr 28 19:19:49.566348 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.566333    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Apr 28 19:19:49.575034 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.573786    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Apr 28 19:19:49.576796 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.576778    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f6cb8675f-ccr8c"]
Apr 28 19:19:49.685854 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.685820    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-oauth-config\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.685854 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.685855    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-config\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.686012 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.685890    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbz52\" (UniqueName: \"kubernetes.io/projected/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-kube-api-access-gbz52\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.686012 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.685906    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-service-ca\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.686012 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.685982    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-trusted-ca-bundle\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.686110 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.686036    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-oauth-serving-cert\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.686110 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.686065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-serving-cert\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.787377 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.787329    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gbz52\" (UniqueName: \"kubernetes.io/projected/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-kube-api-access-gbz52\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.787377 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.787380    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-service-ca\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.787587 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.787459    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-trusted-ca-bundle\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.787587 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.787490    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-oauth-serving-cert\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.787587 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.787514    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-serving-cert\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.787587 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.787541    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-oauth-config\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.787759 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.787663    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-config\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.788192 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.788166    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-service-ca\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.788293 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.788226    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-oauth-serving-cert\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.788293 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.788265    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-config\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.789015 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.788996    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-trusted-ca-bundle\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.789963 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.789944    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-serving-cert\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.789963 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.789958    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-oauth-config\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.796529 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.796508    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbz52\" (UniqueName: \"kubernetes.io/projected/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-kube-api-access-gbz52\") pod \"console-5f6cb8675f-ccr8c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") " pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.861058 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.860996    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:49.978804 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:49.978724    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f6cb8675f-ccr8c"]
Apr 28 19:19:49.982468 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:19:49.982436    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a216f6e_4215_47a2_bfa3_2e0e5868e25c.slice/crio-070a3c34c7ed144ac6e317d1b8f43008d88f51514f5295162696d40eb910c569 WatchSource:0}: Error finding container 070a3c34c7ed144ac6e317d1b8f43008d88f51514f5295162696d40eb910c569: Status 404 returned error can't find the container with id 070a3c34c7ed144ac6e317d1b8f43008d88f51514f5295162696d40eb910c569
Apr 28 19:19:50.474558 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:50.474517    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f6cb8675f-ccr8c" event={"ID":"0a216f6e-4215-47a2-bfa3-2e0e5868e25c","Type":"ContainerStarted","Data":"070a3c34c7ed144ac6e317d1b8f43008d88f51514f5295162696d40eb910c569"}
Apr 28 19:19:52.481772 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:52.481734    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f6cb8675f-ccr8c" event={"ID":"0a216f6e-4215-47a2-bfa3-2e0e5868e25c","Type":"ContainerStarted","Data":"294316338f1126226410d9d379b6460c9b5a6852a65ad904722bcae387188eb0"}
Apr 28 19:19:52.514382 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:52.514334    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5f6cb8675f-ccr8c" podStartSLOduration=1.102055993 podStartE2EDuration="3.514318354s" podCreationTimestamp="2026-04-28 19:19:49 +0000 UTC" firstStartedPulling="2026-04-28 19:19:49.984474961 +0000 UTC m=+176.808815048" lastFinishedPulling="2026-04-28 19:19:52.39673731 +0000 UTC m=+179.221077409" observedRunningTime="2026-04-28 19:19:52.513531656 +0000 UTC m=+179.337871777" watchObservedRunningTime="2026-04-28 19:19:52.514318354 +0000 UTC m=+179.338658459"
Apr 28 19:19:56.872156 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:56.872128    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:19:59.447442 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:59.447395    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-549b6c67db-42h9s"
Apr 28 19:19:59.861978 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:59.861898    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:59.862228 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:59.862187    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:19:59.866855 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:19:59.866836    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:20:00.507724 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:00.507699    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:20:01.798016 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.797980    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"]
Apr 28 19:20:01.801147 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.801118    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:01.804296 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.804266    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Apr 28 19:20:01.804977 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.804955    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Apr 28 19:20:01.805089 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.804961    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zx9h2\""
Apr 28 19:20:01.805415 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.805380    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Apr 28 19:20:01.805526 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.805418    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Apr 28 19:20:01.805526 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.805424    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Apr 28 19:20:01.806775 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.806757    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8d5rz"]
Apr 28 19:20:01.808729 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.808701    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:01.811290 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.811271    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Apr 28 19:20:01.811373 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.811309    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Apr 28 19:20:01.811669 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.811650    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Apr 28 19:20:01.811871 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.811855    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-srbh6\""
Apr 28 19:20:01.822860 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.822840    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"]
Apr 28 19:20:01.880098 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.880068    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e7e390c-5479-46d7-af9f-3bcdc26f7a64-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9d44df66c-ljn9c\" (UID: \"6e7e390c-5479-46d7-af9f-3bcdc26f7a64\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:01.880321 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.880131    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7e390c-5479-46d7-af9f-3bcdc26f7a64-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9d44df66c-ljn9c\" (UID: \"6e7e390c-5479-46d7-af9f-3bcdc26f7a64\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:01.880321 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.880251    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb6zz\" (UniqueName: \"kubernetes.io/projected/6e7e390c-5479-46d7-af9f-3bcdc26f7a64-kube-api-access-nb6zz\") pod \"openshift-state-metrics-9d44df66c-ljn9c\" (UID: \"6e7e390c-5479-46d7-af9f-3bcdc26f7a64\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:01.880321 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.880287    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e7e390c-5479-46d7-af9f-3bcdc26f7a64-metrics-client-ca\") pod \"openshift-state-metrics-9d44df66c-ljn9c\" (UID: \"6e7e390c-5479-46d7-af9f-3bcdc26f7a64\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:01.887506 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.887464    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6789dcc546-g9gsl" podUID="af47071b-d8de-49e7-b564-1ab0414ff0e9" containerName="registry" containerID="cri-o://d13b326655b342b4c5a3ef8e4c053c33977611a54bd7821ceb041de3c7c96f12" gracePeriod=30
Apr 28 19:20:01.981294 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.981261    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6205f07-25d7-4a09-8c48-c01358a1e07c-metrics-client-ca\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:01.981495 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.981324    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:01.981495 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.981365    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-textfile\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:01.981495 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.981417    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nb6zz\" (UniqueName: \"kubernetes.io/projected/6e7e390c-5479-46d7-af9f-3bcdc26f7a64-kube-api-access-nb6zz\") pod \"openshift-state-metrics-9d44df66c-ljn9c\" (UID: \"6e7e390c-5479-46d7-af9f-3bcdc26f7a64\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:01.981495 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.981447    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e7e390c-5479-46d7-af9f-3bcdc26f7a64-metrics-client-ca\") pod \"openshift-state-metrics-9d44df66c-ljn9c\" (UID: \"6e7e390c-5479-46d7-af9f-3bcdc26f7a64\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:01.981495 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.981479    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f6205f07-25d7-4a09-8c48-c01358a1e07c-root\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:01.981757 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.981520    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e7e390c-5479-46d7-af9f-3bcdc26f7a64-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9d44df66c-ljn9c\" (UID: \"6e7e390c-5479-46d7-af9f-3bcdc26f7a64\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:01.981757 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.981592    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-tls\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:01.981757 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.981636    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-wtmp\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:01.981757 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.981666    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6205f07-25d7-4a09-8c48-c01358a1e07c-sys\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:01.981757 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.981692    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9fpc\" (UniqueName: \"kubernetes.io/projected/f6205f07-25d7-4a09-8c48-c01358a1e07c-kube-api-access-t9fpc\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:01.981757 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.981745    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7e390c-5479-46d7-af9f-3bcdc26f7a64-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9d44df66c-ljn9c\" (UID: \"6e7e390c-5479-46d7-af9f-3bcdc26f7a64\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:01.982043 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.981782    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-accelerators-collector-config\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:01.982462 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.982433    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e7e390c-5479-46d7-af9f-3bcdc26f7a64-metrics-client-ca\") pod \"openshift-state-metrics-9d44df66c-ljn9c\" (UID: \"6e7e390c-5479-46d7-af9f-3bcdc26f7a64\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:01.984339 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.984312    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e7e390c-5479-46d7-af9f-3bcdc26f7a64-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9d44df66c-ljn9c\" (UID: \"6e7e390c-5479-46d7-af9f-3bcdc26f7a64\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:01.984478 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.984348    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7e390c-5479-46d7-af9f-3bcdc26f7a64-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9d44df66c-ljn9c\" (UID: \"6e7e390c-5479-46d7-af9f-3bcdc26f7a64\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:01.999105 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:01.999055    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb6zz\" (UniqueName: \"kubernetes.io/projected/6e7e390c-5479-46d7-af9f-3bcdc26f7a64-kube-api-access-nb6zz\") pod \"openshift-state-metrics-9d44df66c-ljn9c\" (UID: \"6e7e390c-5479-46d7-af9f-3bcdc26f7a64\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:02.083076 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.083043    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-tls\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.083222 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.083095    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-wtmp\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.083222 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.083126    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6205f07-25d7-4a09-8c48-c01358a1e07c-sys\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.083222 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.083149    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t9fpc\" (UniqueName: \"kubernetes.io/projected/f6205f07-25d7-4a09-8c48-c01358a1e07c-kube-api-access-t9fpc\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.083222 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:20:02.083188    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Apr 28 19:20:02.083426 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.083245    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6205f07-25d7-4a09-8c48-c01358a1e07c-sys\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.083426 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:20:02.083274    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-tls podName:f6205f07-25d7-4a09-8c48-c01358a1e07c nodeName:}" failed. No retries permitted until 2026-04-28 19:20:02.583243971 +0000 UTC m=+189.407584073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-tls") pod "node-exporter-8d5rz" (UID: "f6205f07-25d7-4a09-8c48-c01358a1e07c") : secret "node-exporter-tls" not found
Apr 28 19:20:02.083426 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.083302    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-wtmp\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.083426 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.083366    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-accelerators-collector-config\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.083676 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.083429    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6205f07-25d7-4a09-8c48-c01358a1e07c-metrics-client-ca\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.083756 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.083723    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.083843 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.083759    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-textfile\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.083843 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.083816    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f6205f07-25d7-4a09-8c48-c01358a1e07c-root\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.083950 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.083921    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f6205f07-25d7-4a09-8c48-c01358a1e07c-root\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.084072 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.084045    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-accelerators-collector-config\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.084150 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.084076    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6205f07-25d7-4a09-8c48-c01358a1e07c-metrics-client-ca\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.084205 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.084181    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-textfile\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.086215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.086191    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.099721 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.099693    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9fpc\" (UniqueName: \"kubernetes.io/projected/f6205f07-25d7-4a09-8c48-c01358a1e07c-kube-api-access-t9fpc\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.112634 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.112609    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"
Apr 28 19:20:02.125843 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.125808    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:20:02.254954 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.254922    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c"]
Apr 28 19:20:02.258962 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:20:02.258927    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e7e390c_5479_46d7_af9f_3bcdc26f7a64.slice/crio-2551d31a652145de55ab955574cf86a75b6af5b83495b03277926c38365421d4 WatchSource:0}: Error finding container 2551d31a652145de55ab955574cf86a75b6af5b83495b03277926c38365421d4: Status 404 returned error can't find the container with id 2551d31a652145de55ab955574cf86a75b6af5b83495b03277926c38365421d4
Apr 28 19:20:02.285195 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.285166    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af47071b-d8de-49e7-b564-1ab0414ff0e9-trusted-ca\") pod \"af47071b-d8de-49e7-b564-1ab0414ff0e9\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") "
Apr 28 19:20:02.285317 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.285205    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6tks\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-kube-api-access-c6tks\") pod \"af47071b-d8de-49e7-b564-1ab0414ff0e9\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") "
Apr 28 19:20:02.285317 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.285231    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-certificates\") pod \"af47071b-d8de-49e7-b564-1ab0414ff0e9\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") "
Apr 28 19:20:02.285317 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.285254    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-bound-sa-token\") pod \"af47071b-d8de-49e7-b564-1ab0414ff0e9\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") "
Apr 28 19:20:02.285317 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.285304    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls\") pod \"af47071b-d8de-49e7-b564-1ab0414ff0e9\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") "
Apr 28 19:20:02.285548 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.285346    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/af47071b-d8de-49e7-b564-1ab0414ff0e9-ca-trust-extracted\") pod \"af47071b-d8de-49e7-b564-1ab0414ff0e9\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") "
Apr 28 19:20:02.285548 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.285373    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/af47071b-d8de-49e7-b564-1ab0414ff0e9-image-registry-private-configuration\") pod \"af47071b-d8de-49e7-b564-1ab0414ff0e9\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") "
Apr 28 19:20:02.285548 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.285437    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/af47071b-d8de-49e7-b564-1ab0414ff0e9-installation-pull-secrets\") pod \"af47071b-d8de-49e7-b564-1ab0414ff0e9\" (UID: \"af47071b-d8de-49e7-b564-1ab0414ff0e9\") "
Apr 28 19:20:02.285699 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.285628    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af47071b-d8de-49e7-b564-1ab0414ff0e9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "af47071b-d8de-49e7-b564-1ab0414ff0e9" (UID: "af47071b-d8de-49e7-b564-1ab0414ff0e9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:20:02.285699 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.285676    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "af47071b-d8de-49e7-b564-1ab0414ff0e9" (UID: "af47071b-d8de-49e7-b564-1ab0414ff0e9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:20:02.285800 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.285768    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af47071b-d8de-49e7-b564-1ab0414ff0e9-trusted-ca\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:02.285800 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.285787    2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-certificates\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:02.288077 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.288033    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-kube-api-access-c6tks" (OuterVolumeSpecName: "kube-api-access-c6tks") pod "af47071b-d8de-49e7-b564-1ab0414ff0e9" (UID: "af47071b-d8de-49e7-b564-1ab0414ff0e9"). InnerVolumeSpecName "kube-api-access-c6tks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:20:02.288077 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.288054    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af47071b-d8de-49e7-b564-1ab0414ff0e9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "af47071b-d8de-49e7-b564-1ab0414ff0e9" (UID: "af47071b-d8de-49e7-b564-1ab0414ff0e9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:20:02.288233 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.288068    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "af47071b-d8de-49e7-b564-1ab0414ff0e9" (UID: "af47071b-d8de-49e7-b564-1ab0414ff0e9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:20:02.288233 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.288096    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af47071b-d8de-49e7-b564-1ab0414ff0e9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "af47071b-d8de-49e7-b564-1ab0414ff0e9" (UID: "af47071b-d8de-49e7-b564-1ab0414ff0e9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:20:02.288233 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.288103    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "af47071b-d8de-49e7-b564-1ab0414ff0e9" (UID: "af47071b-d8de-49e7-b564-1ab0414ff0e9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:20:02.295458 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.295433    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af47071b-d8de-49e7-b564-1ab0414ff0e9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "af47071b-d8de-49e7-b564-1ab0414ff0e9" (UID: "af47071b-d8de-49e7-b564-1ab0414ff0e9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:20:02.387080 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.387008    2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/af47071b-d8de-49e7-b564-1ab0414ff0e9-installation-pull-secrets\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:02.387080 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.387031    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6tks\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-kube-api-access-c6tks\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:02.387080 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.387041    2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-bound-sa-token\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:02.387080 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.387050    2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af47071b-d8de-49e7-b564-1ab0414ff0e9-registry-tls\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:02.387080 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.387058    2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/af47071b-d8de-49e7-b564-1ab0414ff0e9-ca-trust-extracted\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:02.387080 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.387068    2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/af47071b-d8de-49e7-b564-1ab0414ff0e9-image-registry-private-configuration\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:02.510497 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.510466    2569 generic.go:358] "Generic (PLEG): container finished" podID="af47071b-d8de-49e7-b564-1ab0414ff0e9" containerID="d13b326655b342b4c5a3ef8e4c053c33977611a54bd7821ceb041de3c7c96f12" exitCode=0
Apr 28 19:20:02.510673 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.510542    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6789dcc546-g9gsl" event={"ID":"af47071b-d8de-49e7-b564-1ab0414ff0e9","Type":"ContainerDied","Data":"d13b326655b342b4c5a3ef8e4c053c33977611a54bd7821ceb041de3c7c96f12"}
Apr 28 19:20:02.510673 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.510553    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6789dcc546-g9gsl"
Apr 28 19:20:02.510673 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.510578    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6789dcc546-g9gsl" event={"ID":"af47071b-d8de-49e7-b564-1ab0414ff0e9","Type":"ContainerDied","Data":"c5c9a47dde4fb431bf359d3ae29cc34c956e42b41fe536153fb7d03fe496c1df"}
Apr 28 19:20:02.510673 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.510599    2569 scope.go:117] "RemoveContainer" containerID="d13b326655b342b4c5a3ef8e4c053c33977611a54bd7821ceb041de3c7c96f12"
Apr 28 19:20:02.517990 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.517955    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c" event={"ID":"6e7e390c-5479-46d7-af9f-3bcdc26f7a64","Type":"ContainerStarted","Data":"50ab414cb4332d37093007fdc15defc62ccd9343bd50dac5630932a2227cbb4b"}
Apr 28 19:20:02.518163 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.518147    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c" event={"ID":"6e7e390c-5479-46d7-af9f-3bcdc26f7a64","Type":"ContainerStarted","Data":"a1aecd6fbb1ed28b790d7797670156c69843d252746ac508aaf348a4f59d1991"}
Apr 28 19:20:02.518251 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.518240    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c" event={"ID":"6e7e390c-5479-46d7-af9f-3bcdc26f7a64","Type":"ContainerStarted","Data":"2551d31a652145de55ab955574cf86a75b6af5b83495b03277926c38365421d4"}
Apr 28 19:20:02.520817 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.520796    2569 scope.go:117] "RemoveContainer" containerID="d13b326655b342b4c5a3ef8e4c053c33977611a54bd7821ceb041de3c7c96f12"
Apr 28 19:20:02.521067 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:20:02.521041    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d13b326655b342b4c5a3ef8e4c053c33977611a54bd7821ceb041de3c7c96f12\": container with ID starting with d13b326655b342b4c5a3ef8e4c053c33977611a54bd7821ceb041de3c7c96f12 not found: ID does not exist" containerID="d13b326655b342b4c5a3ef8e4c053c33977611a54bd7821ceb041de3c7c96f12"
Apr 28 19:20:02.521128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.521078    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d13b326655b342b4c5a3ef8e4c053c33977611a54bd7821ceb041de3c7c96f12"} err="failed to get container status \"d13b326655b342b4c5a3ef8e4c053c33977611a54bd7821ceb041de3c7c96f12\": rpc error: code = NotFound desc = could not find container \"d13b326655b342b4c5a3ef8e4c053c33977611a54bd7821ceb041de3c7c96f12\": container with ID starting with d13b326655b342b4c5a3ef8e4c053c33977611a54bd7821ceb041de3c7c96f12 not found: ID does not exist"
Apr 28 19:20:02.552314 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.552288    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6789dcc546-g9gsl"]
Apr 28 19:20:02.557619 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.557595    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6789dcc546-g9gsl"]
Apr 28 19:20:02.589534 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.589472    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-tls\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.591652 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.591627    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f6205f07-25d7-4a09-8c48-c01358a1e07c-node-exporter-tls\") pod \"node-exporter-8d5rz\" (UID: \"f6205f07-25d7-4a09-8c48-c01358a1e07c\") " pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.721137 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:02.721104    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8d5rz"
Apr 28 19:20:02.729498 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:20:02.729473    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6205f07_25d7_4a09_8c48_c01358a1e07c.slice/crio-55a6e1a3ab562cf154465a8db24ab5896007dfb639995e0f932e301134819992 WatchSource:0}: Error finding container 55a6e1a3ab562cf154465a8db24ab5896007dfb639995e0f932e301134819992: Status 404 returned error can't find the container with id 55a6e1a3ab562cf154465a8db24ab5896007dfb639995e0f932e301134819992
Apr 28 19:20:03.119612 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.119579    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Apr 28 19:20:03.120048 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.119918    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af47071b-d8de-49e7-b564-1ab0414ff0e9" containerName="registry"
Apr 28 19:20:03.120048 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.119945    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="af47071b-d8de-49e7-b564-1ab0414ff0e9" containerName="registry"
Apr 28 19:20:03.120048 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.119988    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="af47071b-d8de-49e7-b564-1ab0414ff0e9" containerName="registry"
Apr 28 19:20:03.122296 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.122270    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.127713 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.127689    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Apr 28 19:20:03.128033 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.128011    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Apr 28 19:20:03.128364 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.128331    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Apr 28 19:20:03.128944 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.128706    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4btj6\""
Apr 28 19:20:03.128944 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.128819    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Apr 28 19:20:03.129624 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.129541    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Apr 28 19:20:03.129724 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.129544    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Apr 28 19:20:03.129787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.129753    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Apr 28 19:20:03.129842 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.129549    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Apr 28 19:20:03.130020 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.129928    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Apr 28 19:20:03.141931 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.141911    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Apr 28 19:20:03.194545 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.194514    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.194719 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.194561    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.194719 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.194587    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0195c259-8934-4087-8e99-353b607f030e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.194719 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.194660    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-242gf\" (UniqueName: \"kubernetes.io/projected/0195c259-8934-4087-8e99-353b607f030e-kube-api-access-242gf\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.194719 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.194703    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-config-volume\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.194921 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.194724    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0195c259-8934-4087-8e99-353b607f030e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.194921 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.194754    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0195c259-8934-4087-8e99-353b607f030e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.194921 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.194786    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0195c259-8934-4087-8e99-353b607f030e-config-out\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.194921 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.194818    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.194921 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.194844    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.194921 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.194887    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0195c259-8934-4087-8e99-353b607f030e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.195148 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.194929    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-web-config\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.195148 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.194985    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.296197 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.296154    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.296389 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.296228    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.296389 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.296262    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.296389 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.296288    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0195c259-8934-4087-8e99-353b607f030e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.296389 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.296319    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-242gf\" (UniqueName: \"kubernetes.io/projected/0195c259-8934-4087-8e99-353b607f030e-kube-api-access-242gf\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.296389 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.296362    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-config-volume\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.296389 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.296389    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0195c259-8934-4087-8e99-353b607f030e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.296727 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.296433    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0195c259-8934-4087-8e99-353b607f030e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.296727 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.296457    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0195c259-8934-4087-8e99-353b607f030e-config-out\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.296727 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.296481    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.296727 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.296513    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.296727 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.296563    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0195c259-8934-4087-8e99-353b607f030e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.296727 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.296598    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-web-config\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.297449 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:20:03.297420    2569 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Apr 28 19:20:03.297560 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:20:03.297499    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-main-tls podName:0195c259-8934-4087-8e99-353b607f030e nodeName:}" failed. No retries permitted until 2026-04-28 19:20:03.79747739 +0000 UTC m=+190.621817492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "0195c259-8934-4087-8e99-353b607f030e") : secret "alertmanager-main-tls" not found
Apr 28 19:20:03.299443 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.298266    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0195c259-8934-4087-8e99-353b607f030e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.299443 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.298382    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0195c259-8934-4087-8e99-353b607f030e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.299443 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.299386    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0195c259-8934-4087-8e99-353b607f030e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.300807 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.299996    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.300807 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.300392    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.300807 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.300767    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.301118 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.301079    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-web-config\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.301305 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.301269    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-config-volume\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.301651 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.301604    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.301974 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.301953    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0195c259-8934-4087-8e99-353b607f030e-config-out\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.303730 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.303707    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0195c259-8934-4087-8e99-353b607f030e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.304811 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.304793    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-242gf\" (UniqueName: \"kubernetes.io/projected/0195c259-8934-4087-8e99-353b607f030e-kube-api-access-242gf\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.524180 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.524139    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c" event={"ID":"6e7e390c-5479-46d7-af9f-3bcdc26f7a64","Type":"ContainerStarted","Data":"63f7f3d7ae2ebd000ee07149ab6e4c79c6c1fbd81e9011676ea1670d96608575"}
Apr 28 19:20:03.525334 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.525309    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8d5rz" event={"ID":"f6205f07-25d7-4a09-8c48-c01358a1e07c","Type":"ContainerStarted","Data":"55a6e1a3ab562cf154465a8db24ab5896007dfb639995e0f932e301134819992"}
Apr 28 19:20:03.801191 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.801165    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.803360 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.803341    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:03.942806 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:03.942768    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af47071b-d8de-49e7-b564-1ab0414ff0e9" path="/var/lib/kubelet/pods/af47071b-d8de-49e7-b564-1ab0414ff0e9/volumes"
Apr 28 19:20:04.033159 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:04.033080    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:20:04.164762 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:04.164701    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-ljn9c" podStartSLOduration=2.138737676 podStartE2EDuration="3.164679787s" podCreationTimestamp="2026-04-28 19:20:01 +0000 UTC" firstStartedPulling="2026-04-28 19:20:02.37143293 +0000 UTC m=+189.195773015" lastFinishedPulling="2026-04-28 19:20:03.397375032 +0000 UTC m=+190.221715126" observedRunningTime="2026-04-28 19:20:03.558094317 +0000 UTC m=+190.382434426" watchObservedRunningTime="2026-04-28 19:20:04.164679787 +0000 UTC m=+190.989019897"
Apr 28 19:20:04.165906 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:04.165884    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Apr 28 19:20:04.169802 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:20:04.169774    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0195c259_8934_4087_8e99_353b607f030e.slice/crio-d2aa409451ea58f1a13296f758da91ea66490ab4ab485ac7a4c8812cb3de5051 WatchSource:0}: Error finding container d2aa409451ea58f1a13296f758da91ea66490ab4ab485ac7a4c8812cb3de5051: Status 404 returned error can't find the container with id d2aa409451ea58f1a13296f758da91ea66490ab4ab485ac7a4c8812cb3de5051
Apr 28 19:20:04.533294 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:04.533256    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerStarted","Data":"d2aa409451ea58f1a13296f758da91ea66490ab4ab485ac7a4c8812cb3de5051"}
Apr 28 19:20:04.534562 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:04.534538    2569 generic.go:358] "Generic (PLEG): container finished" podID="f6205f07-25d7-4a09-8c48-c01358a1e07c" containerID="aefc31e2605bcbd85d3e505ee76084313db944b6b791bec41d4935b236659628" exitCode=0
Apr 28 19:20:04.534680 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:04.534617    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8d5rz" event={"ID":"f6205f07-25d7-4a09-8c48-c01358a1e07c","Type":"ContainerDied","Data":"aefc31e2605bcbd85d3e505ee76084313db944b6b791bec41d4935b236659628"}
Apr 28 19:20:05.013559 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.013522    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7896b685b8-qtmcb"]
Apr 28 19:20:05.016754 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.016731    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.021921 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.021706    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Apr 28 19:20:05.021921 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.021706    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Apr 28 19:20:05.021921 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.021756    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Apr 28 19:20:05.021921 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.021799    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-v7tmj\""
Apr 28 19:20:05.021921 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.021807    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e51n0talorut4\""
Apr 28 19:20:05.022237 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.022072    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Apr 28 19:20:05.022293 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.022235    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Apr 28 19:20:05.034974 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.034936    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7896b685b8-qtmcb"]
Apr 28 19:20:05.113147 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.113103    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.113337 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.113172    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-grpc-tls\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.113337 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.113206    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6wqj\" (UniqueName: \"kubernetes.io/projected/a9eb99a8-0b0b-48a5-9edc-69806958ec99-kube-api-access-j6wqj\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.113337 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.113309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.113510 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.113353    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.113510 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.113436    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9eb99a8-0b0b-48a5-9edc-69806958ec99-metrics-client-ca\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.113614 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.113541    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.113666 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.113615    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-tls\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.214564 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.214522    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.214987 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.214604    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-tls\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.214987 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.214653    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.214987 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.214682    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-grpc-tls\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.214987 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.214709    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j6wqj\" (UniqueName: \"kubernetes.io/projected/a9eb99a8-0b0b-48a5-9edc-69806958ec99-kube-api-access-j6wqj\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.214987 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.214750    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.214987 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.214774    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.214987 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.214813    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9eb99a8-0b0b-48a5-9edc-69806958ec99-metrics-client-ca\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.216092 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.216066    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9eb99a8-0b0b-48a5-9edc-69806958ec99-metrics-client-ca\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.217954 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.217690    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.218255 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.218231    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-tls\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.218361 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.218268    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-grpc-tls\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.218361 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.218273    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.218499 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.218434    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.218702 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.218683    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a9eb99a8-0b0b-48a5-9edc-69806958ec99-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.227018 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.226984    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6wqj\" (UniqueName: \"kubernetes.io/projected/a9eb99a8-0b0b-48a5-9edc-69806958ec99-kube-api-access-j6wqj\") pod \"thanos-querier-7896b685b8-qtmcb\" (UID: \"a9eb99a8-0b0b-48a5-9edc-69806958ec99\") " pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.328024 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.328002    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:05.468910 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.468842    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7896b685b8-qtmcb"]
Apr 28 19:20:05.472509 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:20:05.472482    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9eb99a8_0b0b_48a5_9edc_69806958ec99.slice/crio-ade34dbe9a57eff1f06c39aeb44f31fde57c51364d873ab1796405db4f394094 WatchSource:0}: Error finding container ade34dbe9a57eff1f06c39aeb44f31fde57c51364d873ab1796405db4f394094: Status 404 returned error can't find the container with id ade34dbe9a57eff1f06c39aeb44f31fde57c51364d873ab1796405db4f394094
Apr 28 19:20:05.539525 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.539489    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8d5rz" event={"ID":"f6205f07-25d7-4a09-8c48-c01358a1e07c","Type":"ContainerStarted","Data":"166f457927ec724e54b87c51f5769e0869d075136eb220a36ae182d9e1c430c6"}
Apr 28 19:20:05.539680 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.539530    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8d5rz" event={"ID":"f6205f07-25d7-4a09-8c48-c01358a1e07c","Type":"ContainerStarted","Data":"16cd7fa2826fcfcd8a6d7ca3414424ffe76679e1dff01780972ca6ad41201166"}
Apr 28 19:20:05.540532 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.540504    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb" event={"ID":"a9eb99a8-0b0b-48a5-9edc-69806958ec99","Type":"ContainerStarted","Data":"ade34dbe9a57eff1f06c39aeb44f31fde57c51364d873ab1796405db4f394094"}
Apr 28 19:20:05.541586 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.541561    2569 generic.go:358] "Generic (PLEG): container finished" podID="0195c259-8934-4087-8e99-353b607f030e" containerID="e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be" exitCode=0
Apr 28 19:20:05.541679 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.541597    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerDied","Data":"e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be"}
Apr 28 19:20:05.596737 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:05.596650    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8d5rz" podStartSLOduration=3.591964269 podStartE2EDuration="4.596636723s" podCreationTimestamp="2026-04-28 19:20:01 +0000 UTC" firstStartedPulling="2026-04-28 19:20:02.731103928 +0000 UTC m=+189.555444014" lastFinishedPulling="2026-04-28 19:20:03.735776371 +0000 UTC m=+190.560116468" observedRunningTime="2026-04-28 19:20:05.595547974 +0000 UTC m=+192.419888082" watchObservedRunningTime="2026-04-28 19:20:05.596636723 +0000 UTC m=+192.420976899"
Apr 28 19:20:06.293791 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.293551    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-8677d4bd78-9jx6t"]
Apr 28 19:20:06.296290 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.296241    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.298880 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.298854    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Apr 28 19:20:06.300183 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.300131    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Apr 28 19:20:06.300301 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.300226    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4tqwj\""
Apr 28 19:20:06.300301 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.300131    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Apr 28 19:20:06.300440 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.300131    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Apr 28 19:20:06.300767 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.300746    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ln6as1ue9bmv\""
Apr 28 19:20:06.306534 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.306162    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8677d4bd78-9jx6t"]
Apr 28 19:20:06.426684 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.426577    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-client-ca-bundle\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.426684 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.426642    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-secret-metrics-server-tls\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.426684 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.426684    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-audit-log\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.426971 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.426710    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-metrics-server-audit-profiles\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.427424 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.427114    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-486fn\" (UniqueName: \"kubernetes.io/projected/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-kube-api-access-486fn\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.427558 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.427442    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.427558 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.427501    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-secret-metrics-server-client-certs\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.529544 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.529179    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.529544 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.529240    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-secret-metrics-server-client-certs\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.529544 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.529316    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-client-ca-bundle\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.529544 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.529367    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-secret-metrics-server-tls\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.529544 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.529393    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-audit-log\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.529909 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.529694    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-metrics-server-audit-profiles\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.529909 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.529750    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-486fn\" (UniqueName: \"kubernetes.io/projected/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-kube-api-access-486fn\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.530034 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.530010    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.530749 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.530721    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-audit-log\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.531024 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.530968    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-metrics-server-audit-profiles\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.535240 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.534745    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-secret-metrics-server-client-certs\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.535240 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.535198    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-secret-metrics-server-tls\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.537112 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.537087    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-client-ca-bundle\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.542780 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.542735    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-486fn\" (UniqueName: \"kubernetes.io/projected/a20ecd11-5310-4c8d-ae67-90b4bf922cb7-kube-api-access-486fn\") pod \"metrics-server-8677d4bd78-9jx6t\" (UID: \"a20ecd11-5310-4c8d-ae67-90b4bf922cb7\") " pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.543983 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.543462    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-d7d9c4d75-5gvsj"]
Apr 28 19:20:06.547571 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.547010    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.558297 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.558265    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d7d9c4d75-5gvsj"]
Apr 28 19:20:06.593112 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.593082    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7dccd58f55-lpxtz"]
Apr 28 19:20:06.595886 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.595614    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-lpxtz"
Apr 28 19:20:06.598742 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.598261    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-kqwkk\""
Apr 28 19:20:06.598742 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.598570    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Apr 28 19:20:06.617429 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.609832    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:06.617429 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.611700    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7dccd58f55-lpxtz"]
Apr 28 19:20:06.736754 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.736658    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-serving-cert\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.736754 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.736716    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/de9f763c-a931-402d-abd1-2f0b4f2971ff-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-lpxtz\" (UID: \"de9f763c-a931-402d-abd1-2f0b4f2971ff\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-lpxtz"
Apr 28 19:20:06.737001 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.736798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-oauth-config\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.737001 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.736861    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-config\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.737001 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.736990    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdzqd\" (UniqueName: \"kubernetes.io/projected/d4754506-9f06-45d8-bf0d-1643f89dbc37-kube-api-access-fdzqd\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.737157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.737039    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-trusted-ca-bundle\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.737157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.737107    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-service-ca\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.737257 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.737182    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-oauth-serving-cert\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.838514 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.838420    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fdzqd\" (UniqueName: \"kubernetes.io/projected/d4754506-9f06-45d8-bf0d-1643f89dbc37-kube-api-access-fdzqd\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.838514 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.838511    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-trusted-ca-bundle\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.838758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.838559    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-service-ca\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.838758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.838619    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-oauth-serving-cert\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.838758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.838673    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-serving-cert\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.838758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.838704    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/de9f763c-a931-402d-abd1-2f0b4f2971ff-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-lpxtz\" (UID: \"de9f763c-a931-402d-abd1-2f0b4f2971ff\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-lpxtz"
Apr 28 19:20:06.838758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.838722    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-oauth-config\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.838758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.838742    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-config\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.839477 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.839386    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-service-ca\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.839610 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.839527    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-oauth-serving-cert\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.839610 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.839579    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-config\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.839610 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.839602    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-trusted-ca-bundle\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.841596 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.841563    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-serving-cert\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.841596 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.841577    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-oauth-config\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.841596 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.841573    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/de9f763c-a931-402d-abd1-2f0b4f2971ff-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-lpxtz\" (UID: \"de9f763c-a931-402d-abd1-2f0b4f2971ff\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-lpxtz"
Apr 28 19:20:06.853982 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.853958    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdzqd\" (UniqueName: \"kubernetes.io/projected/d4754506-9f06-45d8-bf0d-1643f89dbc37-kube-api-access-fdzqd\") pod \"console-d7d9c4d75-5gvsj\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") " pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.864713 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.864690    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:06.906959 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:06.906923    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-lpxtz"
Apr 28 19:20:07.609359 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:07.609311    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7dccd58f55-lpxtz"]
Apr 28 19:20:07.630517 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:07.630494    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8677d4bd78-9jx6t"]
Apr 28 19:20:07.635077 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:20:07.635051    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda20ecd11_5310_4c8d_ae67_90b4bf922cb7.slice/crio-225086a9a2898f466359f2f03520c3b299814132ca2ba4973fc9247ed81d3859 WatchSource:0}: Error finding container 225086a9a2898f466359f2f03520c3b299814132ca2ba4973fc9247ed81d3859: Status 404 returned error can't find the container with id 225086a9a2898f466359f2f03520c3b299814132ca2ba4973fc9247ed81d3859
Apr 28 19:20:07.668589 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:07.668565    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d7d9c4d75-5gvsj"]
Apr 28 19:20:07.677554 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:20:07.677526    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4754506_9f06_45d8_bf0d_1643f89dbc37.slice/crio-5056b32b1d76ed6fc6ef8db4d287a31e8e806016b11b7e889409b83b3227dcb4 WatchSource:0}: Error finding container 5056b32b1d76ed6fc6ef8db4d287a31e8e806016b11b7e889409b83b3227dcb4: Status 404 returned error can't find the container with id 5056b32b1d76ed6fc6ef8db4d287a31e8e806016b11b7e889409b83b3227dcb4
Apr 28 19:20:08.558298 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:08.557542    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7d9c4d75-5gvsj" event={"ID":"d4754506-9f06-45d8-bf0d-1643f89dbc37","Type":"ContainerStarted","Data":"dcdfea1d0fbde2f08a83c7ea21bb7e5307c1fc4e74bb07c7054893796e4cf4ef"}
Apr 28 19:20:08.558298 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:08.557589    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7d9c4d75-5gvsj" event={"ID":"d4754506-9f06-45d8-bf0d-1643f89dbc37","Type":"ContainerStarted","Data":"5056b32b1d76ed6fc6ef8db4d287a31e8e806016b11b7e889409b83b3227dcb4"}
Apr 28 19:20:08.561617 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:08.561535    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb" event={"ID":"a9eb99a8-0b0b-48a5-9edc-69806958ec99","Type":"ContainerStarted","Data":"32891e8db8557837da7bfb797964e9e3556a2f7e7f20eb25447848871426f2ea"}
Apr 28 19:20:08.561617 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:08.561566    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb" event={"ID":"a9eb99a8-0b0b-48a5-9edc-69806958ec99","Type":"ContainerStarted","Data":"99057148eb8e32022aa043c59ac936ea0fa441965eb4fa0927fe36c6afee0bfa"}
Apr 28 19:20:08.561617 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:08.561581    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb" event={"ID":"a9eb99a8-0b0b-48a5-9edc-69806958ec99","Type":"ContainerStarted","Data":"007df35e398b4a66ef3d7dd07364213857d5fa64ab26755805995c25b8ad9dd8"}
Apr 28 19:20:08.570423 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:08.569297    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerStarted","Data":"2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931"}
Apr 28 19:20:08.570423 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:08.569334    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerStarted","Data":"45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151"}
Apr 28 19:20:08.570423 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:08.569348    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerStarted","Data":"417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe"}
Apr 28 19:20:08.570423 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:08.569362    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerStarted","Data":"409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105"}
Apr 28 19:20:08.570423 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:08.569376    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerStarted","Data":"2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23"}
Apr 28 19:20:08.575982 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:08.572245    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t" event={"ID":"a20ecd11-5310-4c8d-ae67-90b4bf922cb7","Type":"ContainerStarted","Data":"225086a9a2898f466359f2f03520c3b299814132ca2ba4973fc9247ed81d3859"}
Apr 28 19:20:08.576086 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:08.576017    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-lpxtz" event={"ID":"de9f763c-a931-402d-abd1-2f0b4f2971ff","Type":"ContainerStarted","Data":"40ea03c21d83bce50ffe232d4cfd106ecc12ea5ca2390870a1f89e47e3cb2f58"}
Apr 28 19:20:08.588508 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:08.587743    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-d7d9c4d75-5gvsj" podStartSLOduration=2.587727963 podStartE2EDuration="2.587727963s" podCreationTimestamp="2026-04-28 19:20:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:20:08.582809394 +0000 UTC m=+195.407149493" watchObservedRunningTime="2026-04-28 19:20:08.587727963 +0000 UTC m=+195.412068073"
Apr 28 19:20:09.581975 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:09.581942    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb" event={"ID":"a9eb99a8-0b0b-48a5-9edc-69806958ec99","Type":"ContainerStarted","Data":"75e511e1c997e3fb04b3c0e9a88e4319a8fe8eada3e428f180758924ad717c21"}
Apr 28 19:20:09.585345 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:09.585316    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerStarted","Data":"6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc"}
Apr 28 19:20:09.586731 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:09.586709    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t" event={"ID":"a20ecd11-5310-4c8d-ae67-90b4bf922cb7","Type":"ContainerStarted","Data":"ab90763d742d913d96495cfd7cf5f6f7147436d950c768426d5f8c85637b6b15"}
Apr 28 19:20:09.588711 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:09.588611    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-lpxtz" event={"ID":"de9f763c-a931-402d-abd1-2f0b4f2971ff","Type":"ContainerStarted","Data":"a8b0dd179dd59fd6688b0c93cda7013c17fceeda258912c96e34d92c73d036ce"}
Apr 28 19:20:09.588886 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:09.588857    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-lpxtz"
Apr 28 19:20:09.594367 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:09.594356    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-lpxtz"
Apr 28 19:20:09.621129 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:09.621069    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.344226051 podStartE2EDuration="6.621052213s" podCreationTimestamp="2026-04-28 19:20:03 +0000 UTC" firstStartedPulling="2026-04-28 19:20:04.172006668 +0000 UTC m=+190.996346754" lastFinishedPulling="2026-04-28 19:20:09.448832823 +0000 UTC m=+196.273172916" observedRunningTime="2026-04-28 19:20:09.617602798 +0000 UTC m=+196.441942908" watchObservedRunningTime="2026-04-28 19:20:09.621052213 +0000 UTC m=+196.445392323"
Apr 28 19:20:09.637336 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:09.636932    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t" podStartSLOduration=1.824365883 podStartE2EDuration="3.636920314s" podCreationTimestamp="2026-04-28 19:20:06 +0000 UTC" firstStartedPulling="2026-04-28 19:20:07.637206514 +0000 UTC m=+194.461546615" lastFinishedPulling="2026-04-28 19:20:09.449760948 +0000 UTC m=+196.274101046" observedRunningTime="2026-04-28 19:20:09.636836769 +0000 UTC m=+196.461176879" watchObservedRunningTime="2026-04-28 19:20:09.636920314 +0000 UTC m=+196.461260423"
Apr 28 19:20:09.653425 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:09.653107    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-lpxtz" podStartSLOduration=1.822668502 podStartE2EDuration="3.653093354s" podCreationTimestamp="2026-04-28 19:20:06 +0000 UTC" firstStartedPulling="2026-04-28 19:20:07.620262331 +0000 UTC m=+194.444602437" lastFinishedPulling="2026-04-28 19:20:09.450687204 +0000 UTC m=+196.275027289" observedRunningTime="2026-04-28 19:20:09.652140406 +0000 UTC m=+196.476480515" watchObservedRunningTime="2026-04-28 19:20:09.653093354 +0000 UTC m=+196.477433452"
Apr 28 19:20:10.594453 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:10.594393    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb" event={"ID":"a9eb99a8-0b0b-48a5-9edc-69806958ec99","Type":"ContainerStarted","Data":"982ee5b467f76bf4f6cac5492a47cf9a561c687e92b22e185dfb264c9d42e6d4"}
Apr 28 19:20:10.594453 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:10.594458    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb" event={"ID":"a9eb99a8-0b0b-48a5-9edc-69806958ec99","Type":"ContainerStarted","Data":"e343c182e89a78e048f6783ae0b4c04a48c0a4ddfaaeaf819a486b25d07458bd"}
Apr 28 19:20:10.626981 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:10.626934    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb" podStartSLOduration=2.649620706 podStartE2EDuration="6.626920468s" podCreationTimestamp="2026-04-28 19:20:04 +0000 UTC" firstStartedPulling="2026-04-28 19:20:05.474261031 +0000 UTC m=+192.298601117" lastFinishedPulling="2026-04-28 19:20:09.451560792 +0000 UTC m=+196.275900879" observedRunningTime="2026-04-28 19:20:10.624209318 +0000 UTC m=+197.448549427" watchObservedRunningTime="2026-04-28 19:20:10.626920468 +0000 UTC m=+197.451260614"
Apr 28 19:20:11.597799 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:11.597767    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:12.319603 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.319573    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d7d9c4d75-5gvsj"]
Apr 28 19:20:12.384438 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.384375    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-648ffdfc7d-gcz5f"]
Apr 28 19:20:12.386915 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.386892    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.416275 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.416246    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-648ffdfc7d-gcz5f"]
Apr 28 19:20:12.489736 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.489713    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-serving-cert\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.489851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.489744    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-oauth-config\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.489851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.489772    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-trusted-ca-bundle\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.489924 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.489854    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-oauth-serving-cert\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.489924 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.489896    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-config\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.489985 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.489934    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dwxr\" (UniqueName: \"kubernetes.io/projected/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-kube-api-access-9dwxr\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.489985 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.489972    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-service-ca\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.591169 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.591096    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9dwxr\" (UniqueName: \"kubernetes.io/projected/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-kube-api-access-9dwxr\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.591169 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.591145    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-service-ca\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.591343 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.591182    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-serving-cert\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.591343 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.591204    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-oauth-config\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.591343 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.591243    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-trusted-ca-bundle\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.591516 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.591491    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-oauth-serving-cert\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.591568 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.591550    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-config\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.592060 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.592030    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-service-ca\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.592175 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.592136    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-trusted-ca-bundle\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.592310 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.592284    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-config\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.592374 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.592290    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-oauth-serving-cert\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.593700 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.593674    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-oauth-config\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.593896 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.593876    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-serving-cert\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.599809 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.599787    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dwxr\" (UniqueName: \"kubernetes.io/projected/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-kube-api-access-9dwxr\") pod \"console-648ffdfc7d-gcz5f\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") " pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.606311 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.606273    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7896b685b8-qtmcb"
Apr 28 19:20:12.696512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.696483    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:12.817933 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:12.817904    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-648ffdfc7d-gcz5f"]
Apr 28 19:20:12.821524 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:20:12.821496    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ff85e51_6ce9_4350_bfa8_d7d0b10e1574.slice/crio-61df176cdce65a05e7ed1c4d70a49f557919956408a21ba5baaa5ecf0a22bfa9 WatchSource:0}: Error finding container 61df176cdce65a05e7ed1c4d70a49f557919956408a21ba5baaa5ecf0a22bfa9: Status 404 returned error can't find the container with id 61df176cdce65a05e7ed1c4d70a49f557919956408a21ba5baaa5ecf0a22bfa9
Apr 28 19:20:13.604340 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:13.604304    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648ffdfc7d-gcz5f" event={"ID":"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574","Type":"ContainerStarted","Data":"843218273d67468710a08ee487a3c9f1230fa59a19e12a7e8836231a3dda78c8"}
Apr 28 19:20:13.604748 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:13.604349    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648ffdfc7d-gcz5f" event={"ID":"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574","Type":"ContainerStarted","Data":"61df176cdce65a05e7ed1c4d70a49f557919956408a21ba5baaa5ecf0a22bfa9"}
Apr 28 19:20:13.632705 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:13.632649    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-648ffdfc7d-gcz5f" podStartSLOduration=1.632635365 podStartE2EDuration="1.632635365s" podCreationTimestamp="2026-04-28 19:20:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:20:13.632353902 +0000 UTC m=+200.456694010" watchObservedRunningTime="2026-04-28 19:20:13.632635365 +0000 UTC m=+200.456975473"
Apr 28 19:20:16.865627 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:16.865590    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:22.696793 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:22.696745    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:22.696793 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:22.696802    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:22.701679 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:22.701655    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:23.636254 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:23.636229    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:20:23.719006 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:23.718977    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f6cb8675f-ccr8c"]
Apr 28 19:20:26.611133 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:26.611102    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:26.611133 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:26.611142    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:37.338210 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.338152    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-d7d9c4d75-5gvsj" podUID="d4754506-9f06-45d8-bf0d-1643f89dbc37" containerName="console" containerID="cri-o://dcdfea1d0fbde2f08a83c7ea21bb7e5307c1fc4e74bb07c7054893796e4cf4ef" gracePeriod=15
Apr 28 19:20:37.572113 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.572087    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d7d9c4d75-5gvsj_d4754506-9f06-45d8-bf0d-1643f89dbc37/console/0.log"
Apr 28 19:20:37.572216 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.572143    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:37.676590 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.676531    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d7d9c4d75-5gvsj_d4754506-9f06-45d8-bf0d-1643f89dbc37/console/0.log"
Apr 28 19:20:37.676590 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.676569    2569 generic.go:358] "Generic (PLEG): container finished" podID="d4754506-9f06-45d8-bf0d-1643f89dbc37" containerID="dcdfea1d0fbde2f08a83c7ea21bb7e5307c1fc4e74bb07c7054893796e4cf4ef" exitCode=2
Apr 28 19:20:37.676730 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.676614    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7d9c4d75-5gvsj" event={"ID":"d4754506-9f06-45d8-bf0d-1643f89dbc37","Type":"ContainerDied","Data":"dcdfea1d0fbde2f08a83c7ea21bb7e5307c1fc4e74bb07c7054893796e4cf4ef"}
Apr 28 19:20:37.676730 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.676637    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d7d9c4d75-5gvsj"
Apr 28 19:20:37.676730 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.676648    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d7d9c4d75-5gvsj" event={"ID":"d4754506-9f06-45d8-bf0d-1643f89dbc37","Type":"ContainerDied","Data":"5056b32b1d76ed6fc6ef8db4d287a31e8e806016b11b7e889409b83b3227dcb4"}
Apr 28 19:20:37.676730 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.676666    2569 scope.go:117] "RemoveContainer" containerID="dcdfea1d0fbde2f08a83c7ea21bb7e5307c1fc4e74bb07c7054893796e4cf4ef"
Apr 28 19:20:37.684069 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.684053    2569 scope.go:117] "RemoveContainer" containerID="dcdfea1d0fbde2f08a83c7ea21bb7e5307c1fc4e74bb07c7054893796e4cf4ef"
Apr 28 19:20:37.684332 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:20:37.684312    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcdfea1d0fbde2f08a83c7ea21bb7e5307c1fc4e74bb07c7054893796e4cf4ef\": container with ID starting with dcdfea1d0fbde2f08a83c7ea21bb7e5307c1fc4e74bb07c7054893796e4cf4ef not found: ID does not exist" containerID="dcdfea1d0fbde2f08a83c7ea21bb7e5307c1fc4e74bb07c7054893796e4cf4ef"
Apr 28 19:20:37.684417 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.684346    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcdfea1d0fbde2f08a83c7ea21bb7e5307c1fc4e74bb07c7054893796e4cf4ef"} err="failed to get container status \"dcdfea1d0fbde2f08a83c7ea21bb7e5307c1fc4e74bb07c7054893796e4cf4ef\": rpc error: code = NotFound desc = could not find container \"dcdfea1d0fbde2f08a83c7ea21bb7e5307c1fc4e74bb07c7054893796e4cf4ef\": container with ID starting with dcdfea1d0fbde2f08a83c7ea21bb7e5307c1fc4e74bb07c7054893796e4cf4ef not found: ID does not exist"
Apr 28 19:20:37.700673 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.700654    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-oauth-serving-cert\") pod \"d4754506-9f06-45d8-bf0d-1643f89dbc37\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") "
Apr 28 19:20:37.700749 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.700678    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-trusted-ca-bundle\") pod \"d4754506-9f06-45d8-bf0d-1643f89dbc37\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") "
Apr 28 19:20:37.700749 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.700695    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-service-ca\") pod \"d4754506-9f06-45d8-bf0d-1643f89dbc37\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") "
Apr 28 19:20:37.700749 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.700724    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-oauth-config\") pod \"d4754506-9f06-45d8-bf0d-1643f89dbc37\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") "
Apr 28 19:20:37.700882 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.700803    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdzqd\" (UniqueName: \"kubernetes.io/projected/d4754506-9f06-45d8-bf0d-1643f89dbc37-kube-api-access-fdzqd\") pod \"d4754506-9f06-45d8-bf0d-1643f89dbc37\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") "
Apr 28 19:20:37.700882 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.700839    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-config\") pod \"d4754506-9f06-45d8-bf0d-1643f89dbc37\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") "
Apr 28 19:20:37.700973 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.700888    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-serving-cert\") pod \"d4754506-9f06-45d8-bf0d-1643f89dbc37\" (UID: \"d4754506-9f06-45d8-bf0d-1643f89dbc37\") "
Apr 28 19:20:37.701055 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.701027    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d4754506-9f06-45d8-bf0d-1643f89dbc37" (UID: "d4754506-9f06-45d8-bf0d-1643f89dbc37"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:20:37.701185 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.701072    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-service-ca" (OuterVolumeSpecName: "service-ca") pod "d4754506-9f06-45d8-bf0d-1643f89dbc37" (UID: "d4754506-9f06-45d8-bf0d-1643f89dbc37"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:20:37.701276 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.701168    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d4754506-9f06-45d8-bf0d-1643f89dbc37" (UID: "d4754506-9f06-45d8-bf0d-1643f89dbc37"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:20:37.701276 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.701246    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-config" (OuterVolumeSpecName: "console-config") pod "d4754506-9f06-45d8-bf0d-1643f89dbc37" (UID: "d4754506-9f06-45d8-bf0d-1643f89dbc37"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:20:37.701370 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.701322    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-oauth-serving-cert\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:37.701370 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.701337    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-trusted-ca-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:37.701370 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.701346    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-service-ca\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:37.701370 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.701355    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-config\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:37.703045 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.703011    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d4754506-9f06-45d8-bf0d-1643f89dbc37" (UID: "d4754506-9f06-45d8-bf0d-1643f89dbc37"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:20:37.703045 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.703028    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d4754506-9f06-45d8-bf0d-1643f89dbc37" (UID: "d4754506-9f06-45d8-bf0d-1643f89dbc37"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:20:37.703045 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.703024    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4754506-9f06-45d8-bf0d-1643f89dbc37-kube-api-access-fdzqd" (OuterVolumeSpecName: "kube-api-access-fdzqd") pod "d4754506-9f06-45d8-bf0d-1643f89dbc37" (UID: "d4754506-9f06-45d8-bf0d-1643f89dbc37"). InnerVolumeSpecName "kube-api-access-fdzqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:20:37.802177 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.802146    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-oauth-config\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:37.802177 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.802173    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdzqd\" (UniqueName: \"kubernetes.io/projected/d4754506-9f06-45d8-bf0d-1643f89dbc37-kube-api-access-fdzqd\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:37.802177 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.802182    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4754506-9f06-45d8-bf0d-1643f89dbc37-console-serving-cert\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:37.993082 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.993048    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d7d9c4d75-5gvsj"]
Apr 28 19:20:37.997042 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:37.997020    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d7d9c4d75-5gvsj"]
Apr 28 19:20:39.942369 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:39.942325    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4754506-9f06-45d8-bf0d-1643f89dbc37" path="/var/lib/kubelet/pods/d4754506-9f06-45d8-bf0d-1643f89dbc37/volumes"
Apr 28 19:20:45.702846 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:45.702812    2569 generic.go:358] "Generic (PLEG): container finished" podID="abc28c3c-a5be-45cd-8095-f9d59dd936dc" containerID="9cd9b7cc241ac855faff2dd1e8bf7c8bc70835f3d2c8b1aece1ae3d722c99661" exitCode=0
Apr 28 19:20:45.703258 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:45.702878    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8" event={"ID":"abc28c3c-a5be-45cd-8095-f9d59dd936dc","Type":"ContainerDied","Data":"9cd9b7cc241ac855faff2dd1e8bf7c8bc70835f3d2c8b1aece1ae3d722c99661"}
Apr 28 19:20:45.703258 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:45.703176    2569 scope.go:117] "RemoveContainer" containerID="9cd9b7cc241ac855faff2dd1e8bf7c8bc70835f3d2c8b1aece1ae3d722c99661"
Apr 28 19:20:46.615739 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:46.615709    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:46.619496 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:46.619461    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-8677d4bd78-9jx6t"
Apr 28 19:20:46.708206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:46.708175    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-x8wk8" event={"ID":"abc28c3c-a5be-45cd-8095-f9d59dd936dc","Type":"ContainerStarted","Data":"38f14a125df2af0e8e6610e7fefa0fb9b3d27a7a1310a7a1b61b2c630331aaff"}
Apr 28 19:20:48.738575 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:48.738516    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-5f6cb8675f-ccr8c" podUID="0a216f6e-4215-47a2-bfa3-2e0e5868e25c" containerName="console" containerID="cri-o://294316338f1126226410d9d379b6460c9b5a6852a65ad904722bcae387188eb0" gracePeriod=15
Apr 28 19:20:48.972353 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:48.972332    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5f6cb8675f-ccr8c_0a216f6e-4215-47a2-bfa3-2e0e5868e25c/console/0.log"
Apr 28 19:20:48.972478 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:48.972386    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:20:49.100886 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.100825    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-oauth-config\") pod \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") "
Apr 28 19:20:49.100886 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.100858    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbz52\" (UniqueName: \"kubernetes.io/projected/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-kube-api-access-gbz52\") pod \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") "
Apr 28 19:20:49.100886 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.100876    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-service-ca\") pod \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") "
Apr 28 19:20:49.101098 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.101010    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-config\") pod \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") "
Apr 28 19:20:49.101098 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.101050    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-trusted-ca-bundle\") pod \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") "
Apr 28 19:20:49.101098 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.101084    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-oauth-serving-cert\") pod \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") "
Apr 28 19:20:49.101230 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.101122    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-serving-cert\") pod \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\" (UID: \"0a216f6e-4215-47a2-bfa3-2e0e5868e25c\") "
Apr 28 19:20:49.101230 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.101179    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-service-ca" (OuterVolumeSpecName: "service-ca") pod "0a216f6e-4215-47a2-bfa3-2e0e5868e25c" (UID: "0a216f6e-4215-47a2-bfa3-2e0e5868e25c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:20:49.101458 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.101434    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-service-ca\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:49.101594 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.101544    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0a216f6e-4215-47a2-bfa3-2e0e5868e25c" (UID: "0a216f6e-4215-47a2-bfa3-2e0e5868e25c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:20:49.101594 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.101552    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0a216f6e-4215-47a2-bfa3-2e0e5868e25c" (UID: "0a216f6e-4215-47a2-bfa3-2e0e5868e25c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:20:49.101701 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.101591    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-config" (OuterVolumeSpecName: "console-config") pod "0a216f6e-4215-47a2-bfa3-2e0e5868e25c" (UID: "0a216f6e-4215-47a2-bfa3-2e0e5868e25c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:20:49.103054 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.103032    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0a216f6e-4215-47a2-bfa3-2e0e5868e25c" (UID: "0a216f6e-4215-47a2-bfa3-2e0e5868e25c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:20:49.103147 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.103078    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0a216f6e-4215-47a2-bfa3-2e0e5868e25c" (UID: "0a216f6e-4215-47a2-bfa3-2e0e5868e25c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:20:49.103147 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.103081    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-kube-api-access-gbz52" (OuterVolumeSpecName: "kube-api-access-gbz52") pod "0a216f6e-4215-47a2-bfa3-2e0e5868e25c" (UID: "0a216f6e-4215-47a2-bfa3-2e0e5868e25c"). InnerVolumeSpecName "kube-api-access-gbz52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:20:49.201783 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.201758    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-trusted-ca-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:49.201783 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.201781    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-oauth-serving-cert\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:49.201913 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.201791    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-serving-cert\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:49.201913 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.201809    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-oauth-config\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:49.201913 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.201820    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbz52\" (UniqueName: \"kubernetes.io/projected/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-kube-api-access-gbz52\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:49.201913 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.201834    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a216f6e-4215-47a2-bfa3-2e0e5868e25c-console-config\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:20:49.719386 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.719359    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5f6cb8675f-ccr8c_0a216f6e-4215-47a2-bfa3-2e0e5868e25c/console/0.log"
Apr 28 19:20:49.719584 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.719413    2569 generic.go:358] "Generic (PLEG): container finished" podID="0a216f6e-4215-47a2-bfa3-2e0e5868e25c" containerID="294316338f1126226410d9d379b6460c9b5a6852a65ad904722bcae387188eb0" exitCode=2
Apr 28 19:20:49.719584 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.719452    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f6cb8675f-ccr8c" event={"ID":"0a216f6e-4215-47a2-bfa3-2e0e5868e25c","Type":"ContainerDied","Data":"294316338f1126226410d9d379b6460c9b5a6852a65ad904722bcae387188eb0"}
Apr 28 19:20:49.719584 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.719482    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f6cb8675f-ccr8c"
Apr 28 19:20:49.719584 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.719492    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f6cb8675f-ccr8c" event={"ID":"0a216f6e-4215-47a2-bfa3-2e0e5868e25c","Type":"ContainerDied","Data":"070a3c34c7ed144ac6e317d1b8f43008d88f51514f5295162696d40eb910c569"}
Apr 28 19:20:49.719584 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.719510    2569 scope.go:117] "RemoveContainer" containerID="294316338f1126226410d9d379b6460c9b5a6852a65ad904722bcae387188eb0"
Apr 28 19:20:49.728133 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.728116    2569 scope.go:117] "RemoveContainer" containerID="294316338f1126226410d9d379b6460c9b5a6852a65ad904722bcae387188eb0"
Apr 28 19:20:49.728372 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:20:49.728351    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"294316338f1126226410d9d379b6460c9b5a6852a65ad904722bcae387188eb0\": container with ID starting with 294316338f1126226410d9d379b6460c9b5a6852a65ad904722bcae387188eb0 not found: ID does not exist" containerID="294316338f1126226410d9d379b6460c9b5a6852a65ad904722bcae387188eb0"
Apr 28 19:20:49.728462 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.728380    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"294316338f1126226410d9d379b6460c9b5a6852a65ad904722bcae387188eb0"} err="failed to get container status \"294316338f1126226410d9d379b6460c9b5a6852a65ad904722bcae387188eb0\": rpc error: code = NotFound desc = could not find container \"294316338f1126226410d9d379b6460c9b5a6852a65ad904722bcae387188eb0\": container with ID starting with 294316338f1126226410d9d379b6460c9b5a6852a65ad904722bcae387188eb0 not found: ID does not exist"
Apr 28 19:20:49.740763 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.740738    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f6cb8675f-ccr8c"]
Apr 28 19:20:49.744463 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.744443    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f6cb8675f-ccr8c"]
Apr 28 19:20:49.943027 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:49.942997    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a216f6e-4215-47a2-bfa3-2e0e5868e25c" path="/var/lib/kubelet/pods/0a216f6e-4215-47a2-bfa3-2e0e5868e25c/volumes"
Apr 28 19:20:50.723724 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:50.723695    2569 generic.go:358] "Generic (PLEG): container finished" podID="10e853da-a692-43bb-b339-3bf2059fd151" containerID="e475b538e0b5217482754d9a0b908e5e25d207147f1f0f69b5fa2f1306563d4a" exitCode=0
Apr 28 19:20:50.723886 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:50.723746    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-x7kx7" event={"ID":"10e853da-a692-43bb-b339-3bf2059fd151","Type":"ContainerDied","Data":"e475b538e0b5217482754d9a0b908e5e25d207147f1f0f69b5fa2f1306563d4a"}
Apr 28 19:20:50.724064 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:50.724050    2569 scope.go:117] "RemoveContainer" containerID="e475b538e0b5217482754d9a0b908e5e25d207147f1f0f69b5fa2f1306563d4a"
Apr 28 19:20:51.728583 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:20:51.728551    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-x7kx7" event={"ID":"10e853da-a692-43bb-b339-3bf2059fd151","Type":"ContainerStarted","Data":"8326a55975892a15f56d742da168e99209802fd601f3324674f79066a05226c7"}
Apr 28 19:21:05.642050 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:05.642012    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:21:05.644285 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:05.644263    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33-metrics-certs\") pod \"network-metrics-daemon-hgm2f\" (UID: \"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33\") " pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:21:05.741017 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:05.740986    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cnh2p\""
Apr 28 19:21:05.749236 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:05.749213    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgm2f"
Apr 28 19:21:05.869108 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:05.869083    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hgm2f"]
Apr 28 19:21:05.871212 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:21:05.871179    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode09c6f20_b816_4ac6_a0f3_37fd5d8a2d33.slice/crio-7c4b42df31b0a816091514ad32e7751866b09c640f892970fb81bb5248aec290 WatchSource:0}: Error finding container 7c4b42df31b0a816091514ad32e7751866b09c640f892970fb81bb5248aec290: Status 404 returned error can't find the container with id 7c4b42df31b0a816091514ad32e7751866b09c640f892970fb81bb5248aec290
Apr 28 19:21:06.776845 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:06.776802    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hgm2f" event={"ID":"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33","Type":"ContainerStarted","Data":"7c4b42df31b0a816091514ad32e7751866b09c640f892970fb81bb5248aec290"}
Apr 28 19:21:07.780704 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:07.780667    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hgm2f" event={"ID":"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33","Type":"ContainerStarted","Data":"7059a2418ecfa3c9626ee45816f6af2d9ff19b529380e3ea073ab67634a52a8d"}
Apr 28 19:21:07.780704 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:07.780704    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hgm2f" event={"ID":"e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33","Type":"ContainerStarted","Data":"e9a162e43092691cb3d89f9ac1e5f901da282e27c8bde72845fbd328f9c1f239"}
Apr 28 19:21:07.801213 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:07.801167    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hgm2f" podStartSLOduration=252.765365904 podStartE2EDuration="4m13.801153053s" podCreationTimestamp="2026-04-28 19:16:54 +0000 UTC" firstStartedPulling="2026-04-28 19:21:05.872934057 +0000 UTC m=+252.697274143" lastFinishedPulling="2026-04-28 19:21:06.908721192 +0000 UTC m=+253.733061292" observedRunningTime="2026-04-28 19:21:07.800226304 +0000 UTC m=+254.624566414" watchObservedRunningTime="2026-04-28 19:21:07.801153053 +0000 UTC m=+254.625493215"
Apr 28 19:21:22.411379 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.411300    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Apr 28 19:21:22.411856 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.411733    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="alertmanager" containerID="cri-o://2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23" gracePeriod=120
Apr 28 19:21:22.411856 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.411783    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="kube-rbac-proxy-metric" containerID="cri-o://2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931" gracePeriod=120
Apr 28 19:21:22.411970 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.411832    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="kube-rbac-proxy" containerID="cri-o://45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151" gracePeriod=120
Apr 28 19:21:22.411970 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.411891    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="prom-label-proxy" containerID="cri-o://6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc" gracePeriod=120
Apr 28 19:21:22.411970 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.411832    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="kube-rbac-proxy-web" containerID="cri-o://417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe" gracePeriod=120
Apr 28 19:21:22.411970 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.411904    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="config-reloader" containerID="cri-o://409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105" gracePeriod=120
Apr 28 19:21:22.828149 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.828118    2569 generic.go:358] "Generic (PLEG): container finished" podID="0195c259-8934-4087-8e99-353b607f030e" containerID="6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc" exitCode=0
Apr 28 19:21:22.828149 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.828141    2569 generic.go:358] "Generic (PLEG): container finished" podID="0195c259-8934-4087-8e99-353b607f030e" containerID="45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151" exitCode=0
Apr 28 19:21:22.828149 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.828147    2569 generic.go:358] "Generic (PLEG): container finished" podID="0195c259-8934-4087-8e99-353b607f030e" containerID="409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105" exitCode=0
Apr 28 19:21:22.828149 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.828153    2569 generic.go:358] "Generic (PLEG): container finished" podID="0195c259-8934-4087-8e99-353b607f030e" containerID="2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23" exitCode=0
Apr 28 19:21:22.828420 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.828186    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerDied","Data":"6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc"}
Apr 28 19:21:22.828420 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.828214    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerDied","Data":"45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151"}
Apr 28 19:21:22.828420 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.828225    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerDied","Data":"409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105"}
Apr 28 19:21:22.828420 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:22.828233    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerDied","Data":"2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23"}
Apr 28 19:21:23.651029 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.651007    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.687932 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.687910    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy-web\") pod \"0195c259-8934-4087-8e99-353b607f030e\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") "
Apr 28 19:21:23.688046 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.687948    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-242gf\" (UniqueName: \"kubernetes.io/projected/0195c259-8934-4087-8e99-353b607f030e-kube-api-access-242gf\") pod \"0195c259-8934-4087-8e99-353b607f030e\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") "
Apr 28 19:21:23.688046 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.687974    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0195c259-8934-4087-8e99-353b607f030e-metrics-client-ca\") pod \"0195c259-8934-4087-8e99-353b607f030e\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") "
Apr 28 19:21:23.688046 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.687993    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0195c259-8934-4087-8e99-353b607f030e-config-out\") pod \"0195c259-8934-4087-8e99-353b607f030e\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") "
Apr 28 19:21:23.688046 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688012    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0195c259-8934-4087-8e99-353b607f030e-alertmanager-trusted-ca-bundle\") pod \"0195c259-8934-4087-8e99-353b607f030e\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") "
Apr 28 19:21:23.688046 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688034    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-config-volume\") pod \"0195c259-8934-4087-8e99-353b607f030e\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") "
Apr 28 19:21:23.688296 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688081    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-web-config\") pod \"0195c259-8934-4087-8e99-353b607f030e\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") "
Apr 28 19:21:23.688374 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688352    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy\") pod \"0195c259-8934-4087-8e99-353b607f030e\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") "
Apr 28 19:21:23.688455 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688418    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-main-tls\") pod \"0195c259-8934-4087-8e99-353b607f030e\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") "
Apr 28 19:21:23.688512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688352    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0195c259-8934-4087-8e99-353b607f030e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "0195c259-8934-4087-8e99-353b607f030e" (UID: "0195c259-8934-4087-8e99-353b607f030e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:21:23.688512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688445    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0195c259-8934-4087-8e99-353b607f030e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "0195c259-8934-4087-8e99-353b607f030e" (UID: "0195c259-8934-4087-8e99-353b607f030e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:21:23.688512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688472    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-cluster-tls-config\") pod \"0195c259-8934-4087-8e99-353b607f030e\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") "
Apr 28 19:21:23.688673 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688525    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0195c259-8934-4087-8e99-353b607f030e-alertmanager-main-db\") pod \"0195c259-8934-4087-8e99-353b607f030e\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") "
Apr 28 19:21:23.688673 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688558    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"0195c259-8934-4087-8e99-353b607f030e\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") "
Apr 28 19:21:23.688673 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688587    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0195c259-8934-4087-8e99-353b607f030e-tls-assets\") pod \"0195c259-8934-4087-8e99-353b607f030e\" (UID: \"0195c259-8934-4087-8e99-353b607f030e\") "
Apr 28 19:21:23.689181 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688867    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0195c259-8934-4087-8e99-353b607f030e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "0195c259-8934-4087-8e99-353b607f030e" (UID: "0195c259-8934-4087-8e99-353b607f030e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:21:23.689181 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688969    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0195c259-8934-4087-8e99-353b607f030e-alertmanager-main-db\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:21:23.689181 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.688990    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0195c259-8934-4087-8e99-353b607f030e-metrics-client-ca\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:21:23.689181 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.689007    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0195c259-8934-4087-8e99-353b607f030e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:21:23.691593 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.691255    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "0195c259-8934-4087-8e99-353b607f030e" (UID: "0195c259-8934-4087-8e99-353b607f030e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:21:23.691593 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.691553    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0195c259-8934-4087-8e99-353b607f030e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0195c259-8934-4087-8e99-353b607f030e" (UID: "0195c259-8934-4087-8e99-353b607f030e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:21:23.691744 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.691642    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-config-volume" (OuterVolumeSpecName: "config-volume") pod "0195c259-8934-4087-8e99-353b607f030e" (UID: "0195c259-8934-4087-8e99-353b607f030e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:21:23.691802 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.691783    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0195c259-8934-4087-8e99-353b607f030e-config-out" (OuterVolumeSpecName: "config-out") pod "0195c259-8934-4087-8e99-353b607f030e" (UID: "0195c259-8934-4087-8e99-353b607f030e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:21:23.691857 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.691796    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "0195c259-8934-4087-8e99-353b607f030e" (UID: "0195c259-8934-4087-8e99-353b607f030e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:21:23.691960 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.691938    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "0195c259-8934-4087-8e99-353b607f030e" (UID: "0195c259-8934-4087-8e99-353b607f030e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:21:23.692232 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.692211    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0195c259-8934-4087-8e99-353b607f030e-kube-api-access-242gf" (OuterVolumeSpecName: "kube-api-access-242gf") pod "0195c259-8934-4087-8e99-353b607f030e" (UID: "0195c259-8934-4087-8e99-353b607f030e"). InnerVolumeSpecName "kube-api-access-242gf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:21:23.692499 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.692475    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "0195c259-8934-4087-8e99-353b607f030e" (UID: "0195c259-8934-4087-8e99-353b607f030e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:21:23.696028 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.695793    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "0195c259-8934-4087-8e99-353b607f030e" (UID: "0195c259-8934-4087-8e99-353b607f030e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:21:23.703502 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.703477    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-web-config" (OuterVolumeSpecName: "web-config") pod "0195c259-8934-4087-8e99-353b607f030e" (UID: "0195c259-8934-4087-8e99-353b607f030e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:21:23.789651 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.789558    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-web-config\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:21:23.789651 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.789593    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:21:23.789651 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.789608    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-main-tls\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:21:23.789651 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.789621    2569 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-cluster-tls-config\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:21:23.789651 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.789636    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:21:23.789651 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.789650    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0195c259-8934-4087-8e99-353b607f030e-tls-assets\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:21:23.789938 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.789663    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:21:23.789938 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.789675    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-242gf\" (UniqueName: \"kubernetes.io/projected/0195c259-8934-4087-8e99-353b607f030e-kube-api-access-242gf\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:21:23.789938 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.789687    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0195c259-8934-4087-8e99-353b607f030e-config-out\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:21:23.789938 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.789698    2569 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0195c259-8934-4087-8e99-353b607f030e-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:21:23.834331 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.834303    2569 generic.go:358] "Generic (PLEG): container finished" podID="0195c259-8934-4087-8e99-353b607f030e" containerID="2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931" exitCode=0
Apr 28 19:21:23.834331 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.834328    2569 generic.go:358] "Generic (PLEG): container finished" podID="0195c259-8934-4087-8e99-353b607f030e" containerID="417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe" exitCode=0
Apr 28 19:21:23.834505 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.834348    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerDied","Data":"2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931"}
Apr 28 19:21:23.834505 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.834369    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerDied","Data":"417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe"}
Apr 28 19:21:23.834505 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.834379    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0195c259-8934-4087-8e99-353b607f030e","Type":"ContainerDied","Data":"d2aa409451ea58f1a13296f758da91ea66490ab4ab485ac7a4c8812cb3de5051"}
Apr 28 19:21:23.834505 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.834393    2569 scope.go:117] "RemoveContainer" containerID="6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc"
Apr 28 19:21:23.834505 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.834428    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.844804 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.843115    2569 scope.go:117] "RemoveContainer" containerID="2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931"
Apr 28 19:21:23.850113 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.850095    2569 scope.go:117] "RemoveContainer" containerID="45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151"
Apr 28 19:21:23.856303 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.856285    2569 scope.go:117] "RemoveContainer" containerID="417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe"
Apr 28 19:21:23.862848 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.862829    2569 scope.go:117] "RemoveContainer" containerID="409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105"
Apr 28 19:21:23.862973 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.862953    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Apr 28 19:21:23.867912 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.867892    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Apr 28 19:21:23.870495 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.870476    2569 scope.go:117] "RemoveContainer" containerID="2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23"
Apr 28 19:21:23.876662 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.876644    2569 scope.go:117] "RemoveContainer" containerID="e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be"
Apr 28 19:21:23.882517 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.882497    2569 scope.go:117] "RemoveContainer" containerID="6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc"
Apr 28 19:21:23.882730 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:21:23.882714    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc\": container with ID starting with 6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc not found: ID does not exist" containerID="6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc"
Apr 28 19:21:23.882772 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.882737    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc"} err="failed to get container status \"6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc\": rpc error: code = NotFound desc = could not find container \"6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc\": container with ID starting with 6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc not found: ID does not exist"
Apr 28 19:21:23.882772 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.882753    2569 scope.go:117] "RemoveContainer" containerID="2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931"
Apr 28 19:21:23.882963 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:21:23.882947    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931\": container with ID starting with 2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931 not found: ID does not exist" containerID="2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931"
Apr 28 19:21:23.882998 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.882970    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931"} err="failed to get container status \"2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931\": rpc error: code = NotFound desc = could not find container \"2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931\": container with ID starting with 2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931 not found: ID does not exist"
Apr 28 19:21:23.882998 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.882993    2569 scope.go:117] "RemoveContainer" containerID="45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151"
Apr 28 19:21:23.883181 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:21:23.883167    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151\": container with ID starting with 45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151 not found: ID does not exist" containerID="45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151"
Apr 28 19:21:23.883221 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.883187    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151"} err="failed to get container status \"45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151\": rpc error: code = NotFound desc = could not find container \"45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151\": container with ID starting with 45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151 not found: ID does not exist"
Apr 28 19:21:23.883221 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.883199    2569 scope.go:117] "RemoveContainer" containerID="417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe"
Apr 28 19:21:23.883380 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:21:23.883366    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe\": container with ID starting with 417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe not found: ID does not exist" containerID="417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe"
Apr 28 19:21:23.883451 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.883384    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe"} err="failed to get container status \"417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe\": rpc error: code = NotFound desc = could not find container \"417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe\": container with ID starting with 417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe not found: ID does not exist"
Apr 28 19:21:23.883451 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.883441    2569 scope.go:117] "RemoveContainer" containerID="409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105"
Apr 28 19:21:23.883691 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:21:23.883676    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105\": container with ID starting with 409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105 not found: ID does not exist" containerID="409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105"
Apr 28 19:21:23.883728 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.883694    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105"} err="failed to get container status \"409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105\": rpc error: code = NotFound desc = could not find container \"409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105\": container with ID starting with 409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105 not found: ID does not exist"
Apr 28 19:21:23.883728 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.883706    2569 scope.go:117] "RemoveContainer" containerID="2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23"
Apr 28 19:21:23.883912 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:21:23.883883    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23\": container with ID starting with 2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23 not found: ID does not exist" containerID="2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23"
Apr 28 19:21:23.883953 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.883915    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23"} err="failed to get container status \"2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23\": rpc error: code = NotFound desc = could not find container \"2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23\": container with ID starting with 2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23 not found: ID does not exist"
Apr 28 19:21:23.883953 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.883928    2569 scope.go:117] "RemoveContainer" containerID="e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be"
Apr 28 19:21:23.884154 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:21:23.884132    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be\": container with ID starting with e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be not found: ID does not exist" containerID="e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be"
Apr 28 19:21:23.884200 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.884160    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be"} err="failed to get container status \"e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be\": rpc error: code = NotFound desc = could not find container \"e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be\": container with ID starting with e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be not found: ID does not exist"
Apr 28 19:21:23.884200 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.884175    2569 scope.go:117] "RemoveContainer" containerID="6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc"
Apr 28 19:21:23.884360 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.884345    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc"} err="failed to get container status \"6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc\": rpc error: code = NotFound desc = could not find container \"6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc\": container with ID starting with 6868618f0f41d952085fb643a35e4c9576ae0e32526e88359c3b5f7c80e331dc not found: ID does not exist"
Apr 28 19:21:23.884428 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.884360    2569 scope.go:117] "RemoveContainer" containerID="2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931"
Apr 28 19:21:23.884645 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.884628    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931"} err="failed to get container status \"2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931\": rpc error: code = NotFound desc = could not find container \"2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931\": container with ID starting with 2bd75341bda0b5147c9c91b30dd24a2f89f3d805299b9c84b598499b73f57931 not found: ID does not exist"
Apr 28 19:21:23.884645 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.884645    2569 scope.go:117] "RemoveContainer" containerID="45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151"
Apr 28 19:21:23.884857 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.884840    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151"} err="failed to get container status \"45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151\": rpc error: code = NotFound desc = could not find container \"45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151\": container with ID starting with 45c7b80e47b1d470ebea1b638abdd262c554be0d28cf6ae3f36f4f5bad5fe151 not found: ID does not exist"
Apr 28 19:21:23.884857 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.884855    2569 scope.go:117] "RemoveContainer" containerID="417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe"
Apr 28 19:21:23.885110 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.885091    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe"} err="failed to get container status \"417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe\": rpc error: code = NotFound desc = could not find container \"417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe\": container with ID starting with 417e233b2370d2d7e0b4437efa9a3dbac5ec9590fa3dd0d14aa527c61c912fbe not found: ID does not exist"
Apr 28 19:21:23.885110 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.885109    2569 scope.go:117] "RemoveContainer" containerID="409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105"
Apr 28 19:21:23.885341 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.885321    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105"} err="failed to get container status \"409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105\": rpc error: code = NotFound desc = could not find container \"409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105\": container with ID starting with 409fe22017949cf575e4b11c61c3e147e9019651cae7ef8e659063a7e5256105 not found: ID does not exist"
Apr 28 19:21:23.885385 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.885341    2569 scope.go:117] "RemoveContainer" containerID="2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23"
Apr 28 19:21:23.885581 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.885564    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23"} err="failed to get container status \"2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23\": rpc error: code = NotFound desc = could not find container \"2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23\": container with ID starting with 2bf5a3f1f50cc1322e2617ea1e3f5db12b2934992109d8182f559380bfe79d23 not found: ID does not exist"
Apr 28 19:21:23.885627 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.885582    2569 scope.go:117] "RemoveContainer" containerID="e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be"
Apr 28 19:21:23.885777 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.885761    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be"} err="failed to get container status \"e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be\": rpc error: code = NotFound desc = could not find container \"e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be\": container with ID starting with e53eee52e4515b62acd944479a8c8d788e9621754b1e946b23b7b55c61fa29be not found: ID does not exist"
Apr 28 19:21:23.894220 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894201    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Apr 28 19:21:23.894512 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894499    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="kube-rbac-proxy"
Apr 28 19:21:23.894564 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894514    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="kube-rbac-proxy"
Apr 28 19:21:23.894564 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894522    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="kube-rbac-proxy-metric"
Apr 28 19:21:23.894564 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894527    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="kube-rbac-proxy-metric"
Apr 28 19:21:23.894564 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894537    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="kube-rbac-proxy-web"
Apr 28 19:21:23.894564 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894543    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="kube-rbac-proxy-web"
Apr 28 19:21:23.894564 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894552    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a216f6e-4215-47a2-bfa3-2e0e5868e25c" containerName="console"
Apr 28 19:21:23.894564 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894559    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a216f6e-4215-47a2-bfa3-2e0e5868e25c" containerName="console"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894572    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4754506-9f06-45d8-bf0d-1643f89dbc37" containerName="console"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894578    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4754506-9f06-45d8-bf0d-1643f89dbc37" containerName="console"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894587    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="init-config-reloader"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894592    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="init-config-reloader"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894600    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="prom-label-proxy"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894605    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="prom-label-proxy"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894612    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="config-reloader"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894617    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="config-reloader"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894623    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="alertmanager"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894628    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="alertmanager"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894672    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="alertmanager"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894681    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="kube-rbac-proxy"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894689    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="d4754506-9f06-45d8-bf0d-1643f89dbc37" containerName="console"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894694    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="config-reloader"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894700    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="kube-rbac-proxy-metric"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894705    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="kube-rbac-proxy-web"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894713    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="0a216f6e-4215-47a2-bfa3-2e0e5868e25c" containerName="console"
Apr 28 19:21:23.894758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.894722    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="0195c259-8934-4087-8e99-353b607f030e" containerName="prom-label-proxy"
Apr 28 19:21:23.897917 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.897904    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.902332 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.902313    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Apr 28 19:21:23.902448 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.902313    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Apr 28 19:21:23.902448 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.902313    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Apr 28 19:21:23.902604 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.902588    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Apr 28 19:21:23.902923 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.902904    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Apr 28 19:21:23.903243 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.903220    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Apr 28 19:21:23.903300 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.903245    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4btj6\""
Apr 28 19:21:23.903300 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.903231    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Apr 28 19:21:23.903415 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.903317    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Apr 28 19:21:23.909606 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.909590    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Apr 28 19:21:23.917021 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.916993    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Apr 28 19:21:23.941739 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.941711    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0195c259-8934-4087-8e99-353b607f030e" path="/var/lib/kubelet/pods/0195c259-8934-4087-8e99-353b607f030e/volumes"
Apr 28 19:21:23.991021 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.990999    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.991136 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.991046    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4t6c\" (UniqueName: \"kubernetes.io/projected/52f98b6c-1246-4518-b036-c837710fa2cf-kube-api-access-w4t6c\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.991198 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.991130    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-web-config\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.991198 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.991159    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/52f98b6c-1246-4518-b036-c837710fa2cf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.991198 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.991183    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52f98b6c-1246-4518-b036-c837710fa2cf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.991332 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.991310    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/52f98b6c-1246-4518-b036-c837710fa2cf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.991372 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.991349    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.991428 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.991375    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.991428 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.991395    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.991498 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.991473    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/52f98b6c-1246-4518-b036-c837710fa2cf-config-out\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.991582 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.991495    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.991582 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.991514    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52f98b6c-1246-4518-b036-c837710fa2cf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:23.991582 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:23.991562    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-config-volume\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.091891 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.091830    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/52f98b6c-1246-4518-b036-c837710fa2cf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.091891 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.091863    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.092038 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.091892    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.092038 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.092004    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.092130 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.092040    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/52f98b6c-1246-4518-b036-c837710fa2cf-config-out\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.092130 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.092061    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.092130 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.092082    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52f98b6c-1246-4518-b036-c837710fa2cf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.092130 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.092117    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-config-volume\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.092350 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.092148    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.092350 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.092173    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w4t6c\" (UniqueName: \"kubernetes.io/projected/52f98b6c-1246-4518-b036-c837710fa2cf-kube-api-access-w4t6c\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.092350 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.092216    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-web-config\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.092350 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.092241    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/52f98b6c-1246-4518-b036-c837710fa2cf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.092350 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.092265    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52f98b6c-1246-4518-b036-c837710fa2cf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.093013 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.092986    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52f98b6c-1246-4518-b036-c837710fa2cf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.094917 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.094888    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/52f98b6c-1246-4518-b036-c837710fa2cf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.094917 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.094907    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.095073 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.094968    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.095073 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.095064    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.095185 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.095099    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.095185 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.095165    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.095277 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.095258    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/52f98b6c-1246-4518-b036-c837710fa2cf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.095334 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.095309    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/52f98b6c-1246-4518-b036-c837710fa2cf-config-out\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.095689 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.095666    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52f98b6c-1246-4518-b036-c837710fa2cf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.095773 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.095704    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-web-config\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.096757 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.096740    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/52f98b6c-1246-4518-b036-c837710fa2cf-config-volume\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.102446 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.102432    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4t6c\" (UniqueName: \"kubernetes.io/projected/52f98b6c-1246-4518-b036-c837710fa2cf-kube-api-access-w4t6c\") pod \"alertmanager-main-0\" (UID: \"52f98b6c-1246-4518-b036-c837710fa2cf\") " pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.206696 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.206668    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Apr 28 19:21:24.345148 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.345113    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Apr 28 19:21:24.347444 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:21:24.347419    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f98b6c_1246_4518_b036_c837710fa2cf.slice/crio-447cb11147de4fde015348055c238e7ac8e1c6e803e743554194c6adc7b429d3 WatchSource:0}: Error finding container 447cb11147de4fde015348055c238e7ac8e1c6e803e743554194c6adc7b429d3: Status 404 returned error can't find the container with id 447cb11147de4fde015348055c238e7ac8e1c6e803e743554194c6adc7b429d3
Apr 28 19:21:24.838289 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.838256    2569 generic.go:358] "Generic (PLEG): container finished" podID="52f98b6c-1246-4518-b036-c837710fa2cf" containerID="fec4e74643bd1f8c57879ff14d983000acd9da81e0a590baa6c15216c2c74377" exitCode=0
Apr 28 19:21:24.838724 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.838348    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52f98b6c-1246-4518-b036-c837710fa2cf","Type":"ContainerDied","Data":"fec4e74643bd1f8c57879ff14d983000acd9da81e0a590baa6c15216c2c74377"}
Apr 28 19:21:24.838724 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:24.838384    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52f98b6c-1246-4518-b036-c837710fa2cf","Type":"ContainerStarted","Data":"447cb11147de4fde015348055c238e7ac8e1c6e803e743554194c6adc7b429d3"}
Apr 28 19:21:25.845373 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:25.845336    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52f98b6c-1246-4518-b036-c837710fa2cf","Type":"ContainerStarted","Data":"b2a28cdbdf0e7eed9240da78d6457fc858c9626c942b2b4ec1eab018469e9a73"}
Apr 28 19:21:25.845373 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:25.845372    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52f98b6c-1246-4518-b036-c837710fa2cf","Type":"ContainerStarted","Data":"1b3fc6acc950facc89c9fc40a09dca4d56e14286771c6ad6990b000e15e9c14f"}
Apr 28 19:21:25.845851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:25.845384    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52f98b6c-1246-4518-b036-c837710fa2cf","Type":"ContainerStarted","Data":"76046186a3434a4b4b08243d55d1c00991d8cfdc14537b6f845ca73a03617c92"}
Apr 28 19:21:25.845851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:25.845418    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52f98b6c-1246-4518-b036-c837710fa2cf","Type":"ContainerStarted","Data":"009eecb64e6e42bbc249ceaeee4f16d108a6fe10e1b09582598a0ff0bb12a0e1"}
Apr 28 19:21:25.845851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:25.845430    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52f98b6c-1246-4518-b036-c837710fa2cf","Type":"ContainerStarted","Data":"974dabead96898b8bfc7eef5ea96578bdb40708ea09c8171a57747e62f0ffe26"}
Apr 28 19:21:25.845851 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:25.845441    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"52f98b6c-1246-4518-b036-c837710fa2cf","Type":"ContainerStarted","Data":"46615e11e68eadef7298a32eb69cd25a32fc3a553f034f62de16fa6b48bde22d"}
Apr 28 19:21:25.879815 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:25.879768    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.879754251 podStartE2EDuration="2.879754251s" podCreationTimestamp="2026-04-28 19:21:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:21:25.878260407 +0000 UTC m=+272.702600515" watchObservedRunningTime="2026-04-28 19:21:25.879754251 +0000 UTC m=+272.704094358"
Apr 28 19:21:26.421803 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.421769    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-8454c5f95f-gwr52"]
Apr 28 19:21:26.423989 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.423970    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.442950 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.442924    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8454c5f95f-gwr52"]
Apr 28 19:21:26.514385 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.514353    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-service-ca\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.514385 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.514384    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-oauth-config\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.514553 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.514420    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ng42\" (UniqueName: \"kubernetes.io/projected/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-kube-api-access-6ng42\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.514553 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.514491    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-serving-cert\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.514553 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.514521    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-oauth-serving-cert\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.514650 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.514551    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-config\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.514650 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.514611    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-trusted-ca-bundle\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.615694 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.615664    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-config\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.615838 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.615712    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-trusted-ca-bundle\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.615838 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.615775    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-service-ca\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.615838 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.615799    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-oauth-config\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.615838 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.615823    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6ng42\" (UniqueName: \"kubernetes.io/projected/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-kube-api-access-6ng42\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.616036 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.615854    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-serving-cert\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.616036 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.615883    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-oauth-serving-cert\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.616474 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.616448    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-config\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.616582 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.616448    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-service-ca\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.616582 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.616541    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-oauth-serving-cert\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.616856 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.616834    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-trusted-ca-bundle\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.618452 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.618431    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-serving-cert\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.618734 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.618716    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-oauth-config\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.644455 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.644431    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ng42\" (UniqueName: \"kubernetes.io/projected/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-kube-api-access-6ng42\") pod \"console-8454c5f95f-gwr52\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") " pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.733331 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.733298    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:26.859374 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:26.859326    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8454c5f95f-gwr52"]
Apr 28 19:21:26.863141 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:21:26.863117    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod074c49c6_ef56_4a60_a1c8_94a8538f3ce0.slice/crio-39f8e361e4ec66a3fc0ab545cee8dd5b6f76216a2f05db5b501bdb3686e07695 WatchSource:0}: Error finding container 39f8e361e4ec66a3fc0ab545cee8dd5b6f76216a2f05db5b501bdb3686e07695: Status 404 returned error can't find the container with id 39f8e361e4ec66a3fc0ab545cee8dd5b6f76216a2f05db5b501bdb3686e07695
Apr 28 19:21:27.855683 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:27.855647    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8454c5f95f-gwr52" event={"ID":"074c49c6-ef56-4a60-a1c8-94a8538f3ce0","Type":"ContainerStarted","Data":"0cc871a626f3bbd566080ebeb0fd743487468ac9ec11f0747a57f423cdf07a98"}
Apr 28 19:21:27.855683 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:27.855688    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8454c5f95f-gwr52" event={"ID":"074c49c6-ef56-4a60-a1c8-94a8538f3ce0","Type":"ContainerStarted","Data":"39f8e361e4ec66a3fc0ab545cee8dd5b6f76216a2f05db5b501bdb3686e07695"}
Apr 28 19:21:27.876898 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:27.876853    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-8454c5f95f-gwr52" podStartSLOduration=1.8768391960000002 podStartE2EDuration="1.876839196s" podCreationTimestamp="2026-04-28 19:21:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:21:27.876017468 +0000 UTC m=+274.700357575" watchObservedRunningTime="2026-04-28 19:21:27.876839196 +0000 UTC m=+274.701179303"
Apr 28 19:21:36.733856 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:36.733818    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:36.734247 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:36.733875    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:36.738510 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:36.738485    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:36.885733 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:36.885708    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:21:36.929500 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:36.929473    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-648ffdfc7d-gcz5f"]
Apr 28 19:21:53.815581 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:53.815549    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:21:53.816334 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:53.816309    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:21:53.819046 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:53.819022    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:21:53.819713 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:53.819692    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:21:53.825488 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:21:53.825471    2569 kubelet.go:1628] "Image garbage collection succeeded"
Apr 28 19:22:01.948766 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:01.948712    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-648ffdfc7d-gcz5f" podUID="6ff85e51-6ce9-4350-bfa8-d7d0b10e1574" containerName="console" containerID="cri-o://843218273d67468710a08ee487a3c9f1230fa59a19e12a7e8836231a3dda78c8" gracePeriod=15
Apr 28 19:22:02.196696 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.196667    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-648ffdfc7d-gcz5f_6ff85e51-6ce9-4350-bfa8-d7d0b10e1574/console/0.log"
Apr 28 19:22:02.196813 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.196727    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:22:02.202723 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.202677    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-config\") pod \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") "
Apr 28 19:22:02.202723 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.202709    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-oauth-config\") pod \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") "
Apr 28 19:22:02.203070 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.202734    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-oauth-serving-cert\") pod \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") "
Apr 28 19:22:02.203070 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.202827    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-serving-cert\") pod \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") "
Apr 28 19:22:02.203070 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.202855    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-trusted-ca-bundle\") pod \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") "
Apr 28 19:22:02.203070 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.202888    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dwxr\" (UniqueName: \"kubernetes.io/projected/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-kube-api-access-9dwxr\") pod \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") "
Apr 28 19:22:02.203070 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.202927    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-service-ca\") pod \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\" (UID: \"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574\") "
Apr 28 19:22:02.203337 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.203079    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-config" (OuterVolumeSpecName: "console-config") pod "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574" (UID: "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:22:02.203337 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.203221    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574" (UID: "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:22:02.203337 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.203318    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574" (UID: "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:22:02.203533 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.203295    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-config\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:22:02.203533 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.203392    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-service-ca" (OuterVolumeSpecName: "service-ca") pod "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574" (UID: "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:22:02.204866 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.204839    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574" (UID: "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:22:02.205017 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.204990    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574" (UID: "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:22:02.205017 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.205004    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-kube-api-access-9dwxr" (OuterVolumeSpecName: "kube-api-access-9dwxr") pod "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574" (UID: "6ff85e51-6ce9-4350-bfa8-d7d0b10e1574"). InnerVolumeSpecName "kube-api-access-9dwxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:22:02.304315 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.304285    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-serving-cert\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:22:02.304315 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.304310    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-trusted-ca-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:22:02.304315 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.304321    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dwxr\" (UniqueName: \"kubernetes.io/projected/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-kube-api-access-9dwxr\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:22:02.304566 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.304330    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-service-ca\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:22:02.304566 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.304339    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-console-oauth-config\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:22:02.304566 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.304347    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574-oauth-serving-cert\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:22:02.955906 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.955874    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-648ffdfc7d-gcz5f_6ff85e51-6ce9-4350-bfa8-d7d0b10e1574/console/0.log"
Apr 28 19:22:02.956329 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.955921    2569 generic.go:358] "Generic (PLEG): container finished" podID="6ff85e51-6ce9-4350-bfa8-d7d0b10e1574" containerID="843218273d67468710a08ee487a3c9f1230fa59a19e12a7e8836231a3dda78c8" exitCode=2
Apr 28 19:22:02.956329 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.955979    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648ffdfc7d-gcz5f"
Apr 28 19:22:02.956329 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.956008    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648ffdfc7d-gcz5f" event={"ID":"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574","Type":"ContainerDied","Data":"843218273d67468710a08ee487a3c9f1230fa59a19e12a7e8836231a3dda78c8"}
Apr 28 19:22:02.956329 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.956043    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648ffdfc7d-gcz5f" event={"ID":"6ff85e51-6ce9-4350-bfa8-d7d0b10e1574","Type":"ContainerDied","Data":"61df176cdce65a05e7ed1c4d70a49f557919956408a21ba5baaa5ecf0a22bfa9"}
Apr 28 19:22:02.956329 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.956066    2569 scope.go:117] "RemoveContainer" containerID="843218273d67468710a08ee487a3c9f1230fa59a19e12a7e8836231a3dda78c8"
Apr 28 19:22:02.964352 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.964338    2569 scope.go:117] "RemoveContainer" containerID="843218273d67468710a08ee487a3c9f1230fa59a19e12a7e8836231a3dda78c8"
Apr 28 19:22:02.964618 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:22:02.964597    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"843218273d67468710a08ee487a3c9f1230fa59a19e12a7e8836231a3dda78c8\": container with ID starting with 843218273d67468710a08ee487a3c9f1230fa59a19e12a7e8836231a3dda78c8 not found: ID does not exist" containerID="843218273d67468710a08ee487a3c9f1230fa59a19e12a7e8836231a3dda78c8"
Apr 28 19:22:02.964686 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.964624    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"843218273d67468710a08ee487a3c9f1230fa59a19e12a7e8836231a3dda78c8"} err="failed to get container status \"843218273d67468710a08ee487a3c9f1230fa59a19e12a7e8836231a3dda78c8\": rpc error: code = NotFound desc = could not find container \"843218273d67468710a08ee487a3c9f1230fa59a19e12a7e8836231a3dda78c8\": container with ID starting with 843218273d67468710a08ee487a3c9f1230fa59a19e12a7e8836231a3dda78c8 not found: ID does not exist"
Apr 28 19:22:02.980719 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.980688    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-648ffdfc7d-gcz5f"]
Apr 28 19:22:02.987214 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:02.987193    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-648ffdfc7d-gcz5f"]
Apr 28 19:22:03.943569 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:22:03.943536    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ff85e51-6ce9-4350-bfa8-d7d0b10e1574" path="/var/lib/kubelet/pods/6ff85e51-6ce9-4350-bfa8-d7d0b10e1574/volumes"
Apr 28 19:23:04.454964 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.454928    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"]
Apr 28 19:23:04.455441 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.455238    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff85e51-6ce9-4350-bfa8-d7d0b10e1574" containerName="console"
Apr 28 19:23:04.455441 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.455249    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff85e51-6ce9-4350-bfa8-d7d0b10e1574" containerName="console"
Apr 28 19:23:04.455441 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.455313    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="6ff85e51-6ce9-4350-bfa8-d7d0b10e1574" containerName="console"
Apr 28 19:23:04.459528 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.459510    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"
Apr 28 19:23:04.462560 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.462538    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nvrrr\""
Apr 28 19:23:04.462671 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.462539    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 28 19:23:04.463647 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.463634    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 28 19:23:04.467885 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.467863    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"]
Apr 28 19:23:04.509328 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.509306    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmn49\" (UniqueName: \"kubernetes.io/projected/a8ed5083-9951-44bc-957e-62605befa725-kube-api-access-pmn49\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd\" (UID: \"a8ed5083-9951-44bc-957e-62605befa725\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"
Apr 28 19:23:04.509473 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.509340    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8ed5083-9951-44bc-957e-62605befa725-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd\" (UID: \"a8ed5083-9951-44bc-957e-62605befa725\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"
Apr 28 19:23:04.509542 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.509475    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8ed5083-9951-44bc-957e-62605befa725-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd\" (UID: \"a8ed5083-9951-44bc-957e-62605befa725\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"
Apr 28 19:23:04.610104 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.610082    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8ed5083-9951-44bc-957e-62605befa725-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd\" (UID: \"a8ed5083-9951-44bc-957e-62605befa725\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"
Apr 28 19:23:04.610210 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.610125    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pmn49\" (UniqueName: \"kubernetes.io/projected/a8ed5083-9951-44bc-957e-62605befa725-kube-api-access-pmn49\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd\" (UID: \"a8ed5083-9951-44bc-957e-62605befa725\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"
Apr 28 19:23:04.610210 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.610145    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8ed5083-9951-44bc-957e-62605befa725-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd\" (UID: \"a8ed5083-9951-44bc-957e-62605befa725\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"
Apr 28 19:23:04.610445 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.610431    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8ed5083-9951-44bc-957e-62605befa725-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd\" (UID: \"a8ed5083-9951-44bc-957e-62605befa725\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"
Apr 28 19:23:04.610518 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.610501    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8ed5083-9951-44bc-957e-62605befa725-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd\" (UID: \"a8ed5083-9951-44bc-957e-62605befa725\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"
Apr 28 19:23:04.618861 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.618843    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmn49\" (UniqueName: \"kubernetes.io/projected/a8ed5083-9951-44bc-957e-62605befa725-kube-api-access-pmn49\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd\" (UID: \"a8ed5083-9951-44bc-957e-62605befa725\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"
Apr 28 19:23:04.770034 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.769954    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"
Apr 28 19:23:04.892101 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.892078    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"]
Apr 28 19:23:04.894419 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:23:04.894375    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ed5083_9951_44bc_957e_62605befa725.slice/crio-59cc38d4a28e5648c7c58424040cabe0a6682a80a8700a389e31c90755b5852d WatchSource:0}: Error finding container 59cc38d4a28e5648c7c58424040cabe0a6682a80a8700a389e31c90755b5852d: Status 404 returned error can't find the container with id 59cc38d4a28e5648c7c58424040cabe0a6682a80a8700a389e31c90755b5852d
Apr 28 19:23:04.896259 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:04.896243    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 28 19:23:05.131127 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:05.131038    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd" event={"ID":"a8ed5083-9951-44bc-957e-62605befa725","Type":"ContainerStarted","Data":"59cc38d4a28e5648c7c58424040cabe0a6682a80a8700a389e31c90755b5852d"}
Apr 28 19:23:12.154421 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:12.154321    2569 generic.go:358] "Generic (PLEG): container finished" podID="a8ed5083-9951-44bc-957e-62605befa725" containerID="7411cd89ce46e18d3f9ceeda7da7ff3ebb73285f63f445c351587ea3a469ad57" exitCode=0
Apr 28 19:23:12.154784 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:12.154423    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd" event={"ID":"a8ed5083-9951-44bc-957e-62605befa725","Type":"ContainerDied","Data":"7411cd89ce46e18d3f9ceeda7da7ff3ebb73285f63f445c351587ea3a469ad57"}
Apr 28 19:23:15.164551 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:15.164516    2569 generic.go:358] "Generic (PLEG): container finished" podID="a8ed5083-9951-44bc-957e-62605befa725" containerID="651741754dcf9d15cd3db925c737674b0c6423e8fde2749be9b384b8bf307ee3" exitCode=0
Apr 28 19:23:15.165015 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:15.164594    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd" event={"ID":"a8ed5083-9951-44bc-957e-62605befa725","Type":"ContainerDied","Data":"651741754dcf9d15cd3db925c737674b0c6423e8fde2749be9b384b8bf307ee3"}
Apr 28 19:23:22.188263 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:22.188222    2569 generic.go:358] "Generic (PLEG): container finished" podID="a8ed5083-9951-44bc-957e-62605befa725" containerID="5bab661f2b1a662f0dc938bb444c84a1ad7f148203bff2fd56e2a405f5c07d7b" exitCode=0
Apr 28 19:23:22.188846 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:22.188272    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd" event={"ID":"a8ed5083-9951-44bc-957e-62605befa725","Type":"ContainerDied","Data":"5bab661f2b1a662f0dc938bb444c84a1ad7f148203bff2fd56e2a405f5c07d7b"}
Apr 28 19:23:23.307895 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:23.307872    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"
Apr 28 19:23:23.373824 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:23.373792    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmn49\" (UniqueName: \"kubernetes.io/projected/a8ed5083-9951-44bc-957e-62605befa725-kube-api-access-pmn49\") pod \"a8ed5083-9951-44bc-957e-62605befa725\" (UID: \"a8ed5083-9951-44bc-957e-62605befa725\") "
Apr 28 19:23:23.373996 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:23.373894    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8ed5083-9951-44bc-957e-62605befa725-util\") pod \"a8ed5083-9951-44bc-957e-62605befa725\" (UID: \"a8ed5083-9951-44bc-957e-62605befa725\") "
Apr 28 19:23:23.373996 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:23.373976    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8ed5083-9951-44bc-957e-62605befa725-bundle\") pod \"a8ed5083-9951-44bc-957e-62605befa725\" (UID: \"a8ed5083-9951-44bc-957e-62605befa725\") "
Apr 28 19:23:23.374540 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:23.374502    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ed5083-9951-44bc-957e-62605befa725-bundle" (OuterVolumeSpecName: "bundle") pod "a8ed5083-9951-44bc-957e-62605befa725" (UID: "a8ed5083-9951-44bc-957e-62605befa725"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:23:23.376032 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:23.376007    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8ed5083-9951-44bc-957e-62605befa725-kube-api-access-pmn49" (OuterVolumeSpecName: "kube-api-access-pmn49") pod "a8ed5083-9951-44bc-957e-62605befa725" (UID: "a8ed5083-9951-44bc-957e-62605befa725"). InnerVolumeSpecName "kube-api-access-pmn49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:23:23.379174 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:23.379143    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ed5083-9951-44bc-957e-62605befa725-util" (OuterVolumeSpecName: "util") pod "a8ed5083-9951-44bc-957e-62605befa725" (UID: "a8ed5083-9951-44bc-957e-62605befa725"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:23:23.474711 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:23.474655    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmn49\" (UniqueName: \"kubernetes.io/projected/a8ed5083-9951-44bc-957e-62605befa725-kube-api-access-pmn49\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:23:23.474711 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:23.474679    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8ed5083-9951-44bc-957e-62605befa725-util\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:23:23.474711 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:23.474689    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8ed5083-9951-44bc-957e-62605befa725-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:23:24.194218 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:24.194185    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd" event={"ID":"a8ed5083-9951-44bc-957e-62605befa725","Type":"ContainerDied","Data":"59cc38d4a28e5648c7c58424040cabe0a6682a80a8700a389e31c90755b5852d"}
Apr 28 19:23:24.194218 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:24.194215    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59cc38d4a28e5648c7c58424040cabe0a6682a80a8700a389e31c90755b5852d"
Apr 28 19:23:24.194467 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:24.194221    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c484hd"
Apr 28 19:23:26.656105 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.656068    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p"]
Apr 28 19:23:26.656590 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.656535    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8ed5083-9951-44bc-957e-62605befa725" containerName="util"
Apr 28 19:23:26.656590 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.656553    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8ed5083-9951-44bc-957e-62605befa725" containerName="util"
Apr 28 19:23:26.656590 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.656572    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8ed5083-9951-44bc-957e-62605befa725" containerName="pull"
Apr 28 19:23:26.656590 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.656579    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8ed5083-9951-44bc-957e-62605befa725" containerName="pull"
Apr 28 19:23:26.656869 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.656609    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8ed5083-9951-44bc-957e-62605befa725" containerName="extract"
Apr 28 19:23:26.656869 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.656617    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8ed5083-9951-44bc-957e-62605befa725" containerName="extract"
Apr 28 19:23:26.656869 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.656695    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="a8ed5083-9951-44bc-957e-62605befa725" containerName="extract"
Apr 28 19:23:26.707153 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.707121    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p"]
Apr 28 19:23:26.707314 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.707250    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p"
Apr 28 19:23:26.710731 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.710707    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Apr 28 19:23:26.710880 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.710739    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Apr 28 19:23:26.710880 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.710803    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-lhlwl\""
Apr 28 19:23:26.710995 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.710935    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Apr 28 19:23:26.800273 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.800237    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-68z4p\" (UID: \"b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p"
Apr 28 19:23:26.800461 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.800296    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd652\" (UniqueName: \"kubernetes.io/projected/b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f-kube-api-access-qd652\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-68z4p\" (UID: \"b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p"
Apr 28 19:23:26.901105 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.901075    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-68z4p\" (UID: \"b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p"
Apr 28 19:23:26.901262 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.901122    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qd652\" (UniqueName: \"kubernetes.io/projected/b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f-kube-api-access-qd652\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-68z4p\" (UID: \"b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p"
Apr 28 19:23:26.903349 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.903328    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-68z4p\" (UID: \"b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p"
Apr 28 19:23:26.911906 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:26.911849    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd652\" (UniqueName: \"kubernetes.io/projected/b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f-kube-api-access-qd652\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-68z4p\" (UID: \"b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p"
Apr 28 19:23:27.017678 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:27.017639    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p"
Apr 28 19:23:27.143982 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:27.143901    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p"]
Apr 28 19:23:27.146520 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:23:27.146489    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3145d6f_d7fa_434b_8bb1_e9ef4bbe372f.slice/crio-bc673f729cf3cd30aa31e2964ed6421a9ae1dde2631a6440c06b7b94370d1fb4 WatchSource:0}: Error finding container bc673f729cf3cd30aa31e2964ed6421a9ae1dde2631a6440c06b7b94370d1fb4: Status 404 returned error can't find the container with id bc673f729cf3cd30aa31e2964ed6421a9ae1dde2631a6440c06b7b94370d1fb4
Apr 28 19:23:27.205237 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:27.205202    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p" event={"ID":"b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f","Type":"ContainerStarted","Data":"bc673f729cf3cd30aa31e2964ed6421a9ae1dde2631a6440c06b7b94370d1fb4"}
Apr 28 19:23:31.220386 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.220351    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p" event={"ID":"b3145d6f-d7fa-434b-8bb1-e9ef4bbe372f","Type":"ContainerStarted","Data":"245fece3bda48affd7c23785c71903d8ce8de2ef6b5308be1b23a41a7bf0118d"}
Apr 28 19:23:31.220788 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.220412    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p"
Apr 28 19:23:31.273065 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.273012    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p" podStartSLOduration=1.662313525 podStartE2EDuration="5.272995839s" podCreationTimestamp="2026-04-28 19:23:26 +0000 UTC" firstStartedPulling="2026-04-28 19:23:27.148447623 +0000 UTC m=+393.972787709" lastFinishedPulling="2026-04-28 19:23:30.759129933 +0000 UTC m=+397.583470023" observedRunningTime="2026-04-28 19:23:31.269199885 +0000 UTC m=+398.093540003" watchObservedRunningTime="2026-04-28 19:23:31.272995839 +0000 UTC m=+398.097335946"
Apr 28 19:23:31.346088 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.346059    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-wj65m"]
Apr 28 19:23:31.349348 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.349330    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:31.352662 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.352632    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Apr 28 19:23:31.352662 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.352640    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Apr 28 19:23:31.352892 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.352879    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-cp5nd\""
Apr 28 19:23:31.360516 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.360479    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-wj65m"]
Apr 28 19:23:31.440981 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.440947    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/3bc13e44-3699-4438-bcc0-fba492f6d9b8-cabundle0\") pod \"keda-operator-ffbb595cb-wj65m\" (UID: \"3bc13e44-3699-4438-bcc0-fba492f6d9b8\") " pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:31.441244 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.441220    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh26g\" (UniqueName: \"kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-kube-api-access-gh26g\") pod \"keda-operator-ffbb595cb-wj65m\" (UID: \"3bc13e44-3699-4438-bcc0-fba492f6d9b8\") " pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:31.441364 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.441321    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates\") pod \"keda-operator-ffbb595cb-wj65m\" (UID: \"3bc13e44-3699-4438-bcc0-fba492f6d9b8\") " pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:31.542874 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.542786    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/3bc13e44-3699-4438-bcc0-fba492f6d9b8-cabundle0\") pod \"keda-operator-ffbb595cb-wj65m\" (UID: \"3bc13e44-3699-4438-bcc0-fba492f6d9b8\") " pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:31.542874 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.542835    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gh26g\" (UniqueName: \"kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-kube-api-access-gh26g\") pod \"keda-operator-ffbb595cb-wj65m\" (UID: \"3bc13e44-3699-4438-bcc0-fba492f6d9b8\") " pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:31.543086 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.542909    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates\") pod \"keda-operator-ffbb595cb-wj65m\" (UID: \"3bc13e44-3699-4438-bcc0-fba492f6d9b8\") " pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:31.543086 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:31.543014    2569 projected.go:264] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Apr 28 19:23:31.543086 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:31.543032    2569 secret.go:281] references non-existent secret key: ca.crt
Apr 28 19:23:31.543086 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:31.543043    2569 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Apr 28 19:23:31.543086 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:31.543058    2569 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-wj65m: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Apr 28 19:23:31.543331 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:31.543125    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates podName:3bc13e44-3699-4438-bcc0-fba492f6d9b8 nodeName:}" failed. No retries permitted until 2026-04-28 19:23:32.043107592 +0000 UTC m=+398.867447686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates") pod "keda-operator-ffbb595cb-wj65m" (UID: "3bc13e44-3699-4438-bcc0-fba492f6d9b8") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Apr 28 19:23:31.543534 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.543512    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/3bc13e44-3699-4438-bcc0-fba492f6d9b8-cabundle0\") pod \"keda-operator-ffbb595cb-wj65m\" (UID: \"3bc13e44-3699-4438-bcc0-fba492f6d9b8\") " pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:31.559064 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.559036    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh26g\" (UniqueName: \"kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-kube-api-access-gh26g\") pod \"keda-operator-ffbb595cb-wj65m\" (UID: \"3bc13e44-3699-4438-bcc0-fba492f6d9b8\") " pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:31.641924 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.641889    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"]
Apr 28 19:23:31.645931 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.645910    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:31.648800 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.648781    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Apr 28 19:23:31.653323 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.653303    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"]
Apr 28 19:23:31.744338 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.744303    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfh5g\" (UniqueName: \"kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-kube-api-access-lfh5g\") pod \"keda-metrics-apiserver-7c9f485588-b9zrs\" (UID: \"7c567112-ab4d-4cc8-bd4a-d29d725d8346\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:31.744519 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.744361    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7c567112-ab4d-4cc8-bd4a-d29d725d8346-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-b9zrs\" (UID: \"7c567112-ab4d-4cc8-bd4a-d29d725d8346\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:31.744519 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.744447    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-certificates\") pod \"keda-metrics-apiserver-7c9f485588-b9zrs\" (UID: \"7c567112-ab4d-4cc8-bd4a-d29d725d8346\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:31.845756 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.845670    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lfh5g\" (UniqueName: \"kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-kube-api-access-lfh5g\") pod \"keda-metrics-apiserver-7c9f485588-b9zrs\" (UID: \"7c567112-ab4d-4cc8-bd4a-d29d725d8346\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:31.845756 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.845713    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7c567112-ab4d-4cc8-bd4a-d29d725d8346-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-b9zrs\" (UID: \"7c567112-ab4d-4cc8-bd4a-d29d725d8346\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:31.845756 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.845753    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-certificates\") pod \"keda-metrics-apiserver-7c9f485588-b9zrs\" (UID: \"7c567112-ab4d-4cc8-bd4a-d29d725d8346\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:31.846026 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:31.845847    2569 secret.go:281] references non-existent secret key: tls.crt
Apr 28 19:23:31.846026 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:31.845862    2569 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Apr 28 19:23:31.846026 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:31.845882    2569 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs: references non-existent secret key: tls.crt
Apr 28 19:23:31.846026 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:31.845942    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-certificates podName:7c567112-ab4d-4cc8-bd4a-d29d725d8346 nodeName:}" failed. No retries permitted until 2026-04-28 19:23:32.345923799 +0000 UTC m=+399.170263896 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-certificates") pod "keda-metrics-apiserver-7c9f485588-b9zrs" (UID: "7c567112-ab4d-4cc8-bd4a-d29d725d8346") : references non-existent secret key: tls.crt
Apr 28 19:23:31.846235 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.846052    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7c567112-ab4d-4cc8-bd4a-d29d725d8346-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-b9zrs\" (UID: \"7c567112-ab4d-4cc8-bd4a-d29d725d8346\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:31.857741 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.857716    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfh5g\" (UniqueName: \"kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-kube-api-access-lfh5g\") pod \"keda-metrics-apiserver-7c9f485588-b9zrs\" (UID: \"7c567112-ab4d-4cc8-bd4a-d29d725d8346\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:31.919047 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.919016    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-cf49989db-kscr8"]
Apr 28 19:23:31.922378 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.922363    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-cf49989db-kscr8"
Apr 28 19:23:31.925259 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.925238    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Apr 28 19:23:31.930974 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:31.930950    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-cf49989db-kscr8"]
Apr 28 19:23:32.048275 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:32.048220    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbfmp\" (UniqueName: \"kubernetes.io/projected/c30172e6-0cea-44a3-89cd-aa8064e1da09-kube-api-access-wbfmp\") pod \"keda-admission-cf49989db-kscr8\" (UID: \"c30172e6-0cea-44a3-89cd-aa8064e1da09\") " pod="openshift-keda/keda-admission-cf49989db-kscr8"
Apr 28 19:23:32.048451 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:32.048333    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c30172e6-0cea-44a3-89cd-aa8064e1da09-certificates\") pod \"keda-admission-cf49989db-kscr8\" (UID: \"c30172e6-0cea-44a3-89cd-aa8064e1da09\") " pod="openshift-keda/keda-admission-cf49989db-kscr8"
Apr 28 19:23:32.048506 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:32.048447    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates\") pod \"keda-operator-ffbb595cb-wj65m\" (UID: \"3bc13e44-3699-4438-bcc0-fba492f6d9b8\") " pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:32.048575 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:32.048563    2569 secret.go:281] references non-existent secret key: ca.crt
Apr 28 19:23:32.048613 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:32.048579    2569 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Apr 28 19:23:32.048613 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:32.048587    2569 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-wj65m: references non-existent secret key: ca.crt
Apr 28 19:23:32.048716 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:32.048640    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates podName:3bc13e44-3699-4438-bcc0-fba492f6d9b8 nodeName:}" failed. No retries permitted until 2026-04-28 19:23:33.048625198 +0000 UTC m=+399.872965284 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates") pod "keda-operator-ffbb595cb-wj65m" (UID: "3bc13e44-3699-4438-bcc0-fba492f6d9b8") : references non-existent secret key: ca.crt
Apr 28 19:23:32.149020 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:32.148945    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wbfmp\" (UniqueName: \"kubernetes.io/projected/c30172e6-0cea-44a3-89cd-aa8064e1da09-kube-api-access-wbfmp\") pod \"keda-admission-cf49989db-kscr8\" (UID: \"c30172e6-0cea-44a3-89cd-aa8064e1da09\") " pod="openshift-keda/keda-admission-cf49989db-kscr8"
Apr 28 19:23:32.149180 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:32.149029    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c30172e6-0cea-44a3-89cd-aa8064e1da09-certificates\") pod \"keda-admission-cf49989db-kscr8\" (UID: \"c30172e6-0cea-44a3-89cd-aa8064e1da09\") " pod="openshift-keda/keda-admission-cf49989db-kscr8"
Apr 28 19:23:32.151626 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:32.151600    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c30172e6-0cea-44a3-89cd-aa8064e1da09-certificates\") pod \"keda-admission-cf49989db-kscr8\" (UID: \"c30172e6-0cea-44a3-89cd-aa8064e1da09\") " pod="openshift-keda/keda-admission-cf49989db-kscr8"
Apr 28 19:23:32.157545 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:32.157524    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbfmp\" (UniqueName: \"kubernetes.io/projected/c30172e6-0cea-44a3-89cd-aa8064e1da09-kube-api-access-wbfmp\") pod \"keda-admission-cf49989db-kscr8\" (UID: \"c30172e6-0cea-44a3-89cd-aa8064e1da09\") " pod="openshift-keda/keda-admission-cf49989db-kscr8"
Apr 28 19:23:32.233517 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:32.233460    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-cf49989db-kscr8"
Apr 28 19:23:32.351206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:32.351161    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-certificates\") pod \"keda-metrics-apiserver-7c9f485588-b9zrs\" (UID: \"7c567112-ab4d-4cc8-bd4a-d29d725d8346\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:32.351378 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:32.351274    2569 secret.go:281] references non-existent secret key: tls.crt
Apr 28 19:23:32.351378 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:32.351296    2569 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Apr 28 19:23:32.351378 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:32.351319    2569 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs: references non-existent secret key: tls.crt
Apr 28 19:23:32.351729 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:32.351390    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-certificates podName:7c567112-ab4d-4cc8-bd4a-d29d725d8346 nodeName:}" failed. No retries permitted until 2026-04-28 19:23:33.351369094 +0000 UTC m=+400.175709195 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-certificates") pod "keda-metrics-apiserver-7c9f485588-b9zrs" (UID: "7c567112-ab4d-4cc8-bd4a-d29d725d8346") : references non-existent secret key: tls.crt
Apr 28 19:23:32.355332 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:32.355307    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-cf49989db-kscr8"]
Apr 28 19:23:32.358921 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:23:32.358895    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30172e6_0cea_44a3_89cd_aa8064e1da09.slice/crio-09276b27316d3c0ae9d3919c3a149b7f4e0315960e92ea37b5f21ab8be56723c WatchSource:0}: Error finding container 09276b27316d3c0ae9d3919c3a149b7f4e0315960e92ea37b5f21ab8be56723c: Status 404 returned error can't find the container with id 09276b27316d3c0ae9d3919c3a149b7f4e0315960e92ea37b5f21ab8be56723c
Apr 28 19:23:33.059609 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:33.059574    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates\") pod \"keda-operator-ffbb595cb-wj65m\" (UID: \"3bc13e44-3699-4438-bcc0-fba492f6d9b8\") " pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:33.059776 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:33.059694    2569 secret.go:281] references non-existent secret key: ca.crt
Apr 28 19:23:33.059776 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:33.059707    2569 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Apr 28 19:23:33.059776 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:33.059716    2569 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-wj65m: references non-existent secret key: ca.crt
Apr 28 19:23:33.059948 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:33.059782    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates podName:3bc13e44-3699-4438-bcc0-fba492f6d9b8 nodeName:}" failed. No retries permitted until 2026-04-28 19:23:35.059765593 +0000 UTC m=+401.884105680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates") pod "keda-operator-ffbb595cb-wj65m" (UID: "3bc13e44-3699-4438-bcc0-fba492f6d9b8") : references non-existent secret key: ca.crt
Apr 28 19:23:33.229147 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:33.229103    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-cf49989db-kscr8" event={"ID":"c30172e6-0cea-44a3-89cd-aa8064e1da09","Type":"ContainerStarted","Data":"09276b27316d3c0ae9d3919c3a149b7f4e0315960e92ea37b5f21ab8be56723c"}
Apr 28 19:23:33.363006 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:33.362930    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-certificates\") pod \"keda-metrics-apiserver-7c9f485588-b9zrs\" (UID: \"7c567112-ab4d-4cc8-bd4a-d29d725d8346\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:33.363394 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:33.363085    2569 secret.go:281] references non-existent secret key: tls.crt
Apr 28 19:23:33.363394 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:33.363106    2569 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Apr 28 19:23:33.363394 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:33.363137    2569 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs: references non-existent secret key: tls.crt
Apr 28 19:23:33.363394 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:33.363190    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-certificates podName:7c567112-ab4d-4cc8-bd4a-d29d725d8346 nodeName:}" failed. No retries permitted until 2026-04-28 19:23:35.36317659 +0000 UTC m=+402.187516676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-certificates") pod "keda-metrics-apiserver-7c9f485588-b9zrs" (UID: "7c567112-ab4d-4cc8-bd4a-d29d725d8346") : references non-existent secret key: tls.crt
Apr 28 19:23:34.234115 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:34.234075    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-cf49989db-kscr8" event={"ID":"c30172e6-0cea-44a3-89cd-aa8064e1da09","Type":"ContainerStarted","Data":"9020e860eb7718ad31d30dbdc4476f6e5aa1af9d31258293be356238dc359a0c"}
Apr 28 19:23:34.234298 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:34.234188    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-cf49989db-kscr8"
Apr 28 19:23:34.256085 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:34.256035    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-admission-cf49989db-kscr8" podStartSLOduration=1.906130535 podStartE2EDuration="3.25601808s" podCreationTimestamp="2026-04-28 19:23:31 +0000 UTC" firstStartedPulling="2026-04-28 19:23:32.360323912 +0000 UTC m=+399.184663998" lastFinishedPulling="2026-04-28 19:23:33.710211442 +0000 UTC m=+400.534551543" observedRunningTime="2026-04-28 19:23:34.253026757 +0000 UTC m=+401.077366867" watchObservedRunningTime="2026-04-28 19:23:34.25601808 +0000 UTC m=+401.080358189"
Apr 28 19:23:35.078521 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:35.078488    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates\") pod \"keda-operator-ffbb595cb-wj65m\" (UID: \"3bc13e44-3699-4438-bcc0-fba492f6d9b8\") " pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:35.078909 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:35.078636    2569 secret.go:281] references non-existent secret key: ca.crt
Apr 28 19:23:35.078909 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:35.078654    2569 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Apr 28 19:23:35.078909 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:35.078663    2569 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-wj65m: references non-existent secret key: ca.crt
Apr 28 19:23:35.078909 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:23:35.078714    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates podName:3bc13e44-3699-4438-bcc0-fba492f6d9b8 nodeName:}" failed. No retries permitted until 2026-04-28 19:23:39.078699238 +0000 UTC m=+405.903039324 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates") pod "keda-operator-ffbb595cb-wj65m" (UID: "3bc13e44-3699-4438-bcc0-fba492f6d9b8") : references non-existent secret key: ca.crt
Apr 28 19:23:35.381637 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:35.381550    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-certificates\") pod \"keda-metrics-apiserver-7c9f485588-b9zrs\" (UID: \"7c567112-ab4d-4cc8-bd4a-d29d725d8346\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:35.383986 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:35.383962    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7c567112-ab4d-4cc8-bd4a-d29d725d8346-certificates\") pod \"keda-metrics-apiserver-7c9f485588-b9zrs\" (UID: \"7c567112-ab4d-4cc8-bd4a-d29d725d8346\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:35.558876 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:35.558832    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:35.680734 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:35.680710    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"]
Apr 28 19:23:35.683034 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:23:35.683004    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c567112_ab4d_4cc8_bd4a_d29d725d8346.slice/crio-38b8e88dd5a8224f7bdd8d732f0d6b8099bb360ff0682bda1d67e0589a49768e WatchSource:0}: Error finding container 38b8e88dd5a8224f7bdd8d732f0d6b8099bb360ff0682bda1d67e0589a49768e: Status 404 returned error can't find the container with id 38b8e88dd5a8224f7bdd8d732f0d6b8099bb360ff0682bda1d67e0589a49768e
Apr 28 19:23:36.241630 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:36.241596    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs" event={"ID":"7c567112-ab4d-4cc8-bd4a-d29d725d8346","Type":"ContainerStarted","Data":"38b8e88dd5a8224f7bdd8d732f0d6b8099bb360ff0682bda1d67e0589a49768e"}
Apr 28 19:23:39.114923 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:39.114891    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates\") pod \"keda-operator-ffbb595cb-wj65m\" (UID: \"3bc13e44-3699-4438-bcc0-fba492f6d9b8\") " pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:39.117395 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:39.117374    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3bc13e44-3699-4438-bcc0-fba492f6d9b8-certificates\") pod \"keda-operator-ffbb595cb-wj65m\" (UID: \"3bc13e44-3699-4438-bcc0-fba492f6d9b8\") " pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:39.166816 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:39.166779    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:39.253481 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:39.253387    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs" event={"ID":"7c567112-ab4d-4cc8-bd4a-d29d725d8346","Type":"ContainerStarted","Data":"ae720a89253df1865f438337412c9d2a4131b4db0a91c5db3fc9ccecd06b9e68"}
Apr 28 19:23:39.253794 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:39.253762    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:39.279504 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:39.279447    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs" podStartSLOduration=4.999657128 podStartE2EDuration="8.279427819s" podCreationTimestamp="2026-04-28 19:23:31 +0000 UTC" firstStartedPulling="2026-04-28 19:23:35.684326859 +0000 UTC m=+402.508666945" lastFinishedPulling="2026-04-28 19:23:38.964097534 +0000 UTC m=+405.788437636" observedRunningTime="2026-04-28 19:23:39.278949195 +0000 UTC m=+406.103289332" watchObservedRunningTime="2026-04-28 19:23:39.279427819 +0000 UTC m=+406.103767923"
Apr 28 19:23:39.296372 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:39.296348    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-wj65m"]
Apr 28 19:23:39.297918 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:23:39.297895    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc13e44_3699_4438_bcc0_fba492f6d9b8.slice/crio-a863889e09b34ef4e2f7f9c88f9339e268bb1d2c3acdb1d8100fc9614935360f WatchSource:0}: Error finding container a863889e09b34ef4e2f7f9c88f9339e268bb1d2c3acdb1d8100fc9614935360f: Status 404 returned error can't find the container with id a863889e09b34ef4e2f7f9c88f9339e268bb1d2c3acdb1d8100fc9614935360f
Apr 28 19:23:40.258659 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:40.258618    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-wj65m" event={"ID":"3bc13e44-3699-4438-bcc0-fba492f6d9b8","Type":"ContainerStarted","Data":"a863889e09b34ef4e2f7f9c88f9339e268bb1d2c3acdb1d8100fc9614935360f"}
Apr 28 19:23:44.273612 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:44.273579    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-wj65m" event={"ID":"3bc13e44-3699-4438-bcc0-fba492f6d9b8","Type":"ContainerStarted","Data":"48a4c9699e87864a6c61e0d728a850ded08d28da67a461e5573f035094d1ef62"}
Apr 28 19:23:44.274041 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:44.273634    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:23:44.299052 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:44.299009    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-operator-ffbb595cb-wj65m" podStartSLOduration=9.216352978 podStartE2EDuration="13.298996808s" podCreationTimestamp="2026-04-28 19:23:31 +0000 UTC" firstStartedPulling="2026-04-28 19:23:39.299424628 +0000 UTC m=+406.123764727" lastFinishedPulling="2026-04-28 19:23:43.382068468 +0000 UTC m=+410.206408557" observedRunningTime="2026-04-28 19:23:44.296578697 +0000 UTC m=+411.120918804" watchObservedRunningTime="2026-04-28 19:23:44.298996808 +0000 UTC m=+411.123336916"
Apr 28 19:23:50.263583 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:50.263555    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-b9zrs"
Apr 28 19:23:52.225940 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:52.225912    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-68z4p"
Apr 28 19:23:55.239730 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:23:55.239697    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-cf49989db-kscr8"
Apr 28 19:24:05.278768 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:05.278734    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-ffbb595cb-wj65m"
Apr 28 19:24:24.222585 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.222552    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"]
Apr 28 19:24:24.231597 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.231570    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"
Apr 28 19:24:24.233965 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.233937    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"]
Apr 28 19:24:24.234447 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.234384    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 28 19:24:24.234571 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.234452    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nvrrr\""
Apr 28 19:24:24.235707 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.235688    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 28 19:24:24.397101 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.397058    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f987462-4429-477a-92c1-0cd1a12b3c8e-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6\" (UID: \"1f987462-4429-477a-92c1-0cd1a12b3c8e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"
Apr 28 19:24:24.397276 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.397127    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7v8m\" (UniqueName: \"kubernetes.io/projected/1f987462-4429-477a-92c1-0cd1a12b3c8e-kube-api-access-z7v8m\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6\" (UID: \"1f987462-4429-477a-92c1-0cd1a12b3c8e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"
Apr 28 19:24:24.397276 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.397149    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f987462-4429-477a-92c1-0cd1a12b3c8e-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6\" (UID: \"1f987462-4429-477a-92c1-0cd1a12b3c8e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"
Apr 28 19:24:24.497580 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.497504    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f987462-4429-477a-92c1-0cd1a12b3c8e-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6\" (UID: \"1f987462-4429-477a-92c1-0cd1a12b3c8e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"
Apr 28 19:24:24.497580 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.497563    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z7v8m\" (UniqueName: \"kubernetes.io/projected/1f987462-4429-477a-92c1-0cd1a12b3c8e-kube-api-access-z7v8m\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6\" (UID: \"1f987462-4429-477a-92c1-0cd1a12b3c8e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"
Apr 28 19:24:24.497749 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.497583    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f987462-4429-477a-92c1-0cd1a12b3c8e-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6\" (UID: \"1f987462-4429-477a-92c1-0cd1a12b3c8e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"
Apr 28 19:24:24.497911 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.497892    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f987462-4429-477a-92c1-0cd1a12b3c8e-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6\" (UID: \"1f987462-4429-477a-92c1-0cd1a12b3c8e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"
Apr 28 19:24:24.497949 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.497913    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f987462-4429-477a-92c1-0cd1a12b3c8e-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6\" (UID: \"1f987462-4429-477a-92c1-0cd1a12b3c8e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"
Apr 28 19:24:24.506277 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.506246    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7v8m\" (UniqueName: \"kubernetes.io/projected/1f987462-4429-477a-92c1-0cd1a12b3c8e-kube-api-access-z7v8m\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6\" (UID: \"1f987462-4429-477a-92c1-0cd1a12b3c8e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"
Apr 28 19:24:24.542144 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.542121    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"
Apr 28 19:24:24.662351 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:24.662323    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"]
Apr 28 19:24:24.665560 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:24:24.665536    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f987462_4429_477a_92c1_0cd1a12b3c8e.slice/crio-2a27a5d4da8ac493b790a64c4f61f7780471ae31cc92346d167bda8fa1862390 WatchSource:0}: Error finding container 2a27a5d4da8ac493b790a64c4f61f7780471ae31cc92346d167bda8fa1862390: Status 404 returned error can't find the container with id 2a27a5d4da8ac493b790a64c4f61f7780471ae31cc92346d167bda8fa1862390
Apr 28 19:24:25.407170 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:25.407135    2569 generic.go:358] "Generic (PLEG): container finished" podID="1f987462-4429-477a-92c1-0cd1a12b3c8e" containerID="4b70509858eece5b598bfcc52406cb18dbd371019dc7937d5c5a462cf455095d" exitCode=0
Apr 28 19:24:25.407580 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:25.407219    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6" event={"ID":"1f987462-4429-477a-92c1-0cd1a12b3c8e","Type":"ContainerDied","Data":"4b70509858eece5b598bfcc52406cb18dbd371019dc7937d5c5a462cf455095d"}
Apr 28 19:24:25.407580 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:25.407249    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6" event={"ID":"1f987462-4429-477a-92c1-0cd1a12b3c8e","Type":"ContainerStarted","Data":"2a27a5d4da8ac493b790a64c4f61f7780471ae31cc92346d167bda8fa1862390"}
Apr 28 19:24:28.419164 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:28.419127    2569 generic.go:358] "Generic (PLEG): container finished" podID="1f987462-4429-477a-92c1-0cd1a12b3c8e" containerID="ff544b6488eb44511628812b95e6fe55a80fc267e208da318aac8e63cbb28c32" exitCode=0
Apr 28 19:24:28.419552 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:28.419196    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6" event={"ID":"1f987462-4429-477a-92c1-0cd1a12b3c8e","Type":"ContainerDied","Data":"ff544b6488eb44511628812b95e6fe55a80fc267e208da318aac8e63cbb28c32"}
Apr 28 19:24:29.424696 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:29.424664    2569 generic.go:358] "Generic (PLEG): container finished" podID="1f987462-4429-477a-92c1-0cd1a12b3c8e" containerID="ec810294266b416731b8c7078e86f70d5e17acfe2733cb800827a126b4354068" exitCode=0
Apr 28 19:24:29.425165 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:29.424730    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6" event={"ID":"1f987462-4429-477a-92c1-0cd1a12b3c8e","Type":"ContainerDied","Data":"ec810294266b416731b8c7078e86f70d5e17acfe2733cb800827a126b4354068"}
Apr 28 19:24:30.545546 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:30.545524    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"
Apr 28 19:24:30.651157 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:30.651126    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f987462-4429-477a-92c1-0cd1a12b3c8e-bundle\") pod \"1f987462-4429-477a-92c1-0cd1a12b3c8e\" (UID: \"1f987462-4429-477a-92c1-0cd1a12b3c8e\") "
Apr 28 19:24:30.651327 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:30.651227    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f987462-4429-477a-92c1-0cd1a12b3c8e-util\") pod \"1f987462-4429-477a-92c1-0cd1a12b3c8e\" (UID: \"1f987462-4429-477a-92c1-0cd1a12b3c8e\") "
Apr 28 19:24:30.651327 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:30.651256    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7v8m\" (UniqueName: \"kubernetes.io/projected/1f987462-4429-477a-92c1-0cd1a12b3c8e-kube-api-access-z7v8m\") pod \"1f987462-4429-477a-92c1-0cd1a12b3c8e\" (UID: \"1f987462-4429-477a-92c1-0cd1a12b3c8e\") "
Apr 28 19:24:30.651820 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:30.651797    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f987462-4429-477a-92c1-0cd1a12b3c8e-bundle" (OuterVolumeSpecName: "bundle") pod "1f987462-4429-477a-92c1-0cd1a12b3c8e" (UID: "1f987462-4429-477a-92c1-0cd1a12b3c8e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:24:30.653335 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:30.653306    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f987462-4429-477a-92c1-0cd1a12b3c8e-kube-api-access-z7v8m" (OuterVolumeSpecName: "kube-api-access-z7v8m") pod "1f987462-4429-477a-92c1-0cd1a12b3c8e" (UID: "1f987462-4429-477a-92c1-0cd1a12b3c8e"). InnerVolumeSpecName "kube-api-access-z7v8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:24:30.656082 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:30.656060    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f987462-4429-477a-92c1-0cd1a12b3c8e-util" (OuterVolumeSpecName: "util") pod "1f987462-4429-477a-92c1-0cd1a12b3c8e" (UID: "1f987462-4429-477a-92c1-0cd1a12b3c8e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:24:30.752787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:30.752756    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7v8m\" (UniqueName: \"kubernetes.io/projected/1f987462-4429-477a-92c1-0cd1a12b3c8e-kube-api-access-z7v8m\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:24:30.752787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:30.752783    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f987462-4429-477a-92c1-0cd1a12b3c8e-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:24:30.752787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:30.752793    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f987462-4429-477a-92c1-0cd1a12b3c8e-util\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:24:31.432704 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:31.432666    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6" event={"ID":"1f987462-4429-477a-92c1-0cd1a12b3c8e","Type":"ContainerDied","Data":"2a27a5d4da8ac493b790a64c4f61f7780471ae31cc92346d167bda8fa1862390"}
Apr 28 19:24:31.432704 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:31.432703    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a27a5d4da8ac493b790a64c4f61f7780471ae31cc92346d167bda8fa1862390"
Apr 28 19:24:31.432704 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:31.432705    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dzm2j6"
Apr 28 19:24:46.370752 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.370715    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"]
Apr 28 19:24:46.371263 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.371057    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f987462-4429-477a-92c1-0cd1a12b3c8e" containerName="util"
Apr 28 19:24:46.371263 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.371068    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f987462-4429-477a-92c1-0cd1a12b3c8e" containerName="util"
Apr 28 19:24:46.371263 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.371080    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f987462-4429-477a-92c1-0cd1a12b3c8e" containerName="extract"
Apr 28 19:24:46.371263 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.371086    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f987462-4429-477a-92c1-0cd1a12b3c8e" containerName="extract"
Apr 28 19:24:46.371263 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.371095    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f987462-4429-477a-92c1-0cd1a12b3c8e" containerName="pull"
Apr 28 19:24:46.371263 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.371100    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f987462-4429-477a-92c1-0cd1a12b3c8e" containerName="pull"
Apr 28 19:24:46.371263 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.371161    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="1f987462-4429-477a-92c1-0cd1a12b3c8e" containerName="extract"
Apr 28 19:24:46.373181 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.373164    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"
Apr 28 19:24:46.376099 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.376075    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nvrrr\""
Apr 28 19:24:46.376242 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.376124    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 28 19:24:46.376242 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.376136    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 28 19:24:46.383742 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.383720    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"]
Apr 28 19:24:46.482984 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.482952    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm42n\" (UniqueName: \"kubernetes.io/projected/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-kube-api-access-bm42n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp\" (UID: \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"
Apr 28 19:24:46.483167 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.482992    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp\" (UID: \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"
Apr 28 19:24:46.483167 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.483075    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp\" (UID: \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"
Apr 28 19:24:46.584494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.584452    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp\" (UID: \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"
Apr 28 19:24:46.584680 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.584540    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bm42n\" (UniqueName: \"kubernetes.io/projected/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-kube-api-access-bm42n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp\" (UID: \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"
Apr 28 19:24:46.584680 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.584574    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp\" (UID: \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"
Apr 28 19:24:46.584938 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.584914    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp\" (UID: \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"
Apr 28 19:24:46.584938 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.584929    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp\" (UID: \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"
Apr 28 19:24:46.594976 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.594946    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm42n\" (UniqueName: \"kubernetes.io/projected/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-kube-api-access-bm42n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp\" (UID: \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"
Apr 28 19:24:46.682676 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.682654    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"
Apr 28 19:24:46.802047 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:46.802011    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"]
Apr 28 19:24:46.804326 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:24:46.804299    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e2ffda_3ef7_48cd_909d_b27ba94042f8.slice/crio-3033acba915245605d6e1159b6bd90f64e2759fa43dbdb573dd825d4014c2cc4 WatchSource:0}: Error finding container 3033acba915245605d6e1159b6bd90f64e2759fa43dbdb573dd825d4014c2cc4: Status 404 returned error can't find the container with id 3033acba915245605d6e1159b6bd90f64e2759fa43dbdb573dd825d4014c2cc4
Apr 28 19:24:47.489524 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:47.489491    2569 generic.go:358] "Generic (PLEG): container finished" podID="a3e2ffda-3ef7-48cd-909d-b27ba94042f8" containerID="694fa8a94b9a138a5330c3efcafcb5610df712299d0e160211567f0ef35d39ef" exitCode=0
Apr 28 19:24:47.490002 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:47.489582    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp" event={"ID":"a3e2ffda-3ef7-48cd-909d-b27ba94042f8","Type":"ContainerDied","Data":"694fa8a94b9a138a5330c3efcafcb5610df712299d0e160211567f0ef35d39ef"}
Apr 28 19:24:47.490002 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:47.489618    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp" event={"ID":"a3e2ffda-3ef7-48cd-909d-b27ba94042f8","Type":"ContainerStarted","Data":"3033acba915245605d6e1159b6bd90f64e2759fa43dbdb573dd825d4014c2cc4"}
Apr 28 19:24:50.503078 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:50.503046    2569 generic.go:358] "Generic (PLEG): container finished" podID="a3e2ffda-3ef7-48cd-909d-b27ba94042f8" containerID="c93fa18aa2d4ad612fa3ae80a9f2ae77729e41468df6e489f257fa00a2b1505a" exitCode=0
Apr 28 19:24:50.503569 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:50.503132    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp" event={"ID":"a3e2ffda-3ef7-48cd-909d-b27ba94042f8","Type":"ContainerDied","Data":"c93fa18aa2d4ad612fa3ae80a9f2ae77729e41468df6e489f257fa00a2b1505a"}
Apr 28 19:24:51.508376 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:51.508342    2569 generic.go:358] "Generic (PLEG): container finished" podID="a3e2ffda-3ef7-48cd-909d-b27ba94042f8" containerID="cf5b5e2ab444b610ff9a33f7a08219047f5d3ac790baf34436425b735f346b9b" exitCode=0
Apr 28 19:24:51.508759 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:51.508426    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp" event={"ID":"a3e2ffda-3ef7-48cd-909d-b27ba94042f8","Type":"ContainerDied","Data":"cf5b5e2ab444b610ff9a33f7a08219047f5d3ac790baf34436425b735f346b9b"}
Apr 28 19:24:52.627884 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:52.627855    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"
Apr 28 19:24:52.736121 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:52.736094    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-util\") pod \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\" (UID: \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\") "
Apr 28 19:24:52.736265 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:52.736132    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bm42n\" (UniqueName: \"kubernetes.io/projected/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-kube-api-access-bm42n\") pod \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\" (UID: \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\") "
Apr 28 19:24:52.736265 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:52.736181    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-bundle\") pod \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\" (UID: \"a3e2ffda-3ef7-48cd-909d-b27ba94042f8\") "
Apr 28 19:24:52.736628 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:52.736605    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-bundle" (OuterVolumeSpecName: "bundle") pod "a3e2ffda-3ef7-48cd-909d-b27ba94042f8" (UID: "a3e2ffda-3ef7-48cd-909d-b27ba94042f8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:24:52.738105 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:52.738076    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-kube-api-access-bm42n" (OuterVolumeSpecName: "kube-api-access-bm42n") pod "a3e2ffda-3ef7-48cd-909d-b27ba94042f8" (UID: "a3e2ffda-3ef7-48cd-909d-b27ba94042f8"). InnerVolumeSpecName "kube-api-access-bm42n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:24:52.778656 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:52.778596    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-util" (OuterVolumeSpecName: "util") pod "a3e2ffda-3ef7-48cd-909d-b27ba94042f8" (UID: "a3e2ffda-3ef7-48cd-909d-b27ba94042f8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:24:52.837017 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:52.836998    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bm42n\" (UniqueName: \"kubernetes.io/projected/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-kube-api-access-bm42n\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:24:52.837017 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:52.837018    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:24:52.837154 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:52.837028    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e2ffda-3ef7-48cd-909d-b27ba94042f8-util\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:24:53.516422 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:53.516373    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp"
Apr 28 19:24:53.516422 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:53.516390    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rcsp" event={"ID":"a3e2ffda-3ef7-48cd-909d-b27ba94042f8","Type":"ContainerDied","Data":"3033acba915245605d6e1159b6bd90f64e2759fa43dbdb573dd825d4014c2cc4"}
Apr 28 19:24:53.516683 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:53.516434    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3033acba915245605d6e1159b6bd90f64e2759fa43dbdb573dd825d4014c2cc4"
Apr 28 19:24:59.428620 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.428578    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl"]
Apr 28 19:24:59.429127 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.429107    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e2ffda-3ef7-48cd-909d-b27ba94042f8" containerName="pull"
Apr 28 19:24:59.429205 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.429130    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e2ffda-3ef7-48cd-909d-b27ba94042f8" containerName="pull"
Apr 28 19:24:59.429205 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.429146    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e2ffda-3ef7-48cd-909d-b27ba94042f8" containerName="util"
Apr 28 19:24:59.429205 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.429154    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e2ffda-3ef7-48cd-909d-b27ba94042f8" containerName="util"
Apr 28 19:24:59.429205 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.429163    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e2ffda-3ef7-48cd-909d-b27ba94042f8" containerName="extract"
Apr 28 19:24:59.429205 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.429171    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e2ffda-3ef7-48cd-909d-b27ba94042f8" containerName="extract"
Apr 28 19:24:59.429474 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.429262    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="a3e2ffda-3ef7-48cd-909d-b27ba94042f8" containerName="extract"
Apr 28 19:24:59.432042 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.432022    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl"
Apr 28 19:24:59.435980 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.435957    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Apr 28 19:24:59.437033 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.437015    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-f22hq\""
Apr 28 19:24:59.437131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.437017    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Apr 28 19:24:59.451936 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.451911    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl"]
Apr 28 19:24:59.495349 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.495325    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f370d3f9-0a7e-4402-8a6c-e94fbf56705c-tmp\") pod \"openshift-lws-operator-bfc7f696d-jvdxl\" (UID: \"f370d3f9-0a7e-4402-8a6c-e94fbf56705c\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl"
Apr 28 19:24:59.495465 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.495371    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbzcv\" (UniqueName: \"kubernetes.io/projected/f370d3f9-0a7e-4402-8a6c-e94fbf56705c-kube-api-access-rbzcv\") pod \"openshift-lws-operator-bfc7f696d-jvdxl\" (UID: \"f370d3f9-0a7e-4402-8a6c-e94fbf56705c\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl"
Apr 28 19:24:59.596766 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.596726    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f370d3f9-0a7e-4402-8a6c-e94fbf56705c-tmp\") pod \"openshift-lws-operator-bfc7f696d-jvdxl\" (UID: \"f370d3f9-0a7e-4402-8a6c-e94fbf56705c\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl"
Apr 28 19:24:59.596766 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.596776    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rbzcv\" (UniqueName: \"kubernetes.io/projected/f370d3f9-0a7e-4402-8a6c-e94fbf56705c-kube-api-access-rbzcv\") pod \"openshift-lws-operator-bfc7f696d-jvdxl\" (UID: \"f370d3f9-0a7e-4402-8a6c-e94fbf56705c\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl"
Apr 28 19:24:59.597138 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.597116    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f370d3f9-0a7e-4402-8a6c-e94fbf56705c-tmp\") pod \"openshift-lws-operator-bfc7f696d-jvdxl\" (UID: \"f370d3f9-0a7e-4402-8a6c-e94fbf56705c\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl"
Apr 28 19:24:59.605774 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.605755    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbzcv\" (UniqueName: \"kubernetes.io/projected/f370d3f9-0a7e-4402-8a6c-e94fbf56705c-kube-api-access-rbzcv\") pod \"openshift-lws-operator-bfc7f696d-jvdxl\" (UID: \"f370d3f9-0a7e-4402-8a6c-e94fbf56705c\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl"
Apr 28 19:24:59.757556 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.757517    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl"
Apr 28 19:24:59.907308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:24:59.907284    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl"]
Apr 28 19:24:59.909440 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:24:59.909412    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf370d3f9_0a7e_4402_8a6c_e94fbf56705c.slice/crio-6e37eb29ed9fa3f169159af8632c2fc53fb6e7db06323090fe6529295216004e WatchSource:0}: Error finding container 6e37eb29ed9fa3f169159af8632c2fc53fb6e7db06323090fe6529295216004e: Status 404 returned error can't find the container with id 6e37eb29ed9fa3f169159af8632c2fc53fb6e7db06323090fe6529295216004e
Apr 28 19:25:00.546278 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:00.546247    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl" event={"ID":"f370d3f9-0a7e-4402-8a6c-e94fbf56705c","Type":"ContainerStarted","Data":"6e37eb29ed9fa3f169159af8632c2fc53fb6e7db06323090fe6529295216004e"}
Apr 28 19:25:03.558484 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:03.558438    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl" event={"ID":"f370d3f9-0a7e-4402-8a6c-e94fbf56705c","Type":"ContainerStarted","Data":"4f8cc58f8f2175e36679e852abd65692f4bbda14303e5f02fd879dcdb84b625b"}
Apr 28 19:25:03.584388 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:03.584335    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-jvdxl" podStartSLOduration=1.93334922 podStartE2EDuration="4.584321742s" podCreationTimestamp="2026-04-28 19:24:59 +0000 UTC" firstStartedPulling="2026-04-28 19:24:59.910770824 +0000 UTC m=+486.735110912" lastFinishedPulling="2026-04-28 19:25:02.561743348 +0000 UTC m=+489.386083434" observedRunningTime="2026-04-28 19:25:03.582653456 +0000 UTC m=+490.406993565" watchObservedRunningTime="2026-04-28 19:25:03.584321742 +0000 UTC m=+490.408661846"
Apr 28 19:25:15.575558 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.575524    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"]
Apr 28 19:25:15.577843 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.577826    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"
Apr 28 19:25:15.580587 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.580564    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 28 19:25:15.581679 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.581651    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nvrrr\""
Apr 28 19:25:15.581679 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.581658    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 28 19:25:15.588635 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.588612    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"]
Apr 28 19:25:15.735354 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.735327    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc\" (UID: \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"
Apr 28 19:25:15.735495 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.735371    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc\" (UID: \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"
Apr 28 19:25:15.735495 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.735458    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjk4b\" (UniqueName: \"kubernetes.io/projected/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-kube-api-access-bjk4b\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc\" (UID: \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"
Apr 28 19:25:15.836178 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.836111    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc\" (UID: \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"
Apr 28 19:25:15.836178 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.836165    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc\" (UID: \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"
Apr 28 19:25:15.836348 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.836252    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bjk4b\" (UniqueName: \"kubernetes.io/projected/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-kube-api-access-bjk4b\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc\" (UID: \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"
Apr 28 19:25:15.836504 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.836486    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc\" (UID: \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"
Apr 28 19:25:15.836576 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.836556    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc\" (UID: \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"
Apr 28 19:25:15.846285 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.846265    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjk4b\" (UniqueName: \"kubernetes.io/projected/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-kube-api-access-bjk4b\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc\" (UID: \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"
Apr 28 19:25:15.887225 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:15.887198    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"
Apr 28 19:25:16.006156 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:16.006134    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"]
Apr 28 19:25:16.008663 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:25:16.008626    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf33c0ac4_b47d_43f4_9204_9bc31279f9ff.slice/crio-c2a7b1df02c8364836546a2dd0a8085486468f31f1e884d07e8b16988f15e5bd WatchSource:0}: Error finding container c2a7b1df02c8364836546a2dd0a8085486468f31f1e884d07e8b16988f15e5bd: Status 404 returned error can't find the container with id c2a7b1df02c8364836546a2dd0a8085486468f31f1e884d07e8b16988f15e5bd
Apr 28 19:25:16.602389 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:16.602352    2569 generic.go:358] "Generic (PLEG): container finished" podID="f33c0ac4-b47d-43f4-9204-9bc31279f9ff" containerID="dc1b8c7332f88b78f3356518310a5d79a48c803a341f03fc17bdb74abbb0af82" exitCode=0
Apr 28 19:25:16.602389 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:16.602383    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc" event={"ID":"f33c0ac4-b47d-43f4-9204-9bc31279f9ff","Type":"ContainerDied","Data":"dc1b8c7332f88b78f3356518310a5d79a48c803a341f03fc17bdb74abbb0af82"}
Apr 28 19:25:16.602941 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:16.602434    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc" event={"ID":"f33c0ac4-b47d-43f4-9204-9bc31279f9ff","Type":"ContainerStarted","Data":"c2a7b1df02c8364836546a2dd0a8085486468f31f1e884d07e8b16988f15e5bd"}
Apr 28 19:25:17.612755 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:17.612728    2569 generic.go:358] "Generic (PLEG): container finished" podID="f33c0ac4-b47d-43f4-9204-9bc31279f9ff" containerID="eb63a28ffbda27441b2cd12c21327f9f98613ce080273d4f0233c6e763464dc8" exitCode=0
Apr 28 19:25:17.613136 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:17.612763    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc" event={"ID":"f33c0ac4-b47d-43f4-9204-9bc31279f9ff","Type":"ContainerDied","Data":"eb63a28ffbda27441b2cd12c21327f9f98613ce080273d4f0233c6e763464dc8"}
Apr 28 19:25:18.621328 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:18.621292    2569 generic.go:358] "Generic (PLEG): container finished" podID="f33c0ac4-b47d-43f4-9204-9bc31279f9ff" containerID="1b2f915fa7f84396f868ac2f095c6dcd012f5f25f1b6d633df23d622fdc4813e" exitCode=0
Apr 28 19:25:18.621702 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:18.621375    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc" event={"ID":"f33c0ac4-b47d-43f4-9204-9bc31279f9ff","Type":"ContainerDied","Data":"1b2f915fa7f84396f868ac2f095c6dcd012f5f25f1b6d633df23d622fdc4813e"}
Apr 28 19:25:19.749959 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:19.749934    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"
Apr 28 19:25:19.876151 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:19.876113    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-bundle\") pod \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\" (UID: \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\") "
Apr 28 19:25:19.876361 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:19.876164    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-util\") pod \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\" (UID: \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\") "
Apr 28 19:25:19.876361 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:19.876199    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjk4b\" (UniqueName: \"kubernetes.io/projected/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-kube-api-access-bjk4b\") pod \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\" (UID: \"f33c0ac4-b47d-43f4-9204-9bc31279f9ff\") "
Apr 28 19:25:19.877129 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:19.877104    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-bundle" (OuterVolumeSpecName: "bundle") pod "f33c0ac4-b47d-43f4-9204-9bc31279f9ff" (UID: "f33c0ac4-b47d-43f4-9204-9bc31279f9ff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:25:19.878290 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:19.878263    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-kube-api-access-bjk4b" (OuterVolumeSpecName: "kube-api-access-bjk4b") pod "f33c0ac4-b47d-43f4-9204-9bc31279f9ff" (UID: "f33c0ac4-b47d-43f4-9204-9bc31279f9ff"). InnerVolumeSpecName "kube-api-access-bjk4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:25:19.881681 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:19.881645    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-util" (OuterVolumeSpecName: "util") pod "f33c0ac4-b47d-43f4-9204-9bc31279f9ff" (UID: "f33c0ac4-b47d-43f4-9204-9bc31279f9ff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:25:19.977230 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:19.977194    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjk4b\" (UniqueName: \"kubernetes.io/projected/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-kube-api-access-bjk4b\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:19.977230 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:19.977223    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:19.977436 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:19.977238    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f33c0ac4-b47d-43f4-9204-9bc31279f9ff-util\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:20.630780 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:20.630745    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc" event={"ID":"f33c0ac4-b47d-43f4-9204-9bc31279f9ff","Type":"ContainerDied","Data":"c2a7b1df02c8364836546a2dd0a8085486468f31f1e884d07e8b16988f15e5bd"}
Apr 28 19:25:20.630780 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:20.630780    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2a7b1df02c8364836546a2dd0a8085486468f31f1e884d07e8b16988f15e5bd"
Apr 28 19:25:20.630992 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:20.630804    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835qg5vc"
Apr 28 19:25:25.779758 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.779713    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"]
Apr 28 19:25:25.780103 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.780076    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f33c0ac4-b47d-43f4-9204-9bc31279f9ff" containerName="extract"
Apr 28 19:25:25.780103 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.780087    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f33c0ac4-b47d-43f4-9204-9bc31279f9ff" containerName="extract"
Apr 28 19:25:25.780103 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.780104    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f33c0ac4-b47d-43f4-9204-9bc31279f9ff" containerName="pull"
Apr 28 19:25:25.780216 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.780109    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f33c0ac4-b47d-43f4-9204-9bc31279f9ff" containerName="pull"
Apr 28 19:25:25.780216 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.780121    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f33c0ac4-b47d-43f4-9204-9bc31279f9ff" containerName="util"
Apr 28 19:25:25.780216 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.780126    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f33c0ac4-b47d-43f4-9204-9bc31279f9ff" containerName="util"
Apr 28 19:25:25.780216 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.780185    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="f33c0ac4-b47d-43f4-9204-9bc31279f9ff" containerName="extract"
Apr 28 19:25:25.782939 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.782920    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"
Apr 28 19:25:25.794766 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.794749    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 28 19:25:25.796498 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.796480    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 28 19:25:25.797615 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.797597    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nvrrr\""
Apr 28 19:25:25.814582 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.814556    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"]
Apr 28 19:25:25.822859 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.822840    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2be8f6d-738c-498b-9a3d-0e0afd194184-util\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf\" (UID: \"c2be8f6d-738c-498b-9a3d-0e0afd194184\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"
Apr 28 19:25:25.822959 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.822892    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2be8f6d-738c-498b-9a3d-0e0afd194184-bundle\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf\" (UID: \"c2be8f6d-738c-498b-9a3d-0e0afd194184\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"
Apr 28 19:25:25.823021 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.822973    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmdxm\" (UniqueName: \"kubernetes.io/projected/c2be8f6d-738c-498b-9a3d-0e0afd194184-kube-api-access-mmdxm\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf\" (UID: \"c2be8f6d-738c-498b-9a3d-0e0afd194184\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"
Apr 28 19:25:25.923902 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.923867    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mmdxm\" (UniqueName: \"kubernetes.io/projected/c2be8f6d-738c-498b-9a3d-0e0afd194184-kube-api-access-mmdxm\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf\" (UID: \"c2be8f6d-738c-498b-9a3d-0e0afd194184\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"
Apr 28 19:25:25.924068 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.923929    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2be8f6d-738c-498b-9a3d-0e0afd194184-util\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf\" (UID: \"c2be8f6d-738c-498b-9a3d-0e0afd194184\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"
Apr 28 19:25:25.924068 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.923976    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2be8f6d-738c-498b-9a3d-0e0afd194184-bundle\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf\" (UID: \"c2be8f6d-738c-498b-9a3d-0e0afd194184\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"
Apr 28 19:25:25.924281 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.924261    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2be8f6d-738c-498b-9a3d-0e0afd194184-util\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf\" (UID: \"c2be8f6d-738c-498b-9a3d-0e0afd194184\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"
Apr 28 19:25:25.924323 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.924279    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2be8f6d-738c-498b-9a3d-0e0afd194184-bundle\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf\" (UID: \"c2be8f6d-738c-498b-9a3d-0e0afd194184\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"
Apr 28 19:25:25.937604 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:25.937578    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmdxm\" (UniqueName: \"kubernetes.io/projected/c2be8f6d-738c-498b-9a3d-0e0afd194184-kube-api-access-mmdxm\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf\" (UID: \"c2be8f6d-738c-498b-9a3d-0e0afd194184\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"
Apr 28 19:25:26.092199 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:26.092129    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"
Apr 28 19:25:26.231108 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:26.231074    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"]
Apr 28 19:25:26.234224 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:25:26.234197    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2be8f6d_738c_498b_9a3d_0e0afd194184.slice/crio-c9f890fb3bba8390af12f815d80aa1b41d4828ed463cd8af717107f1a64cc731 WatchSource:0}: Error finding container c9f890fb3bba8390af12f815d80aa1b41d4828ed463cd8af717107f1a64cc731: Status 404 returned error can't find the container with id c9f890fb3bba8390af12f815d80aa1b41d4828ed463cd8af717107f1a64cc731
Apr 28 19:25:26.654259 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:26.654223    2569 generic.go:358] "Generic (PLEG): container finished" podID="c2be8f6d-738c-498b-9a3d-0e0afd194184" containerID="21363f29e2ab7e28d0210fdca354cd3ee3b6a9c078711462e1cd718e573013d5" exitCode=0
Apr 28 19:25:26.654454 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:26.654307    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf" event={"ID":"c2be8f6d-738c-498b-9a3d-0e0afd194184","Type":"ContainerDied","Data":"21363f29e2ab7e28d0210fdca354cd3ee3b6a9c078711462e1cd718e573013d5"}
Apr 28 19:25:26.654454 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:26.654345    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf" event={"ID":"c2be8f6d-738c-498b-9a3d-0e0afd194184","Type":"ContainerStarted","Data":"c9f890fb3bba8390af12f815d80aa1b41d4828ed463cd8af717107f1a64cc731"}
Apr 28 19:25:27.099351 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.099320    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd"]
Apr 28 19:25:27.101563 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.101547    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd"
Apr 28 19:25:27.104625 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.104605    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\""
Apr 28 19:25:27.104724 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.104635    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-9txd8\""
Apr 28 19:25:27.104930 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.104914    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\""
Apr 28 19:25:27.118365 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.118338    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd"]
Apr 28 19:25:27.135799 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.135774    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttjql\" (UniqueName: \"kubernetes.io/projected/78b8dca9-58b6-47f3-bb43-f26104e5f63e-kube-api-access-ttjql\") pod \"servicemesh-operator3-55f49c5f94-5c6rd\" (UID: \"78b8dca9-58b6-47f3-bb43-f26104e5f63e\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd"
Apr 28 19:25:27.135894 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.135875    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/78b8dca9-58b6-47f3-bb43-f26104e5f63e-operator-config\") pod \"servicemesh-operator3-55f49c5f94-5c6rd\" (UID: \"78b8dca9-58b6-47f3-bb43-f26104e5f63e\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd"
Apr 28 19:25:27.236885 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.236793    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/78b8dca9-58b6-47f3-bb43-f26104e5f63e-operator-config\") pod \"servicemesh-operator3-55f49c5f94-5c6rd\" (UID: \"78b8dca9-58b6-47f3-bb43-f26104e5f63e\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd"
Apr 28 19:25:27.236885 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.236864    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ttjql\" (UniqueName: \"kubernetes.io/projected/78b8dca9-58b6-47f3-bb43-f26104e5f63e-kube-api-access-ttjql\") pod \"servicemesh-operator3-55f49c5f94-5c6rd\" (UID: \"78b8dca9-58b6-47f3-bb43-f26104e5f63e\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd"
Apr 28 19:25:27.239191 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.239169    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/78b8dca9-58b6-47f3-bb43-f26104e5f63e-operator-config\") pod \"servicemesh-operator3-55f49c5f94-5c6rd\" (UID: \"78b8dca9-58b6-47f3-bb43-f26104e5f63e\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd"
Apr 28 19:25:27.255763 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.255735    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttjql\" (UniqueName: \"kubernetes.io/projected/78b8dca9-58b6-47f3-bb43-f26104e5f63e-kube-api-access-ttjql\") pod \"servicemesh-operator3-55f49c5f94-5c6rd\" (UID: \"78b8dca9-58b6-47f3-bb43-f26104e5f63e\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd"
Apr 28 19:25:27.410985 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.410954    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd"
Apr 28 19:25:27.547164 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.547077    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd"]
Apr 28 19:25:27.551093 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:25:27.551067    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78b8dca9_58b6_47f3_bb43_f26104e5f63e.slice/crio-b68ac65cf3fdcc9d262ba508e80aa78e368c98e35e941f890c8596ffdd81bbb6 WatchSource:0}: Error finding container b68ac65cf3fdcc9d262ba508e80aa78e368c98e35e941f890c8596ffdd81bbb6: Status 404 returned error can't find the container with id b68ac65cf3fdcc9d262ba508e80aa78e368c98e35e941f890c8596ffdd81bbb6
Apr 28 19:25:27.658979 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:27.658942    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd" event={"ID":"78b8dca9-58b6-47f3-bb43-f26104e5f63e","Type":"ContainerStarted","Data":"b68ac65cf3fdcc9d262ba508e80aa78e368c98e35e941f890c8596ffdd81bbb6"}
Apr 28 19:25:28.664749 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:28.664712    2569 generic.go:358] "Generic (PLEG): container finished" podID="c2be8f6d-738c-498b-9a3d-0e0afd194184" containerID="dd4a018f0ed47a0df92520c9967db207471a6b1670dce1c9824bf352c0beae66" exitCode=0
Apr 28 19:25:28.665124 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:28.664777    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf" event={"ID":"c2be8f6d-738c-498b-9a3d-0e0afd194184","Type":"ContainerDied","Data":"dd4a018f0ed47a0df92520c9967db207471a6b1670dce1c9824bf352c0beae66"}
Apr 28 19:25:29.670805 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:29.670765    2569 generic.go:358] "Generic (PLEG): container finished" podID="c2be8f6d-738c-498b-9a3d-0e0afd194184" containerID="944a4e41dfa61f8eccb2144d7565bbc4f012958b47487ab0dd6ca23970d870ec" exitCode=0
Apr 28 19:25:29.671253 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:29.670848    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf" event={"ID":"c2be8f6d-738c-498b-9a3d-0e0afd194184","Type":"ContainerDied","Data":"944a4e41dfa61f8eccb2144d7565bbc4f012958b47487ab0dd6ca23970d870ec"}
Apr 28 19:25:30.259317 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.258813    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-778b87f87-v9zfd"]
Apr 28 19:25:30.261741 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.261716    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.286113 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.286084    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-778b87f87-v9zfd"]
Apr 28 19:25:30.365032 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.364903    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8243766-5ac7-495c-af6d-85b6df87fae2-trusted-ca-bundle\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.365032 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.364968    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8243766-5ac7-495c-af6d-85b6df87fae2-service-ca\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.365252 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.365069    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8243766-5ac7-495c-af6d-85b6df87fae2-oauth-serving-cert\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.365252 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.365094    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcxl6\" (UniqueName: \"kubernetes.io/projected/f8243766-5ac7-495c-af6d-85b6df87fae2-kube-api-access-mcxl6\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.365252 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.365148    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8243766-5ac7-495c-af6d-85b6df87fae2-console-serving-cert\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.365252 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.365215    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8243766-5ac7-495c-af6d-85b6df87fae2-console-oauth-config\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.365440 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.365253    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8243766-5ac7-495c-af6d-85b6df87fae2-console-config\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.466728 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.466692    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8243766-5ac7-495c-af6d-85b6df87fae2-oauth-serving-cert\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.466728 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.466728    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mcxl6\" (UniqueName: \"kubernetes.io/projected/f8243766-5ac7-495c-af6d-85b6df87fae2-kube-api-access-mcxl6\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.466978 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.466763    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8243766-5ac7-495c-af6d-85b6df87fae2-console-serving-cert\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.466978 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.466812    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8243766-5ac7-495c-af6d-85b6df87fae2-console-oauth-config\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.466978 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.466840    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8243766-5ac7-495c-af6d-85b6df87fae2-console-config\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.466978 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.466881    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8243766-5ac7-495c-af6d-85b6df87fae2-trusted-ca-bundle\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.466978 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.466920    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8243766-5ac7-495c-af6d-85b6df87fae2-service-ca\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.467530 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.467502    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8243766-5ac7-495c-af6d-85b6df87fae2-oauth-serving-cert\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.467683 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.467537    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8243766-5ac7-495c-af6d-85b6df87fae2-console-config\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.467757 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.467722    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8243766-5ac7-495c-af6d-85b6df87fae2-service-ca\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.467879 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.467859    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8243766-5ac7-495c-af6d-85b6df87fae2-trusted-ca-bundle\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.469734 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.469710    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8243766-5ac7-495c-af6d-85b6df87fae2-console-serving-cert\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.469809 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.469796    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8243766-5ac7-495c-af6d-85b6df87fae2-console-oauth-config\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.498630 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.498603    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcxl6\" (UniqueName: \"kubernetes.io/projected/f8243766-5ac7-495c-af6d-85b6df87fae2-kube-api-access-mcxl6\") pod \"console-778b87f87-v9zfd\" (UID: \"f8243766-5ac7-495c-af6d-85b6df87fae2\") " pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.575751 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.575725    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:30.676324 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.676241    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd" event={"ID":"78b8dca9-58b6-47f3-bb43-f26104e5f63e","Type":"ContainerStarted","Data":"b8febfc3e706b2f941573c4ad23419fddd42a5a51ddb2007cd3a86e6442dcd6a"}
Apr 28 19:25:30.676732 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.676536    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd"
Apr 28 19:25:30.712807 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.712526    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd" podStartSLOduration=1.204960671 podStartE2EDuration="3.712508626s" podCreationTimestamp="2026-04-28 19:25:27 +0000 UTC" firstStartedPulling="2026-04-28 19:25:27.553613572 +0000 UTC m=+514.377953671" lastFinishedPulling="2026-04-28 19:25:30.06116154 +0000 UTC m=+516.885501626" observedRunningTime="2026-04-28 19:25:30.711643975 +0000 UTC m=+517.535984097" watchObservedRunningTime="2026-04-28 19:25:30.712508626 +0000 UTC m=+517.536848733"
Apr 28 19:25:30.719429 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.719275    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-778b87f87-v9zfd"]
Apr 28 19:25:30.722439 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:25:30.722394    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8243766_5ac7_495c_af6d_85b6df87fae2.slice/crio-f76f51f52f6b82d26a13b034867b49fee3d467e172eaf7f011deb8e97ccd30d1 WatchSource:0}: Error finding container f76f51f52f6b82d26a13b034867b49fee3d467e172eaf7f011deb8e97ccd30d1: Status 404 returned error can't find the container with id f76f51f52f6b82d26a13b034867b49fee3d467e172eaf7f011deb8e97ccd30d1
Apr 28 19:25:30.796813 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.796792    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"
Apr 28 19:25:30.870156 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.870123    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmdxm\" (UniqueName: \"kubernetes.io/projected/c2be8f6d-738c-498b-9a3d-0e0afd194184-kube-api-access-mmdxm\") pod \"c2be8f6d-738c-498b-9a3d-0e0afd194184\" (UID: \"c2be8f6d-738c-498b-9a3d-0e0afd194184\") "
Apr 28 19:25:30.870316 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.870223    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2be8f6d-738c-498b-9a3d-0e0afd194184-util\") pod \"c2be8f6d-738c-498b-9a3d-0e0afd194184\" (UID: \"c2be8f6d-738c-498b-9a3d-0e0afd194184\") "
Apr 28 19:25:30.870316 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.870252    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2be8f6d-738c-498b-9a3d-0e0afd194184-bundle\") pod \"c2be8f6d-738c-498b-9a3d-0e0afd194184\" (UID: \"c2be8f6d-738c-498b-9a3d-0e0afd194184\") "
Apr 28 19:25:30.871154 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.871122    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2be8f6d-738c-498b-9a3d-0e0afd194184-bundle" (OuterVolumeSpecName: "bundle") pod "c2be8f6d-738c-498b-9a3d-0e0afd194184" (UID: "c2be8f6d-738c-498b-9a3d-0e0afd194184"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:25:30.872636 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.872608    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2be8f6d-738c-498b-9a3d-0e0afd194184-kube-api-access-mmdxm" (OuterVolumeSpecName: "kube-api-access-mmdxm") pod "c2be8f6d-738c-498b-9a3d-0e0afd194184" (UID: "c2be8f6d-738c-498b-9a3d-0e0afd194184"). InnerVolumeSpecName "kube-api-access-mmdxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:25:30.877116 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.877061    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2be8f6d-738c-498b-9a3d-0e0afd194184-util" (OuterVolumeSpecName: "util") pod "c2be8f6d-738c-498b-9a3d-0e0afd194184" (UID: "c2be8f6d-738c-498b-9a3d-0e0afd194184"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:25:30.972095 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.972057    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmdxm\" (UniqueName: \"kubernetes.io/projected/c2be8f6d-738c-498b-9a3d-0e0afd194184-kube-api-access-mmdxm\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:30.972272 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.972110    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2be8f6d-738c-498b-9a3d-0e0afd194184-util\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:30.972272 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:30.972126    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2be8f6d-738c-498b-9a3d-0e0afd194184-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:31.682086 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:31.682052    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-778b87f87-v9zfd" event={"ID":"f8243766-5ac7-495c-af6d-85b6df87fae2","Type":"ContainerStarted","Data":"0e8da0a4e37177cf3a04f404f426ceb684ccb0d55e302527ebd81158490c0919"}
Apr 28 19:25:31.682086 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:31.682092    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-778b87f87-v9zfd" event={"ID":"f8243766-5ac7-495c-af6d-85b6df87fae2","Type":"ContainerStarted","Data":"f76f51f52f6b82d26a13b034867b49fee3d467e172eaf7f011deb8e97ccd30d1"}
Apr 28 19:25:31.683895 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:31.683873    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf"
Apr 28 19:25:31.683895 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:31.683880    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebjbdrf" event={"ID":"c2be8f6d-738c-498b-9a3d-0e0afd194184","Type":"ContainerDied","Data":"c9f890fb3bba8390af12f815d80aa1b41d4828ed463cd8af717107f1a64cc731"}
Apr 28 19:25:31.684084 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:31.683907    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9f890fb3bba8390af12f815d80aa1b41d4828ed463cd8af717107f1a64cc731"
Apr 28 19:25:31.721320 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:31.721274    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-778b87f87-v9zfd" podStartSLOduration=1.7212595560000001 podStartE2EDuration="1.721259556s" podCreationTimestamp="2026-04-28 19:25:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:25:31.719779989 +0000 UTC m=+518.544120096" watchObservedRunningTime="2026-04-28 19:25:31.721259556 +0000 UTC m=+518.545599665"
Apr 28 19:25:37.315798 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.315769    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"]
Apr 28 19:25:37.316272 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.316254    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2be8f6d-738c-498b-9a3d-0e0afd194184" containerName="util"
Apr 28 19:25:37.316342 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.316275    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2be8f6d-738c-498b-9a3d-0e0afd194184" containerName="util"
Apr 28 19:25:37.316342 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.316291    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2be8f6d-738c-498b-9a3d-0e0afd194184" containerName="pull"
Apr 28 19:25:37.316342 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.316299    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2be8f6d-738c-498b-9a3d-0e0afd194184" containerName="pull"
Apr 28 19:25:37.316342 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.316337    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2be8f6d-738c-498b-9a3d-0e0afd194184" containerName="extract"
Apr 28 19:25:37.316576 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.316346    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2be8f6d-738c-498b-9a3d-0e0afd194184" containerName="extract"
Apr 28 19:25:37.316576 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.316453    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="c2be8f6d-738c-498b-9a3d-0e0afd194184" containerName="extract"
Apr 28 19:25:37.319114 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.319074    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.323200 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.323179    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Apr 28 19:25:37.323291 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.323223    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Apr 28 19:25:37.323343 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.323300    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Apr 28 19:25:37.323343 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.323299    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Apr 28 19:25:37.323453 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.323341    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"istio-ca-root-cert\""
Apr 28 19:25:37.323531 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.323478    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Apr 28 19:25:37.323911 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.323895    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-b9k8h\""
Apr 28 19:25:37.337781 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.337751    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"]
Apr 28 19:25:37.432535 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.432502    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.432702 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.432547    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dd59c0b-7182-4864-af7f-a93352e9ac8b-local-certs\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.432702 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.432571    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-cacerts\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.432702 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.432612    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.432702 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.432658    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.432702 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.432698    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-token\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.432860 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.432752    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whl4b\" (UniqueName: \"kubernetes.io/projected/1dd59c0b-7182-4864-af7f-a93352e9ac8b-kube-api-access-whl4b\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.534329 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.534295    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.534494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.534356    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dd59c0b-7182-4864-af7f-a93352e9ac8b-local-certs\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.534494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.534389    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-cacerts\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.534494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.534431    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.534494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.534463    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.534701 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.534495    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-token\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.534701 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.534583    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-whl4b\" (UniqueName: \"kubernetes.io/projected/1dd59c0b-7182-4864-af7f-a93352e9ac8b-kube-api-access-whl4b\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.535235 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.535210    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.536694 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.536673    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dd59c0b-7182-4864-af7f-a93352e9ac8b-local-certs\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.536930 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.536907    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.536930 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.536925    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-cacerts\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.537105 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.537086    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.546659 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.546632    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-token\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.547056 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.547040    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whl4b\" (UniqueName: \"kubernetes.io/projected/1dd59c0b-7182-4864-af7f-a93352e9ac8b-kube-api-access-whl4b\") pod \"istiod-openshift-gateway-7cd77c7ffd-2hngl\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") " pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.628954 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.628889    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:37.760165 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:37.760132    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"]
Apr 28 19:25:37.760792 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:25:37.760764    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dd59c0b_7182_4864_af7f_a93352e9ac8b.slice/crio-e585505b425a01937533b93cf52ceb0f8a542030699bc4b788de51090773f832 WatchSource:0}: Error finding container e585505b425a01937533b93cf52ceb0f8a542030699bc4b788de51090773f832: Status 404 returned error can't find the container with id e585505b425a01937533b93cf52ceb0f8a542030699bc4b788de51090773f832
Apr 28 19:25:38.712583 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:38.712549    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl" event={"ID":"1dd59c0b-7182-4864-af7f-a93352e9ac8b","Type":"ContainerStarted","Data":"e585505b425a01937533b93cf52ceb0f8a542030699bc4b788de51090773f832"}
Apr 28 19:25:40.562323 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:40.562288    2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Apr 28 19:25:40.562617 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:40.562353    2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Apr 28 19:25:40.575868 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:40.575839    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:40.575969 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:40.575885    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:40.580606 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:40.580588    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:40.723471 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:40.723433    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl" event={"ID":"1dd59c0b-7182-4864-af7f-a93352e9ac8b","Type":"ContainerStarted","Data":"222e68f5ceefa0525b5eee7c31382c106bcbb531957d318445e2a36cc68317ce"}
Apr 28 19:25:40.723639 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:40.723616    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:40.727410 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:40.727373    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-778b87f87-v9zfd"
Apr 28 19:25:40.745163 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:40.745104    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl" podStartSLOduration=0.945761291 podStartE2EDuration="3.74508805s" podCreationTimestamp="2026-04-28 19:25:37 +0000 UTC" firstStartedPulling="2026-04-28 19:25:37.762735443 +0000 UTC m=+524.587075529" lastFinishedPulling="2026-04-28 19:25:40.562062199 +0000 UTC m=+527.386402288" observedRunningTime="2026-04-28 19:25:40.744140787 +0000 UTC m=+527.568480932" watchObservedRunningTime="2026-04-28 19:25:40.74508805 +0000 UTC m=+527.569428160"
Apr 28 19:25:40.796022 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:40.795995    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8454c5f95f-gwr52"]
Apr 28 19:25:41.686817 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:41.686786    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-5c6rd"
Apr 28 19:25:41.728957 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:41.728925    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:25:43.822978 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:43.822899    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"]
Apr 28 19:25:43.825549 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:43.825525    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:43.832310 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:43.832290    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-gk86h\""
Apr 28 19:25:43.896234 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:43.896196    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"]
Apr 28 19:25:43.899113 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:43.899076    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-workload-socket\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:43.899265 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:43.899124    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-istio-data\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:43.899265 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:43.899163    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10308d5b-7938-424c-8136-95e5bac92f07-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:43.899265 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:43.899242    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10308d5b-7938-424c-8136-95e5bac92f07-istio-token\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:43.899454 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:43.899308    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-workload-certs\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:43.899454 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:43.899336    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-istio-envoy\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:43.899454 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:43.899359    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb6d6\" (UniqueName: \"kubernetes.io/projected/10308d5b-7938-424c-8136-95e5bac92f07-kube-api-access-rb6d6\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:43.899617 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:43.899488    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10308d5b-7938-424c-8136-95e5bac92f07-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:43.899617 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:43.899564    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-credential-socket\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.001054 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.001023    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-workload-certs\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.001541 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.001061    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-istio-envoy\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.001823 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.001795    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rb6d6\" (UniqueName: \"kubernetes.io/projected/10308d5b-7938-424c-8136-95e5bac92f07-kube-api-access-rb6d6\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.001920 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.001902    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10308d5b-7938-424c-8136-95e5bac92f07-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.001983 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.001964    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-credential-socket\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.002079 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.002063    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-workload-socket\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.002133 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.002096    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-istio-data\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.002186 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.002135    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10308d5b-7938-424c-8136-95e5bac92f07-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.002318 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.002200    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10308d5b-7938-424c-8136-95e5bac92f07-istio-token\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.005519 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.002545    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-workload-socket\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.005519 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.003014    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-istio-data\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.005519 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.003234    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10308d5b-7938-424c-8136-95e5bac92f07-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.005519 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.001904    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-workload-certs\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.005519 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.003557    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-credential-socket\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.005838 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.005682    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10308d5b-7938-424c-8136-95e5bac92f07-istio-envoy\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.008830 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.006606    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10308d5b-7938-424c-8136-95e5bac92f07-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.012850 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.012828    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb6d6\" (UniqueName: \"kubernetes.io/projected/10308d5b-7938-424c-8136-95e5bac92f07-kube-api-access-rb6d6\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.013084 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.013063    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10308d5b-7938-424c-8136-95e5bac92f07-istio-token\") pod \"openshift-ai-inference-openshift-default-7c5447bb76-2dmrp\" (UID: \"10308d5b-7938-424c-8136-95e5bac92f07\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.138900 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.138826    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:44.280248 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.280216    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"]
Apr 28 19:25:44.281246 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:25:44.281215    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10308d5b_7938_424c_8136_95e5bac92f07.slice/crio-2d446f8962826bb7fb13745f1df78f6b57a0e72c1f92db078fad10e83afddb0b WatchSource:0}: Error finding container 2d446f8962826bb7fb13745f1df78f6b57a0e72c1f92db078fad10e83afddb0b: Status 404 returned error can't find the container with id 2d446f8962826bb7fb13745f1df78f6b57a0e72c1f92db078fad10e83afddb0b
Apr 28 19:25:44.738376 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:44.738334    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp" event={"ID":"10308d5b-7938-424c-8136-95e5bac92f07","Type":"ContainerStarted","Data":"2d446f8962826bb7fb13745f1df78f6b57a0e72c1f92db078fad10e83afddb0b"}
Apr 28 19:25:46.877079 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:46.877043    2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Apr 28 19:25:46.877437 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:46.877111    2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Apr 28 19:25:46.877437 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:46.877141    2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Apr 28 19:25:47.752488 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:47.752453    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp" event={"ID":"10308d5b-7938-424c-8136-95e5bac92f07","Type":"ContainerStarted","Data":"9d244132b5b74299ac0b9033cb6ee8fcf7b17294ef1c5d220c7d260cceed6b02"}
Apr 28 19:25:47.778255 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:47.778209    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp" podStartSLOduration=2.184861354 podStartE2EDuration="4.778196685s" podCreationTimestamp="2026-04-28 19:25:43 +0000 UTC" firstStartedPulling="2026-04-28 19:25:44.283436343 +0000 UTC m=+531.107776429" lastFinishedPulling="2026-04-28 19:25:46.876771671 +0000 UTC m=+533.701111760" observedRunningTime="2026-04-28 19:25:47.777047309 +0000 UTC m=+534.601387418" watchObservedRunningTime="2026-04-28 19:25:47.778196685 +0000 UTC m=+534.602536790"
Apr 28 19:25:48.139494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:48.139391    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:48.144019 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:48.143995    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:48.756701 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:48.756671    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:48.757699 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:48.757682    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-2dmrp"
Apr 28 19:25:53.797556 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.797526    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"]
Apr 28 19:25:53.800534 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.800516    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"
Apr 28 19:25:53.803114 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.803092    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 28 19:25:53.803230 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.803161    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nvrrr\""
Apr 28 19:25:53.804067 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.804053    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 28 19:25:53.809638 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.809616    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"]
Apr 28 19:25:53.892034 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.891990    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"]
Apr 28 19:25:53.894614 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.894597    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"
Apr 28 19:25:53.898236 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.898213    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/591264b7-89fb-470d-a861-efd47d87b8ed-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc\" (UID: \"591264b7-89fb-470d-a861-efd47d87b8ed\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"
Apr 28 19:25:53.898342 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.898281    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/591264b7-89fb-470d-a861-efd47d87b8ed-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc\" (UID: \"591264b7-89fb-470d-a861-efd47d87b8ed\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"
Apr 28 19:25:53.898342 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.898324    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h765\" (UniqueName: \"kubernetes.io/projected/591264b7-89fb-470d-a861-efd47d87b8ed-kube-api-access-5h765\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc\" (UID: \"591264b7-89fb-470d-a861-efd47d87b8ed\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"
Apr 28 19:25:53.903821 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.903798    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"]
Apr 28 19:25:53.995216 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.995185    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"]
Apr 28 19:25:53.998032 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.998015    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"
Apr 28 19:25:53.999166 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.999143    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/591264b7-89fb-470d-a861-efd47d87b8ed-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc\" (UID: \"591264b7-89fb-470d-a861-efd47d87b8ed\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"
Apr 28 19:25:53.999243 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.999180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11ca737d-dd0e-4a74-b2f4-02aa35926db4-util\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k\" (UID: \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"
Apr 28 19:25:53.999243 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.999229    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhnvs\" (UniqueName: \"kubernetes.io/projected/11ca737d-dd0e-4a74-b2f4-02aa35926db4-kube-api-access-dhnvs\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k\" (UID: \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"
Apr 28 19:25:53.999321 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.999258    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/591264b7-89fb-470d-a861-efd47d87b8ed-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc\" (UID: \"591264b7-89fb-470d-a861-efd47d87b8ed\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"
Apr 28 19:25:53.999321 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.999284    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11ca737d-dd0e-4a74-b2f4-02aa35926db4-bundle\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k\" (UID: \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"
Apr 28 19:25:53.999415 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.999321    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5h765\" (UniqueName: \"kubernetes.io/projected/591264b7-89fb-470d-a861-efd47d87b8ed-kube-api-access-5h765\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc\" (UID: \"591264b7-89fb-470d-a861-efd47d87b8ed\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"
Apr 28 19:25:53.999556 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.999534    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/591264b7-89fb-470d-a861-efd47d87b8ed-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc\" (UID: \"591264b7-89fb-470d-a861-efd47d87b8ed\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"
Apr 28 19:25:53.999598 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:53.999563    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/591264b7-89fb-470d-a861-efd47d87b8ed-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc\" (UID: \"591264b7-89fb-470d-a861-efd47d87b8ed\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"
Apr 28 19:25:54.006956 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.006930    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"]
Apr 28 19:25:54.008696 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.008672    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h765\" (UniqueName: \"kubernetes.io/projected/591264b7-89fb-470d-a861-efd47d87b8ed-kube-api-access-5h765\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc\" (UID: \"591264b7-89fb-470d-a861-efd47d87b8ed\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"
Apr 28 19:25:54.100485 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.100429    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"]
Apr 28 19:25:54.100593 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.100574    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85bb7896-0ace-46cd-bbf4-7453385bd87f-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn\" (UID: \"85bb7896-0ace-46cd-bbf4-7453385bd87f\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"
Apr 28 19:25:54.100641 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.100603    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpkh2\" (UniqueName: \"kubernetes.io/projected/85bb7896-0ace-46cd-bbf4-7453385bd87f-kube-api-access-mpkh2\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn\" (UID: \"85bb7896-0ace-46cd-bbf4-7453385bd87f\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"
Apr 28 19:25:54.100686 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.100643    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85bb7896-0ace-46cd-bbf4-7453385bd87f-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn\" (UID: \"85bb7896-0ace-46cd-bbf4-7453385bd87f\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"
Apr 28 19:25:54.100761 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.100742    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11ca737d-dd0e-4a74-b2f4-02aa35926db4-util\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k\" (UID: \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"
Apr 28 19:25:54.100947 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.100811    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dhnvs\" (UniqueName: \"kubernetes.io/projected/11ca737d-dd0e-4a74-b2f4-02aa35926db4-kube-api-access-dhnvs\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k\" (UID: \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"
Apr 28 19:25:54.100947 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.100861    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11ca737d-dd0e-4a74-b2f4-02aa35926db4-bundle\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k\" (UID: \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"
Apr 28 19:25:54.101083 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.101065    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11ca737d-dd0e-4a74-b2f4-02aa35926db4-util\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k\" (UID: \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"
Apr 28 19:25:54.101222 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.101204    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11ca737d-dd0e-4a74-b2f4-02aa35926db4-bundle\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k\" (UID: \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"
Apr 28 19:25:54.103147 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.103133    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"
Apr 28 19:25:54.108915 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.108897    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhnvs\" (UniqueName: \"kubernetes.io/projected/11ca737d-dd0e-4a74-b2f4-02aa35926db4-kube-api-access-dhnvs\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k\" (UID: \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"
Apr 28 19:25:54.111689 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.111666    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"
Apr 28 19:25:54.112066 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.112041    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"]
Apr 28 19:25:54.202027 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.201998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj7mh\" (UniqueName: \"kubernetes.io/projected/a63b5257-d85e-4174-9649-208545d1c7bb-kube-api-access-hj7mh\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6\" (UID: \"a63b5257-d85e-4174-9649-208545d1c7bb\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"
Apr 28 19:25:54.202166 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.202142    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a63b5257-d85e-4174-9649-208545d1c7bb-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6\" (UID: \"a63b5257-d85e-4174-9649-208545d1c7bb\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"
Apr 28 19:25:54.202229 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.202197    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a63b5257-d85e-4174-9649-208545d1c7bb-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6\" (UID: \"a63b5257-d85e-4174-9649-208545d1c7bb\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"
Apr 28 19:25:54.202279 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.202265    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85bb7896-0ace-46cd-bbf4-7453385bd87f-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn\" (UID: \"85bb7896-0ace-46cd-bbf4-7453385bd87f\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"
Apr 28 19:25:54.202316 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.202298    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mpkh2\" (UniqueName: \"kubernetes.io/projected/85bb7896-0ace-46cd-bbf4-7453385bd87f-kube-api-access-mpkh2\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn\" (UID: \"85bb7896-0ace-46cd-bbf4-7453385bd87f\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"
Apr 28 19:25:54.202364 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.202349    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85bb7896-0ace-46cd-bbf4-7453385bd87f-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn\" (UID: \"85bb7896-0ace-46cd-bbf4-7453385bd87f\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"
Apr 28 19:25:54.202738 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.202712    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85bb7896-0ace-46cd-bbf4-7453385bd87f-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn\" (UID: \"85bb7896-0ace-46cd-bbf4-7453385bd87f\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"
Apr 28 19:25:54.202830 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.202749    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85bb7896-0ace-46cd-bbf4-7453385bd87f-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn\" (UID: \"85bb7896-0ace-46cd-bbf4-7453385bd87f\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"
Apr 28 19:25:54.204818 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.204801    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"
Apr 28 19:25:54.213082 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.213051    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpkh2\" (UniqueName: \"kubernetes.io/projected/85bb7896-0ace-46cd-bbf4-7453385bd87f-kube-api-access-mpkh2\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn\" (UID: \"85bb7896-0ace-46cd-bbf4-7453385bd87f\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"
Apr 28 19:25:54.239998 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.239965    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"]
Apr 28 19:25:54.240943 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:25:54.240906    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod591264b7_89fb_470d_a861_efd47d87b8ed.slice/crio-7cb499d925a49017740704b241cd48289432eb4cf2c5f06fb0e30615ee3e6d23 WatchSource:0}: Error finding container 7cb499d925a49017740704b241cd48289432eb4cf2c5f06fb0e30615ee3e6d23: Status 404 returned error can't find the container with id 7cb499d925a49017740704b241cd48289432eb4cf2c5f06fb0e30615ee3e6d23
Apr 28 19:25:54.304269 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.303536    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a63b5257-d85e-4174-9649-208545d1c7bb-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6\" (UID: \"a63b5257-d85e-4174-9649-208545d1c7bb\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"
Apr 28 19:25:54.304269 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.303978    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a63b5257-d85e-4174-9649-208545d1c7bb-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6\" (UID: \"a63b5257-d85e-4174-9649-208545d1c7bb\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"
Apr 28 19:25:54.304269 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.304202    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a63b5257-d85e-4174-9649-208545d1c7bb-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6\" (UID: \"a63b5257-d85e-4174-9649-208545d1c7bb\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"
Apr 28 19:25:54.304516 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.304348    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a63b5257-d85e-4174-9649-208545d1c7bb-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6\" (UID: \"a63b5257-d85e-4174-9649-208545d1c7bb\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"
Apr 28 19:25:54.305508 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.304679    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hj7mh\" (UniqueName: \"kubernetes.io/projected/a63b5257-d85e-4174-9649-208545d1c7bb-kube-api-access-hj7mh\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6\" (UID: \"a63b5257-d85e-4174-9649-208545d1c7bb\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"
Apr 28 19:25:54.314953 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.311930    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"
Apr 28 19:25:54.318674 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.318650    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj7mh\" (UniqueName: \"kubernetes.io/projected/a63b5257-d85e-4174-9649-208545d1c7bb-kube-api-access-hj7mh\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6\" (UID: \"a63b5257-d85e-4174-9649-208545d1c7bb\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"
Apr 28 19:25:54.345190 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.345162    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"]
Apr 28 19:25:54.389318 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:25:54.389278    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ca737d_dd0e_4a74_b2f4_02aa35926db4.slice/crio-a83dcd052424c669be703b8e1cdae35e23e5dd8c351369f41853cf92b254f38e WatchSource:0}: Error finding container a83dcd052424c669be703b8e1cdae35e23e5dd8c351369f41853cf92b254f38e: Status 404 returned error can't find the container with id a83dcd052424c669be703b8e1cdae35e23e5dd8c351369f41853cf92b254f38e
Apr 28 19:25:54.435258 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.435234    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"
Apr 28 19:25:54.447285 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.447248    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"]
Apr 28 19:25:54.448613 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:25:54.448586    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85bb7896_0ace_46cd_bbf4_7453385bd87f.slice/crio-a02cffcda2a0d392da3519d1a051388b8f4e0893a4cd92ee774e1571dec7d885 WatchSource:0}: Error finding container a02cffcda2a0d392da3519d1a051388b8f4e0893a4cd92ee774e1571dec7d885: Status 404 returned error can't find the container with id a02cffcda2a0d392da3519d1a051388b8f4e0893a4cd92ee774e1571dec7d885
Apr 28 19:25:54.588115 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.588088    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"]
Apr 28 19:25:54.621225 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:25:54.621161    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda63b5257_d85e_4174_9649_208545d1c7bb.slice/crio-34f86c328b7277c56a0e100a14c4294ab9c2a2ed4711bd4da2c99b313d8c0be3 WatchSource:0}: Error finding container 34f86c328b7277c56a0e100a14c4294ab9c2a2ed4711bd4da2c99b313d8c0be3: Status 404 returned error can't find the container with id 34f86c328b7277c56a0e100a14c4294ab9c2a2ed4711bd4da2c99b313d8c0be3
Apr 28 19:25:54.779364 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.779333    2569 generic.go:358] "Generic (PLEG): container finished" podID="11ca737d-dd0e-4a74-b2f4-02aa35926db4" containerID="2051caa3121e3d190b16ad78242d04b0a55db8c8f4e70db3a97c90bbc6d57c95" exitCode=0
Apr 28 19:25:54.779514 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.779436    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k" event={"ID":"11ca737d-dd0e-4a74-b2f4-02aa35926db4","Type":"ContainerDied","Data":"2051caa3121e3d190b16ad78242d04b0a55db8c8f4e70db3a97c90bbc6d57c95"}
Apr 28 19:25:54.779514 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.779467    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k" event={"ID":"11ca737d-dd0e-4a74-b2f4-02aa35926db4","Type":"ContainerStarted","Data":"a83dcd052424c669be703b8e1cdae35e23e5dd8c351369f41853cf92b254f38e"}
Apr 28 19:25:54.780831 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.780768    2569 generic.go:358] "Generic (PLEG): container finished" podID="85bb7896-0ace-46cd-bbf4-7453385bd87f" containerID="4218c68230cbfa14585b8a14f66cd72a57f0eb4196c8bba7a773cd1d1fe62bfc" exitCode=0
Apr 28 19:25:54.780904 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.780857    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn" event={"ID":"85bb7896-0ace-46cd-bbf4-7453385bd87f","Type":"ContainerDied","Data":"4218c68230cbfa14585b8a14f66cd72a57f0eb4196c8bba7a773cd1d1fe62bfc"}
Apr 28 19:25:54.780904 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.780888    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn" event={"ID":"85bb7896-0ace-46cd-bbf4-7453385bd87f","Type":"ContainerStarted","Data":"a02cffcda2a0d392da3519d1a051388b8f4e0893a4cd92ee774e1571dec7d885"}
Apr 28 19:25:54.782209 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.782190    2569 generic.go:358] "Generic (PLEG): container finished" podID="591264b7-89fb-470d-a861-efd47d87b8ed" containerID="3387df008048244309f18af93606b9482be0a40c6f7efc637befccc9bd2d227d" exitCode=0
Apr 28 19:25:54.782270 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.782251    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc" event={"ID":"591264b7-89fb-470d-a861-efd47d87b8ed","Type":"ContainerDied","Data":"3387df008048244309f18af93606b9482be0a40c6f7efc637befccc9bd2d227d"}
Apr 28 19:25:54.782316 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.782279    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc" event={"ID":"591264b7-89fb-470d-a861-efd47d87b8ed","Type":"ContainerStarted","Data":"7cb499d925a49017740704b241cd48289432eb4cf2c5f06fb0e30615ee3e6d23"}
Apr 28 19:25:54.783857 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.783830    2569 generic.go:358] "Generic (PLEG): container finished" podID="a63b5257-d85e-4174-9649-208545d1c7bb" containerID="bc80d410eddb1297d1121466b098b228998ce19d5fdd14cb5af928b100f3ec56" exitCode=0
Apr 28 19:25:54.783927 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.783889    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6" event={"ID":"a63b5257-d85e-4174-9649-208545d1c7bb","Type":"ContainerDied","Data":"bc80d410eddb1297d1121466b098b228998ce19d5fdd14cb5af928b100f3ec56"}
Apr 28 19:25:54.783927 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:54.783907    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6" event={"ID":"a63b5257-d85e-4174-9649-208545d1c7bb","Type":"ContainerStarted","Data":"34f86c328b7277c56a0e100a14c4294ab9c2a2ed4711bd4da2c99b313d8c0be3"}
Apr 28 19:25:55.790303 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:55.790274    2569 generic.go:358] "Generic (PLEG): container finished" podID="a63b5257-d85e-4174-9649-208545d1c7bb" containerID="0d23aafc06c95107c388d54dbba21c84d53c952b38adbe919102f0d681f6fb60" exitCode=0
Apr 28 19:25:55.790676 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:55.790339    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6" event={"ID":"a63b5257-d85e-4174-9649-208545d1c7bb","Type":"ContainerDied","Data":"0d23aafc06c95107c388d54dbba21c84d53c952b38adbe919102f0d681f6fb60"}
Apr 28 19:25:56.795425 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:56.795367    2569 generic.go:358] "Generic (PLEG): container finished" podID="a63b5257-d85e-4174-9649-208545d1c7bb" containerID="93408c3eb47afc8cbd4a74f553f936c7c9a1d66c29e2f32f0468b13ff155693a" exitCode=0
Apr 28 19:25:56.795834 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:56.795427    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6" event={"ID":"a63b5257-d85e-4174-9649-208545d1c7bb","Type":"ContainerDied","Data":"93408c3eb47afc8cbd4a74f553f936c7c9a1d66c29e2f32f0468b13ff155693a"}
Apr 28 19:25:56.796991 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:56.796968    2569 generic.go:358] "Generic (PLEG): container finished" podID="11ca737d-dd0e-4a74-b2f4-02aa35926db4" containerID="0d905979cc1f29d406aa61855af79b47008726406b4671cb1bf6512572c9bf1c" exitCode=0
Apr 28 19:25:56.797104 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:56.796994    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k" event={"ID":"11ca737d-dd0e-4a74-b2f4-02aa35926db4","Type":"ContainerDied","Data":"0d905979cc1f29d406aa61855af79b47008726406b4671cb1bf6512572c9bf1c"}
Apr 28 19:25:56.798676 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:56.798592    2569 generic.go:358] "Generic (PLEG): container finished" podID="85bb7896-0ace-46cd-bbf4-7453385bd87f" containerID="e27e51e34da69142a8cf619c815ba96bedfd00b8f795a443d29d01bd8b2c4e09" exitCode=0
Apr 28 19:25:56.798676 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:56.798619    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn" event={"ID":"85bb7896-0ace-46cd-bbf4-7453385bd87f","Type":"ContainerDied","Data":"e27e51e34da69142a8cf619c815ba96bedfd00b8f795a443d29d01bd8b2c4e09"}
Apr 28 19:25:56.800504 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:56.800472    2569 generic.go:358] "Generic (PLEG): container finished" podID="591264b7-89fb-470d-a861-efd47d87b8ed" containerID="642cced8e7040a96daad3e3aeb4bef595beac9508ecf5b9488cdb7a4d8f7ad75" exitCode=0
Apr 28 19:25:56.800504 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:56.800502    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc" event={"ID":"591264b7-89fb-470d-a861-efd47d87b8ed","Type":"ContainerDied","Data":"642cced8e7040a96daad3e3aeb4bef595beac9508ecf5b9488cdb7a4d8f7ad75"}
Apr 28 19:25:57.806003 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:57.805966    2569 generic.go:358] "Generic (PLEG): container finished" podID="591264b7-89fb-470d-a861-efd47d87b8ed" containerID="237371e5ede54e7f936077de044a6a2a8567670c86efabd1079aacd0cbc0ce8e" exitCode=0
Apr 28 19:25:57.806410 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:57.806049    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc" event={"ID":"591264b7-89fb-470d-a861-efd47d87b8ed","Type":"ContainerDied","Data":"237371e5ede54e7f936077de044a6a2a8567670c86efabd1079aacd0cbc0ce8e"}
Apr 28 19:25:57.807916 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:57.807890    2569 generic.go:358] "Generic (PLEG): container finished" podID="11ca737d-dd0e-4a74-b2f4-02aa35926db4" containerID="dcf06ee1ee83851049110b8567f706d615fd35ea4c187432592030b17cc45662" exitCode=0
Apr 28 19:25:57.808020 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:57.807978    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k" event={"ID":"11ca737d-dd0e-4a74-b2f4-02aa35926db4","Type":"ContainerDied","Data":"dcf06ee1ee83851049110b8567f706d615fd35ea4c187432592030b17cc45662"}
Apr 28 19:25:57.809601 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:57.809582    2569 generic.go:358] "Generic (PLEG): container finished" podID="85bb7896-0ace-46cd-bbf4-7453385bd87f" containerID="07551e98af94f2b220ac06984a68da0fe75604ba5a9fef73dc55c44375964536" exitCode=0
Apr 28 19:25:57.809710 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:57.809625    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn" event={"ID":"85bb7896-0ace-46cd-bbf4-7453385bd87f","Type":"ContainerDied","Data":"07551e98af94f2b220ac06984a68da0fe75604ba5a9fef73dc55c44375964536"}
Apr 28 19:25:57.937255 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:57.937233    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"
Apr 28 19:25:57.941559 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:57.941531    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a63b5257-d85e-4174-9649-208545d1c7bb-bundle\") pod \"a63b5257-d85e-4174-9649-208545d1c7bb\" (UID: \"a63b5257-d85e-4174-9649-208545d1c7bb\") "
Apr 28 19:25:57.941683 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:57.941576    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a63b5257-d85e-4174-9649-208545d1c7bb-util\") pod \"a63b5257-d85e-4174-9649-208545d1c7bb\" (UID: \"a63b5257-d85e-4174-9649-208545d1c7bb\") "
Apr 28 19:25:57.941683 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:57.941615    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj7mh\" (UniqueName: \"kubernetes.io/projected/a63b5257-d85e-4174-9649-208545d1c7bb-kube-api-access-hj7mh\") pod \"a63b5257-d85e-4174-9649-208545d1c7bb\" (UID: \"a63b5257-d85e-4174-9649-208545d1c7bb\") "
Apr 28 19:25:57.942075 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:57.942052    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a63b5257-d85e-4174-9649-208545d1c7bb-bundle" (OuterVolumeSpecName: "bundle") pod "a63b5257-d85e-4174-9649-208545d1c7bb" (UID: "a63b5257-d85e-4174-9649-208545d1c7bb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:25:57.943747 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:57.943722    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a63b5257-d85e-4174-9649-208545d1c7bb-kube-api-access-hj7mh" (OuterVolumeSpecName: "kube-api-access-hj7mh") pod "a63b5257-d85e-4174-9649-208545d1c7bb" (UID: "a63b5257-d85e-4174-9649-208545d1c7bb"). InnerVolumeSpecName "kube-api-access-hj7mh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:25:57.946951 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:57.946927    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a63b5257-d85e-4174-9649-208545d1c7bb-util" (OuterVolumeSpecName: "util") pod "a63b5257-d85e-4174-9649-208545d1c7bb" (UID: "a63b5257-d85e-4174-9649-208545d1c7bb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:25:58.043077 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:58.043047    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a63b5257-d85e-4174-9649-208545d1c7bb-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:58.043077 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:58.043076    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a63b5257-d85e-4174-9649-208545d1c7bb-util\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:58.043265 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:58.043090    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj7mh\" (UniqueName: \"kubernetes.io/projected/a63b5257-d85e-4174-9649-208545d1c7bb-kube-api-access-hj7mh\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:58.816496 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:58.816468    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6"
Apr 28 19:25:58.816956 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:58.816462    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30446q6" event={"ID":"a63b5257-d85e-4174-9649-208545d1c7bb","Type":"ContainerDied","Data":"34f86c328b7277c56a0e100a14c4294ab9c2a2ed4711bd4da2c99b313d8c0be3"}
Apr 28 19:25:58.816956 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:58.816577    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34f86c328b7277c56a0e100a14c4294ab9c2a2ed4711bd4da2c99b313d8c0be3"
Apr 28 19:25:58.956012 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:58.955988    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"
Apr 28 19:25:58.981015 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:58.980993    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"
Apr 28 19:25:59.003436 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.003416    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"
Apr 28 19:25:59.052875 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.052846    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85bb7896-0ace-46cd-bbf4-7453385bd87f-bundle\") pod \"85bb7896-0ace-46cd-bbf4-7453385bd87f\" (UID: \"85bb7896-0ace-46cd-bbf4-7453385bd87f\") "
Apr 28 19:25:59.053029 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.052931    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhnvs\" (UniqueName: \"kubernetes.io/projected/11ca737d-dd0e-4a74-b2f4-02aa35926db4-kube-api-access-dhnvs\") pod \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\" (UID: \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\") "
Apr 28 19:25:59.053029 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.052965    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5h765\" (UniqueName: \"kubernetes.io/projected/591264b7-89fb-470d-a861-efd47d87b8ed-kube-api-access-5h765\") pod \"591264b7-89fb-470d-a861-efd47d87b8ed\" (UID: \"591264b7-89fb-470d-a861-efd47d87b8ed\") "
Apr 28 19:25:59.053029 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.052989    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/591264b7-89fb-470d-a861-efd47d87b8ed-bundle\") pod \"591264b7-89fb-470d-a861-efd47d87b8ed\" (UID: \"591264b7-89fb-470d-a861-efd47d87b8ed\") "
Apr 28 19:25:59.053204 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.053155    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85bb7896-0ace-46cd-bbf4-7453385bd87f-util\") pod \"85bb7896-0ace-46cd-bbf4-7453385bd87f\" (UID: \"85bb7896-0ace-46cd-bbf4-7453385bd87f\") "
Apr 28 19:25:59.053204 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.053194    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/591264b7-89fb-470d-a861-efd47d87b8ed-util\") pod \"591264b7-89fb-470d-a861-efd47d87b8ed\" (UID: \"591264b7-89fb-470d-a861-efd47d87b8ed\") "
Apr 28 19:25:59.053304 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.053225    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11ca737d-dd0e-4a74-b2f4-02aa35926db4-util\") pod \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\" (UID: \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\") "
Apr 28 19:25:59.053304 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.053248    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11ca737d-dd0e-4a74-b2f4-02aa35926db4-bundle\") pod \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\" (UID: \"11ca737d-dd0e-4a74-b2f4-02aa35926db4\") "
Apr 28 19:25:59.053304 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.053274    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpkh2\" (UniqueName: \"kubernetes.io/projected/85bb7896-0ace-46cd-bbf4-7453385bd87f-kube-api-access-mpkh2\") pod \"85bb7896-0ace-46cd-bbf4-7453385bd87f\" (UID: \"85bb7896-0ace-46cd-bbf4-7453385bd87f\") "
Apr 28 19:25:59.055131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.053605    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/591264b7-89fb-470d-a861-efd47d87b8ed-bundle" (OuterVolumeSpecName: "bundle") pod "591264b7-89fb-470d-a861-efd47d87b8ed" (UID: "591264b7-89fb-470d-a861-efd47d87b8ed"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:25:59.055131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.053761    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85bb7896-0ace-46cd-bbf4-7453385bd87f-bundle" (OuterVolumeSpecName: "bundle") pod "85bb7896-0ace-46cd-bbf4-7453385bd87f" (UID: "85bb7896-0ace-46cd-bbf4-7453385bd87f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:25:59.055131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.054465    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11ca737d-dd0e-4a74-b2f4-02aa35926db4-bundle" (OuterVolumeSpecName: "bundle") pod "11ca737d-dd0e-4a74-b2f4-02aa35926db4" (UID: "11ca737d-dd0e-4a74-b2f4-02aa35926db4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:25:59.055664 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.055631    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/591264b7-89fb-470d-a861-efd47d87b8ed-kube-api-access-5h765" (OuterVolumeSpecName: "kube-api-access-5h765") pod "591264b7-89fb-470d-a861-efd47d87b8ed" (UID: "591264b7-89fb-470d-a861-efd47d87b8ed"). InnerVolumeSpecName "kube-api-access-5h765". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:25:59.055785 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.055702    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11ca737d-dd0e-4a74-b2f4-02aa35926db4-kube-api-access-dhnvs" (OuterVolumeSpecName: "kube-api-access-dhnvs") pod "11ca737d-dd0e-4a74-b2f4-02aa35926db4" (UID: "11ca737d-dd0e-4a74-b2f4-02aa35926db4"). InnerVolumeSpecName "kube-api-access-dhnvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:25:59.055935 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.055913    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85bb7896-0ace-46cd-bbf4-7453385bd87f-kube-api-access-mpkh2" (OuterVolumeSpecName: "kube-api-access-mpkh2") pod "85bb7896-0ace-46cd-bbf4-7453385bd87f" (UID: "85bb7896-0ace-46cd-bbf4-7453385bd87f"). InnerVolumeSpecName "kube-api-access-mpkh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:25:59.060000 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.059976    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85bb7896-0ace-46cd-bbf4-7453385bd87f-util" (OuterVolumeSpecName: "util") pod "85bb7896-0ace-46cd-bbf4-7453385bd87f" (UID: "85bb7896-0ace-46cd-bbf4-7453385bd87f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:25:59.061871 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.061850    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11ca737d-dd0e-4a74-b2f4-02aa35926db4-util" (OuterVolumeSpecName: "util") pod "11ca737d-dd0e-4a74-b2f4-02aa35926db4" (UID: "11ca737d-dd0e-4a74-b2f4-02aa35926db4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:25:59.062068 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.062054    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/591264b7-89fb-470d-a861-efd47d87b8ed-util" (OuterVolumeSpecName: "util") pod "591264b7-89fb-470d-a861-efd47d87b8ed" (UID: "591264b7-89fb-470d-a861-efd47d87b8ed"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:25:59.154390 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.154311    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhnvs\" (UniqueName: \"kubernetes.io/projected/11ca737d-dd0e-4a74-b2f4-02aa35926db4-kube-api-access-dhnvs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:59.154390 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.154338    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5h765\" (UniqueName: \"kubernetes.io/projected/591264b7-89fb-470d-a861-efd47d87b8ed-kube-api-access-5h765\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:59.154390 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.154349    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/591264b7-89fb-470d-a861-efd47d87b8ed-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:59.154390 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.154358    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85bb7896-0ace-46cd-bbf4-7453385bd87f-util\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:59.154390 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.154365    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/591264b7-89fb-470d-a861-efd47d87b8ed-util\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:59.154390 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.154372    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11ca737d-dd0e-4a74-b2f4-02aa35926db4-util\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:59.154390 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.154380    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11ca737d-dd0e-4a74-b2f4-02aa35926db4-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:59.154738 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.154388    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpkh2\" (UniqueName: \"kubernetes.io/projected/85bb7896-0ace-46cd-bbf4-7453385bd87f-kube-api-access-mpkh2\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:59.154738 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.154438    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85bb7896-0ace-46cd-bbf4-7453385bd87f-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:25:59.822134 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.822102    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc"
Apr 28 19:25:59.822134 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.822123    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88r4lsc" event={"ID":"591264b7-89fb-470d-a861-efd47d87b8ed","Type":"ContainerDied","Data":"7cb499d925a49017740704b241cd48289432eb4cf2c5f06fb0e30615ee3e6d23"}
Apr 28 19:25:59.822633 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.822154    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7cb499d925a49017740704b241cd48289432eb4cf2c5f06fb0e30615ee3e6d23"
Apr 28 19:25:59.823879 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.823855    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k"
Apr 28 19:25:59.823879 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.823863    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503j2g4k" event={"ID":"11ca737d-dd0e-4a74-b2f4-02aa35926db4","Type":"ContainerDied","Data":"a83dcd052424c669be703b8e1cdae35e23e5dd8c351369f41853cf92b254f38e"}
Apr 28 19:25:59.824080 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.823885    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a83dcd052424c669be703b8e1cdae35e23e5dd8c351369f41853cf92b254f38e"
Apr 28 19:25:59.825589 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.825567    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn" event={"ID":"85bb7896-0ace-46cd-bbf4-7453385bd87f","Type":"ContainerDied","Data":"a02cffcda2a0d392da3519d1a051388b8f4e0893a4cd92ee774e1571dec7d885"}
Apr 28 19:25:59.825589 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.825583    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bpljdn"
Apr 28 19:25:59.825763 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:25:59.825593    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a02cffcda2a0d392da3519d1a051388b8f4e0893a4cd92ee774e1571dec7d885"
Apr 28 19:26:05.817775 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:05.817710    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-8454c5f95f-gwr52" podUID="074c49c6-ef56-4a60-a1c8-94a8538f3ce0" containerName="console" containerID="cri-o://0cc871a626f3bbd566080ebeb0fd743487468ac9ec11f0747a57f423cdf07a98" gracePeriod=15
Apr 28 19:26:06.157557 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.157534    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8454c5f95f-gwr52_074c49c6-ef56-4a60-a1c8-94a8538f3ce0/console/0.log"
Apr 28 19:26:06.157657 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.157593    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:26:06.219993 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.219961    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-trusted-ca-bundle\") pod \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") "
Apr 28 19:26:06.220150 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.220043    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-oauth-serving-cert\") pod \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") "
Apr 28 19:26:06.220150 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.220062    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-service-ca\") pod \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") "
Apr 28 19:26:06.220150 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.220081    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-oauth-config\") pod \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") "
Apr 28 19:26:06.220150 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.220129    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-config\") pod \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") "
Apr 28 19:26:06.220530 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.220162    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ng42\" (UniqueName: \"kubernetes.io/projected/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-kube-api-access-6ng42\") pod \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") "
Apr 28 19:26:06.220530 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.220188    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-serving-cert\") pod \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\" (UID: \"074c49c6-ef56-4a60-a1c8-94a8538f3ce0\") "
Apr 28 19:26:06.220530 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.220359    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "074c49c6-ef56-4a60-a1c8-94a8538f3ce0" (UID: "074c49c6-ef56-4a60-a1c8-94a8538f3ce0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:26:06.220530 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.220523    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-trusted-ca-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:26:06.220679 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.220525    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "074c49c6-ef56-4a60-a1c8-94a8538f3ce0" (UID: "074c49c6-ef56-4a60-a1c8-94a8538f3ce0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:26:06.220679 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.220532    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-service-ca" (OuterVolumeSpecName: "service-ca") pod "074c49c6-ef56-4a60-a1c8-94a8538f3ce0" (UID: "074c49c6-ef56-4a60-a1c8-94a8538f3ce0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:26:06.220679 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.220584    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-config" (OuterVolumeSpecName: "console-config") pod "074c49c6-ef56-4a60-a1c8-94a8538f3ce0" (UID: "074c49c6-ef56-4a60-a1c8-94a8538f3ce0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:26:06.222261 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.222232    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "074c49c6-ef56-4a60-a1c8-94a8538f3ce0" (UID: "074c49c6-ef56-4a60-a1c8-94a8538f3ce0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:26:06.222261 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.222252    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "074c49c6-ef56-4a60-a1c8-94a8538f3ce0" (UID: "074c49c6-ef56-4a60-a1c8-94a8538f3ce0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:26:06.222392 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.222316    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-kube-api-access-6ng42" (OuterVolumeSpecName: "kube-api-access-6ng42") pod "074c49c6-ef56-4a60-a1c8-94a8538f3ce0" (UID: "074c49c6-ef56-4a60-a1c8-94a8538f3ce0"). InnerVolumeSpecName "kube-api-access-6ng42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:26:06.320998 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.320963    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-oauth-config\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:26:06.320998 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.320993    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-config\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:26:06.320998 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.321003    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ng42\" (UniqueName: \"kubernetes.io/projected/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-kube-api-access-6ng42\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:26:06.321216 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.321013    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-console-serving-cert\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:26:06.321216 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.321023    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-oauth-serving-cert\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:26:06.321216 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.321031    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/074c49c6-ef56-4a60-a1c8-94a8538f3ce0-service-ca\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:26:06.676351 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676318    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-qrpft"]
Apr 28 19:26:06.676724 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676711    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85bb7896-0ace-46cd-bbf4-7453385bd87f" containerName="util"
Apr 28 19:26:06.676782 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676726    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="85bb7896-0ace-46cd-bbf4-7453385bd87f" containerName="util"
Apr 28 19:26:06.676782 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676737    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a63b5257-d85e-4174-9649-208545d1c7bb" containerName="extract"
Apr 28 19:26:06.676782 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676742    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a63b5257-d85e-4174-9649-208545d1c7bb" containerName="extract"
Apr 28 19:26:06.676782 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676748    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85bb7896-0ace-46cd-bbf4-7453385bd87f" containerName="extract"
Apr 28 19:26:06.676782 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676753    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="85bb7896-0ace-46cd-bbf4-7453385bd87f" containerName="extract"
Apr 28 19:26:06.676782 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676762    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="074c49c6-ef56-4a60-a1c8-94a8538f3ce0" containerName="console"
Apr 28 19:26:06.676782 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676767    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="074c49c6-ef56-4a60-a1c8-94a8538f3ce0" containerName="console"
Apr 28 19:26:06.676782 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676776    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="591264b7-89fb-470d-a861-efd47d87b8ed" containerName="pull"
Apr 28 19:26:06.676782 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676781    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="591264b7-89fb-470d-a861-efd47d87b8ed" containerName="pull"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676788    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="591264b7-89fb-470d-a861-efd47d87b8ed" containerName="util"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676793    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="591264b7-89fb-470d-a861-efd47d87b8ed" containerName="util"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676799    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a63b5257-d85e-4174-9649-208545d1c7bb" containerName="pull"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676804    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a63b5257-d85e-4174-9649-208545d1c7bb" containerName="pull"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676809    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11ca737d-dd0e-4a74-b2f4-02aa35926db4" containerName="pull"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676814    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="11ca737d-dd0e-4a74-b2f4-02aa35926db4" containerName="pull"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676821    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a63b5257-d85e-4174-9649-208545d1c7bb" containerName="util"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676826    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a63b5257-d85e-4174-9649-208545d1c7bb" containerName="util"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676832    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11ca737d-dd0e-4a74-b2f4-02aa35926db4" containerName="util"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676837    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="11ca737d-dd0e-4a74-b2f4-02aa35926db4" containerName="util"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676841    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11ca737d-dd0e-4a74-b2f4-02aa35926db4" containerName="extract"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676846    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="11ca737d-dd0e-4a74-b2f4-02aa35926db4" containerName="extract"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676854    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="591264b7-89fb-470d-a861-efd47d87b8ed" containerName="extract"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676860    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="591264b7-89fb-470d-a861-efd47d87b8ed" containerName="extract"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676869    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85bb7896-0ace-46cd-bbf4-7453385bd87f" containerName="pull"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676873    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="85bb7896-0ace-46cd-bbf4-7453385bd87f" containerName="pull"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676931    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="a63b5257-d85e-4174-9649-208545d1c7bb" containerName="extract"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676939    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="591264b7-89fb-470d-a861-efd47d87b8ed" containerName="extract"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676944    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="11ca737d-dd0e-4a74-b2f4-02aa35926db4" containerName="extract"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676951    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="85bb7896-0ace-46cd-bbf4-7453385bd87f" containerName="extract"
Apr 28 19:26:06.677039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.676959    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="074c49c6-ef56-4a60-a1c8-94a8538f3ce0" containerName="console"
Apr 28 19:26:06.679953 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.679938    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7587b89b76-qrpft"
Apr 28 19:26:06.683583 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.683562    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-gk8r6\""
Apr 28 19:26:06.683711 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.683589    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Apr 28 19:26:06.683960 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.683942    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Apr 28 19:26:06.693680 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.693657    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-qrpft"]
Apr 28 19:26:06.724335 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.724307    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkz27\" (UniqueName: \"kubernetes.io/projected/21d8f832-2d09-4067-af3c-56fcf6e4d485-kube-api-access-pkz27\") pod \"authorino-operator-7587b89b76-qrpft\" (UID: \"21d8f832-2d09-4067-af3c-56fcf6e4d485\") " pod="kuadrant-system/authorino-operator-7587b89b76-qrpft"
Apr 28 19:26:06.825897 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.825858    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pkz27\" (UniqueName: \"kubernetes.io/projected/21d8f832-2d09-4067-af3c-56fcf6e4d485-kube-api-access-pkz27\") pod \"authorino-operator-7587b89b76-qrpft\" (UID: \"21d8f832-2d09-4067-af3c-56fcf6e4d485\") " pod="kuadrant-system/authorino-operator-7587b89b76-qrpft"
Apr 28 19:26:06.843056 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.843029    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkz27\" (UniqueName: \"kubernetes.io/projected/21d8f832-2d09-4067-af3c-56fcf6e4d485-kube-api-access-pkz27\") pod \"authorino-operator-7587b89b76-qrpft\" (UID: \"21d8f832-2d09-4067-af3c-56fcf6e4d485\") " pod="kuadrant-system/authorino-operator-7587b89b76-qrpft"
Apr 28 19:26:06.854955 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.854932    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8454c5f95f-gwr52_074c49c6-ef56-4a60-a1c8-94a8538f3ce0/console/0.log"
Apr 28 19:26:06.855095 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.854970    2569 generic.go:358] "Generic (PLEG): container finished" podID="074c49c6-ef56-4a60-a1c8-94a8538f3ce0" containerID="0cc871a626f3bbd566080ebeb0fd743487468ac9ec11f0747a57f423cdf07a98" exitCode=2
Apr 28 19:26:06.855095 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.855053    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8454c5f95f-gwr52"
Apr 28 19:26:06.855095 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.855071    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8454c5f95f-gwr52" event={"ID":"074c49c6-ef56-4a60-a1c8-94a8538f3ce0","Type":"ContainerDied","Data":"0cc871a626f3bbd566080ebeb0fd743487468ac9ec11f0747a57f423cdf07a98"}
Apr 28 19:26:06.855239 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.855117    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8454c5f95f-gwr52" event={"ID":"074c49c6-ef56-4a60-a1c8-94a8538f3ce0","Type":"ContainerDied","Data":"39f8e361e4ec66a3fc0ab545cee8dd5b6f76216a2f05db5b501bdb3686e07695"}
Apr 28 19:26:06.855239 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.855139    2569 scope.go:117] "RemoveContainer" containerID="0cc871a626f3bbd566080ebeb0fd743487468ac9ec11f0747a57f423cdf07a98"
Apr 28 19:26:06.863935 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.863915    2569 scope.go:117] "RemoveContainer" containerID="0cc871a626f3bbd566080ebeb0fd743487468ac9ec11f0747a57f423cdf07a98"
Apr 28 19:26:06.864189 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:26:06.864171    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cc871a626f3bbd566080ebeb0fd743487468ac9ec11f0747a57f423cdf07a98\": container with ID starting with 0cc871a626f3bbd566080ebeb0fd743487468ac9ec11f0747a57f423cdf07a98 not found: ID does not exist" containerID="0cc871a626f3bbd566080ebeb0fd743487468ac9ec11f0747a57f423cdf07a98"
Apr 28 19:26:06.864250 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.864198    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cc871a626f3bbd566080ebeb0fd743487468ac9ec11f0747a57f423cdf07a98"} err="failed to get container status \"0cc871a626f3bbd566080ebeb0fd743487468ac9ec11f0747a57f423cdf07a98\": rpc error: code = NotFound desc = could not find container \"0cc871a626f3bbd566080ebeb0fd743487468ac9ec11f0747a57f423cdf07a98\": container with ID starting with 0cc871a626f3bbd566080ebeb0fd743487468ac9ec11f0747a57f423cdf07a98 not found: ID does not exist"
Apr 28 19:26:06.908767 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.908731    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8454c5f95f-gwr52"]
Apr 28 19:26:06.931231 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.931208    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8454c5f95f-gwr52"]
Apr 28 19:26:06.989671 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:06.989637    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7587b89b76-qrpft"
Apr 28 19:26:07.120303 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:07.120280    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-qrpft"]
Apr 28 19:26:07.122606 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:26:07.122579    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d8f832_2d09_4067_af3c_56fcf6e4d485.slice/crio-3660240f9484bc5860e475fb20375366e950213629663f7ad6b1e11fe6dd0508 WatchSource:0}: Error finding container 3660240f9484bc5860e475fb20375366e950213629663f7ad6b1e11fe6dd0508: Status 404 returned error can't find the container with id 3660240f9484bc5860e475fb20375366e950213629663f7ad6b1e11fe6dd0508
Apr 28 19:26:07.860470 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:07.860427    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7587b89b76-qrpft" event={"ID":"21d8f832-2d09-4067-af3c-56fcf6e4d485","Type":"ContainerStarted","Data":"3660240f9484bc5860e475fb20375366e950213629663f7ad6b1e11fe6dd0508"}
Apr 28 19:26:07.945304 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:07.945271    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="074c49c6-ef56-4a60-a1c8-94a8538f3ce0" path="/var/lib/kubelet/pods/074c49c6-ef56-4a60-a1c8-94a8538f3ce0/volumes"
Apr 28 19:26:09.871449 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:09.871338    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7587b89b76-qrpft" event={"ID":"21d8f832-2d09-4067-af3c-56fcf6e4d485","Type":"ContainerStarted","Data":"a532ec6238bd8bc290a4041ab28c966dd730090c42416569789704e318078f44"}
Apr 28 19:26:09.871449 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:09.871441    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7587b89b76-qrpft"
Apr 28 19:26:09.891048 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:09.890972    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7587b89b76-qrpft" podStartSLOduration=1.544772423 podStartE2EDuration="3.89095864s" podCreationTimestamp="2026-04-28 19:26:06 +0000 UTC" firstStartedPulling="2026-04-28 19:26:07.125085419 +0000 UTC m=+553.949425508" lastFinishedPulling="2026-04-28 19:26:09.471271638 +0000 UTC m=+556.295611725" observedRunningTime="2026-04-28 19:26:09.888955553 +0000 UTC m=+556.713295702" watchObservedRunningTime="2026-04-28 19:26:09.89095864 +0000 UTC m=+556.715298748"
Apr 28 19:26:15.007395 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:15.007360    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg"]
Apr 28 19:26:15.009793 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:15.009778    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg"
Apr 28 19:26:15.012583 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:15.012560    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kkx4f\""
Apr 28 19:26:15.024295 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:15.024268    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg"]
Apr 28 19:26:15.105021 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:15.104995    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkw8v\" (UniqueName: \"kubernetes.io/projected/c314cdb3-b328-4177-9e72-6910aaeeed25-kube-api-access-mkw8v\") pod \"kuadrant-operator-controller-manager-6ddf9554fc-6qqhg\" (UID: \"c314cdb3-b328-4177-9e72-6910aaeeed25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg"
Apr 28 19:26:15.105141 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:15.105029    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c314cdb3-b328-4177-9e72-6910aaeeed25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6ddf9554fc-6qqhg\" (UID: \"c314cdb3-b328-4177-9e72-6910aaeeed25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg"
Apr 28 19:26:15.205997 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:15.205972    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mkw8v\" (UniqueName: \"kubernetes.io/projected/c314cdb3-b328-4177-9e72-6910aaeeed25-kube-api-access-mkw8v\") pod \"kuadrant-operator-controller-manager-6ddf9554fc-6qqhg\" (UID: \"c314cdb3-b328-4177-9e72-6910aaeeed25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg"
Apr 28 19:26:15.206108 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:15.206006    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c314cdb3-b328-4177-9e72-6910aaeeed25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6ddf9554fc-6qqhg\" (UID: \"c314cdb3-b328-4177-9e72-6910aaeeed25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg"
Apr 28 19:26:15.206374 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:15.206355    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c314cdb3-b328-4177-9e72-6910aaeeed25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6ddf9554fc-6qqhg\" (UID: \"c314cdb3-b328-4177-9e72-6910aaeeed25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg"
Apr 28 19:26:15.240370 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:15.240340    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkw8v\" (UniqueName: \"kubernetes.io/projected/c314cdb3-b328-4177-9e72-6910aaeeed25-kube-api-access-mkw8v\") pod \"kuadrant-operator-controller-manager-6ddf9554fc-6qqhg\" (UID: \"c314cdb3-b328-4177-9e72-6910aaeeed25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg"
Apr 28 19:26:15.319993 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:15.319938    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg"
Apr 28 19:26:15.659506 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:26:15.659474    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc314cdb3_b328_4177_9e72_6910aaeeed25.slice/crio-c6d71e027b4c359a5ff93fec681447ac1b0398510f40c1cae44682a8fa7fb289 WatchSource:0}: Error finding container c6d71e027b4c359a5ff93fec681447ac1b0398510f40c1cae44682a8fa7fb289: Status 404 returned error can't find the container with id c6d71e027b4c359a5ff93fec681447ac1b0398510f40c1cae44682a8fa7fb289
Apr 28 19:26:15.659675 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:15.659653    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg"]
Apr 28 19:26:15.895903 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:15.895868    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg" event={"ID":"c314cdb3-b328-4177-9e72-6910aaeeed25","Type":"ContainerStarted","Data":"c6d71e027b4c359a5ff93fec681447ac1b0398510f40c1cae44682a8fa7fb289"}
Apr 28 19:26:20.878085 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:20.878053    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7587b89b76-qrpft"
Apr 28 19:26:20.918622 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:20.918533    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg" event={"ID":"c314cdb3-b328-4177-9e72-6910aaeeed25","Type":"ContainerStarted","Data":"f9e9856c1ebc2c18211df4ca3ff16edc7e51222cfef8b506bca53e3a927ab285"}
Apr 28 19:26:20.918622 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:20.918603    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg"
Apr 28 19:26:20.941149 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:20.941098    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg" podStartSLOduration=1.951236837 podStartE2EDuration="6.94107862s" podCreationTimestamp="2026-04-28 19:26:14 +0000 UTC" firstStartedPulling="2026-04-28 19:26:15.661809515 +0000 UTC m=+562.486149601" lastFinishedPulling="2026-04-28 19:26:20.651651298 +0000 UTC m=+567.475991384" observedRunningTime="2026-04-28 19:26:20.938802853 +0000 UTC m=+567.763142976" watchObservedRunningTime="2026-04-28 19:26:20.94107862 +0000 UTC m=+567.765418743"
Apr 28 19:26:31.924607 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:31.924574    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-6qqhg"
Apr 28 19:26:53.848873 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:53.848843    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:26:53.849447 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:53.849023    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:26:53.851837 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:53.851813    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:26:53.852071 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:26:53.852055    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:27:04.883194 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:04.883153    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-674b59b84c-sncn5"]
Apr 28 19:27:04.886534 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:04.886516    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-sncn5"
Apr 28 19:27:04.889992 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:04.889964    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m9msd\""
Apr 28 19:27:04.897373 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:04.897350    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674b59b84c-sncn5"]
Apr 28 19:27:05.046560 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:05.046525    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjhqk\" (UniqueName: \"kubernetes.io/projected/82b71598-0578-4c01-8594-c5fd0487e16e-kube-api-access-wjhqk\") pod \"authorino-674b59b84c-sncn5\" (UID: \"82b71598-0578-4c01-8594-c5fd0487e16e\") " pod="kuadrant-system/authorino-674b59b84c-sncn5"
Apr 28 19:27:05.147142 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:05.147059    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wjhqk\" (UniqueName: \"kubernetes.io/projected/82b71598-0578-4c01-8594-c5fd0487e16e-kube-api-access-wjhqk\") pod \"authorino-674b59b84c-sncn5\" (UID: \"82b71598-0578-4c01-8594-c5fd0487e16e\") " pod="kuadrant-system/authorino-674b59b84c-sncn5"
Apr 28 19:27:05.155078 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:05.155054    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjhqk\" (UniqueName: \"kubernetes.io/projected/82b71598-0578-4c01-8594-c5fd0487e16e-kube-api-access-wjhqk\") pod \"authorino-674b59b84c-sncn5\" (UID: \"82b71598-0578-4c01-8594-c5fd0487e16e\") " pod="kuadrant-system/authorino-674b59b84c-sncn5"
Apr 28 19:27:05.197732 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:05.197708    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-sncn5"
Apr 28 19:27:05.319046 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:05.319013    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674b59b84c-sncn5"]
Apr 28 19:27:05.320588 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:27:05.320546    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82b71598_0578_4c01_8594_c5fd0487e16e.slice/crio-7c510c7f937d63905dd2087c7ecd16bb69d95a24673dde98e7c642ba1a17957c WatchSource:0}: Error finding container 7c510c7f937d63905dd2087c7ecd16bb69d95a24673dde98e7c642ba1a17957c: Status 404 returned error can't find the container with id 7c510c7f937d63905dd2087c7ecd16bb69d95a24673dde98e7c642ba1a17957c
Apr 28 19:27:06.085442 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:06.085383    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-sncn5" event={"ID":"82b71598-0578-4c01-8594-c5fd0487e16e","Type":"ContainerStarted","Data":"7c510c7f937d63905dd2087c7ecd16bb69d95a24673dde98e7c642ba1a17957c"}
Apr 28 19:27:10.072869 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:10.072838    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-674b59b84c-sncn5"]
Apr 28 19:27:10.103708 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:10.103678    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-sncn5" event={"ID":"82b71598-0578-4c01-8594-c5fd0487e16e","Type":"ContainerStarted","Data":"b8d5e7ffcfa27baa9072c0230ce9101a8e0fc81b42d6c8b5a85ab7c234a6bb28"}
Apr 28 19:27:10.130480 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:10.130431    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-674b59b84c-sncn5" podStartSLOduration=2.139669733 podStartE2EDuration="6.130381984s" podCreationTimestamp="2026-04-28 19:27:04 +0000 UTC" firstStartedPulling="2026-04-28 19:27:05.321737615 +0000 UTC m=+612.146077701" lastFinishedPulling="2026-04-28 19:27:09.312449862 +0000 UTC m=+616.136789952" observedRunningTime="2026-04-28 19:27:10.129535382 +0000 UTC m=+616.953875491" watchObservedRunningTime="2026-04-28 19:27:10.130381984 +0000 UTC m=+616.954722092"
Apr 28 19:27:11.107345 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:11.107308    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-674b59b84c-sncn5" podUID="82b71598-0578-4c01-8594-c5fd0487e16e" containerName="authorino" containerID="cri-o://b8d5e7ffcfa27baa9072c0230ce9101a8e0fc81b42d6c8b5a85ab7c234a6bb28" gracePeriod=30
Apr 28 19:27:11.348476 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:11.348451    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-sncn5"
Apr 28 19:27:11.503235 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:11.503203    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjhqk\" (UniqueName: \"kubernetes.io/projected/82b71598-0578-4c01-8594-c5fd0487e16e-kube-api-access-wjhqk\") pod \"82b71598-0578-4c01-8594-c5fd0487e16e\" (UID: \"82b71598-0578-4c01-8594-c5fd0487e16e\") "
Apr 28 19:27:11.505288 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:11.505265    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82b71598-0578-4c01-8594-c5fd0487e16e-kube-api-access-wjhqk" (OuterVolumeSpecName: "kube-api-access-wjhqk") pod "82b71598-0578-4c01-8594-c5fd0487e16e" (UID: "82b71598-0578-4c01-8594-c5fd0487e16e"). InnerVolumeSpecName "kube-api-access-wjhqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:27:11.604890 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:11.604852    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjhqk\" (UniqueName: \"kubernetes.io/projected/82b71598-0578-4c01-8594-c5fd0487e16e-kube-api-access-wjhqk\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:27:12.112263 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:12.112225    2569 generic.go:358] "Generic (PLEG): container finished" podID="82b71598-0578-4c01-8594-c5fd0487e16e" containerID="b8d5e7ffcfa27baa9072c0230ce9101a8e0fc81b42d6c8b5a85ab7c234a6bb28" exitCode=0
Apr 28 19:27:12.112263 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:12.112266    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-sncn5" event={"ID":"82b71598-0578-4c01-8594-c5fd0487e16e","Type":"ContainerDied","Data":"b8d5e7ffcfa27baa9072c0230ce9101a8e0fc81b42d6c8b5a85ab7c234a6bb28"}
Apr 28 19:27:12.112784 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:12.112273    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-sncn5"
Apr 28 19:27:12.112784 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:12.112301    2569 scope.go:117] "RemoveContainer" containerID="b8d5e7ffcfa27baa9072c0230ce9101a8e0fc81b42d6c8b5a85ab7c234a6bb28"
Apr 28 19:27:12.112784 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:12.112291    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-sncn5" event={"ID":"82b71598-0578-4c01-8594-c5fd0487e16e","Type":"ContainerDied","Data":"7c510c7f937d63905dd2087c7ecd16bb69d95a24673dde98e7c642ba1a17957c"}
Apr 28 19:27:12.120828 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:12.120809    2569 scope.go:117] "RemoveContainer" containerID="b8d5e7ffcfa27baa9072c0230ce9101a8e0fc81b42d6c8b5a85ab7c234a6bb28"
Apr 28 19:27:12.121078 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:27:12.121059    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8d5e7ffcfa27baa9072c0230ce9101a8e0fc81b42d6c8b5a85ab7c234a6bb28\": container with ID starting with b8d5e7ffcfa27baa9072c0230ce9101a8e0fc81b42d6c8b5a85ab7c234a6bb28 not found: ID does not exist" containerID="b8d5e7ffcfa27baa9072c0230ce9101a8e0fc81b42d6c8b5a85ab7c234a6bb28"
Apr 28 19:27:12.121123 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:12.121087    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8d5e7ffcfa27baa9072c0230ce9101a8e0fc81b42d6c8b5a85ab7c234a6bb28"} err="failed to get container status \"b8d5e7ffcfa27baa9072c0230ce9101a8e0fc81b42d6c8b5a85ab7c234a6bb28\": rpc error: code = NotFound desc = could not find container \"b8d5e7ffcfa27baa9072c0230ce9101a8e0fc81b42d6c8b5a85ab7c234a6bb28\": container with ID starting with b8d5e7ffcfa27baa9072c0230ce9101a8e0fc81b42d6c8b5a85ab7c234a6bb28 not found: ID does not exist"
Apr 28 19:27:12.132670 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:12.132646    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-674b59b84c-sncn5"]
Apr 28 19:27:12.136636 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:12.136613    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-674b59b84c-sncn5"]
Apr 28 19:27:13.944550 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:13.944514    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82b71598-0578-4c01-8594-c5fd0487e16e" path="/var/lib/kubelet/pods/82b71598-0578-4c01-8594-c5fd0487e16e/volumes"
Apr 28 19:27:27.005700 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.005667    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68bd676465-gslhk"]
Apr 28 19:27:27.006144 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.006061    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82b71598-0578-4c01-8594-c5fd0487e16e" containerName="authorino"
Apr 28 19:27:27.006144 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.006073    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="82b71598-0578-4c01-8594-c5fd0487e16e" containerName="authorino"
Apr 28 19:27:27.006144 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.006138    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="82b71598-0578-4c01-8594-c5fd0487e16e" containerName="authorino"
Apr 28 19:27:27.009084 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.009067    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68bd676465-gslhk"
Apr 28 19:27:27.012640 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.012612    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Apr 28 19:27:27.012640 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.012635    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m9msd\""
Apr 28 19:27:27.013920 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.013902    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68bd676465-gslhk"]
Apr 28 19:27:27.042594 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.042555    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svr5v\" (UniqueName: \"kubernetes.io/projected/b1410653-ac40-400a-82c8-c26c0ce1ea45-kube-api-access-svr5v\") pod \"authorino-68bd676465-gslhk\" (UID: \"b1410653-ac40-400a-82c8-c26c0ce1ea45\") " pod="kuadrant-system/authorino-68bd676465-gslhk"
Apr 28 19:27:27.042778 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.042654    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b1410653-ac40-400a-82c8-c26c0ce1ea45-tls-cert\") pod \"authorino-68bd676465-gslhk\" (UID: \"b1410653-ac40-400a-82c8-c26c0ce1ea45\") " pod="kuadrant-system/authorino-68bd676465-gslhk"
Apr 28 19:27:27.143319 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.143292    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-svr5v\" (UniqueName: \"kubernetes.io/projected/b1410653-ac40-400a-82c8-c26c0ce1ea45-kube-api-access-svr5v\") pod \"authorino-68bd676465-gslhk\" (UID: \"b1410653-ac40-400a-82c8-c26c0ce1ea45\") " pod="kuadrant-system/authorino-68bd676465-gslhk"
Apr 28 19:27:27.143499 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.143362    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b1410653-ac40-400a-82c8-c26c0ce1ea45-tls-cert\") pod \"authorino-68bd676465-gslhk\" (UID: \"b1410653-ac40-400a-82c8-c26c0ce1ea45\") " pod="kuadrant-system/authorino-68bd676465-gslhk"
Apr 28 19:27:27.145887 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.145863    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b1410653-ac40-400a-82c8-c26c0ce1ea45-tls-cert\") pod \"authorino-68bd676465-gslhk\" (UID: \"b1410653-ac40-400a-82c8-c26c0ce1ea45\") " pod="kuadrant-system/authorino-68bd676465-gslhk"
Apr 28 19:27:27.151589 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.151570    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svr5v\" (UniqueName: \"kubernetes.io/projected/b1410653-ac40-400a-82c8-c26c0ce1ea45-kube-api-access-svr5v\") pod \"authorino-68bd676465-gslhk\" (UID: \"b1410653-ac40-400a-82c8-c26c0ce1ea45\") " pod="kuadrant-system/authorino-68bd676465-gslhk"
Apr 28 19:27:27.322114 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.322028    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68bd676465-gslhk"
Apr 28 19:27:27.446531 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:27.446508    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68bd676465-gslhk"]
Apr 28 19:27:27.447913 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:27:27.447886    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1410653_ac40_400a_82c8_c26c0ce1ea45.slice/crio-476f97061c7857d35896b3889193e9fb9c431d37fe6229228d4a936fd30c0c3d WatchSource:0}: Error finding container 476f97061c7857d35896b3889193e9fb9c431d37fe6229228d4a936fd30c0c3d: Status 404 returned error can't find the container with id 476f97061c7857d35896b3889193e9fb9c431d37fe6229228d4a936fd30c0c3d
Apr 28 19:27:28.175638 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:28.175542    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68bd676465-gslhk" event={"ID":"b1410653-ac40-400a-82c8-c26c0ce1ea45","Type":"ContainerStarted","Data":"0b00fd729f39d180a01fd7f2baab2e3518c68998f1c33347f9b75722772fd1fe"}
Apr 28 19:27:28.175638 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:28.175585    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68bd676465-gslhk" event={"ID":"b1410653-ac40-400a-82c8-c26c0ce1ea45","Type":"ContainerStarted","Data":"476f97061c7857d35896b3889193e9fb9c431d37fe6229228d4a936fd30c0c3d"}
Apr 28 19:27:28.192745 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:28.192698    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-68bd676465-gslhk" podStartSLOduration=1.850777672 podStartE2EDuration="2.192685098s" podCreationTimestamp="2026-04-28 19:27:26 +0000 UTC" firstStartedPulling="2026-04-28 19:27:27.449258778 +0000 UTC m=+634.273598864" lastFinishedPulling="2026-04-28 19:27:27.791166201 +0000 UTC m=+634.615506290" observedRunningTime="2026-04-28 19:27:28.190253824 +0000 UTC m=+635.014593931" watchObservedRunningTime="2026-04-28 19:27:28.192685098 +0000 UTC m=+635.017025205"
Apr 28 19:27:37.150667 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.150636    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"]
Apr 28 19:27:37.154445 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.154419    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.165503 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.165478    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"]
Apr 28 19:27:37.223194 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.223165    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5138a970-8b27-4382-a1e4-d2e459940e30-cacerts\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.223194 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.223197    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5138a970-8b27-4382-a1e4-d2e459940e30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.223406 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.223241    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92fzd\" (UniqueName: \"kubernetes.io/projected/5138a970-8b27-4382-a1e4-d2e459940e30-kube-api-access-92fzd\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.223406 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.223311    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5138a970-8b27-4382-a1e4-d2e459940e30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.223406 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.223379    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5138a970-8b27-4382-a1e4-d2e459940e30-local-certs\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.223563 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.223434    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5138a970-8b27-4382-a1e4-d2e459940e30-istio-kubeconfig\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.223563 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.223499    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5138a970-8b27-4382-a1e4-d2e459940e30-istio-token\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.324898 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.324868    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5138a970-8b27-4382-a1e4-d2e459940e30-cacerts\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.325085 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.324907    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5138a970-8b27-4382-a1e4-d2e459940e30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.325085 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.324974    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-92fzd\" (UniqueName: \"kubernetes.io/projected/5138a970-8b27-4382-a1e4-d2e459940e30-kube-api-access-92fzd\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.325085 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.325007    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5138a970-8b27-4382-a1e4-d2e459940e30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.325085 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.325053    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5138a970-8b27-4382-a1e4-d2e459940e30-local-certs\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.325085 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.325076    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5138a970-8b27-4382-a1e4-d2e459940e30-istio-kubeconfig\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.325360 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.325164    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5138a970-8b27-4382-a1e4-d2e459940e30-istio-token\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.325953 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.325923    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5138a970-8b27-4382-a1e4-d2e459940e30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.327518 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.327495    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5138a970-8b27-4382-a1e4-d2e459940e30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.327518 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.327502    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5138a970-8b27-4382-a1e4-d2e459940e30-local-certs\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.327704 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.327619    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5138a970-8b27-4382-a1e4-d2e459940e30-cacerts\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.327760 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.327699    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5138a970-8b27-4382-a1e4-d2e459940e30-istio-kubeconfig\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.345492 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.345463    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5138a970-8b27-4382-a1e4-d2e459940e30-istio-token\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.345637 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.345555    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92fzd\" (UniqueName: \"kubernetes.io/projected/5138a970-8b27-4382-a1e4-d2e459940e30-kube-api-access-92fzd\") pod \"istiod-openshift-gateway-55ff986f96-98lk2\" (UID: \"5138a970-8b27-4382-a1e4-d2e459940e30\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.465034 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.464999    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:37.610199 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.610175    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"]
Apr 28 19:27:37.612498 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:27:37.612462    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5138a970_8b27_4382_a1e4_d2e459940e30.slice/crio-7039ee2595f0bb969d29ad06d8c47a0ad8912c485f927ae0c17499f9bdddfc71 WatchSource:0}: Error finding container 7039ee2595f0bb969d29ad06d8c47a0ad8912c485f927ae0c17499f9bdddfc71: Status 404 returned error can't find the container with id 7039ee2595f0bb969d29ad06d8c47a0ad8912c485f927ae0c17499f9bdddfc71
Apr 28 19:27:37.614720 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.614690    2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Apr 28 19:27:37.614799 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:37.614761    2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Apr 28 19:27:38.217138 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:38.215972    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2" event={"ID":"5138a970-8b27-4382-a1e4-d2e459940e30","Type":"ContainerStarted","Data":"7746d2a773c951cd2e767128a77bae92d14ce00773614883658ff3a23284a6ca"}
Apr 28 19:27:38.217138 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:38.216018    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2" event={"ID":"5138a970-8b27-4382-a1e4-d2e459940e30","Type":"ContainerStarted","Data":"7039ee2595f0bb969d29ad06d8c47a0ad8912c485f927ae0c17499f9bdddfc71"}
Apr 28 19:27:38.217138 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:38.217025    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:38.219643 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:38.219606    2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-55ff986f96-98lk2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Apr 28 19:27:38.219810 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:38.219669    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2" podUID="5138a970-8b27-4382-a1e4-d2e459940e30" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Apr 28 19:27:38.244056 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:38.243993    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2" podStartSLOduration=1.243975469 podStartE2EDuration="1.243975469s" podCreationTimestamp="2026-04-28 19:27:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:27:38.241991797 +0000 UTC m=+645.066331906" watchObservedRunningTime="2026-04-28 19:27:38.243975469 +0000 UTC m=+645.068315579"
Apr 28 19:27:39.220938 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.220910    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-98lk2"
Apr 28 19:27:39.293298 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.293270    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"]
Apr 28 19:27:39.293511 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.293490    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl" podUID="1dd59c0b-7182-4864-af7f-a93352e9ac8b" containerName="discovery" containerID="cri-o://222e68f5ceefa0525b5eee7c31382c106bcbb531957d318445e2a36cc68317ce" gracePeriod=30
Apr 28 19:27:39.537286 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.537266    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:27:39.647716 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.647689    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-csr-dns-cert\") pod \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") "
Apr 28 19:27:39.647896 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.647729    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whl4b\" (UniqueName: \"kubernetes.io/projected/1dd59c0b-7182-4864-af7f-a93352e9ac8b-kube-api-access-whl4b\") pod \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") "
Apr 28 19:27:39.647896 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.647767    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dd59c0b-7182-4864-af7f-a93352e9ac8b-local-certs\") pod \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") "
Apr 28 19:27:39.647896 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.647792    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-cacerts\") pod \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") "
Apr 28 19:27:39.647896 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.647846    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-kubeconfig\") pod \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") "
Apr 28 19:27:39.647896 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.647881    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-csr-ca-configmap\") pod \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") "
Apr 28 19:27:39.648181 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.647904    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-token\") pod \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\" (UID: \"1dd59c0b-7182-4864-af7f-a93352e9ac8b\") "
Apr 28 19:27:39.648609 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.648579    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "1dd59c0b-7182-4864-af7f-a93352e9ac8b" (UID: "1dd59c0b-7182-4864-af7f-a93352e9ac8b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 28 19:27:39.650483 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.650346    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "1dd59c0b-7182-4864-af7f-a93352e9ac8b" (UID: "1dd59c0b-7182-4864-af7f-a93352e9ac8b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:27:39.650483 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.650394    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dd59c0b-7182-4864-af7f-a93352e9ac8b-local-certs" (OuterVolumeSpecName: "local-certs") pod "1dd59c0b-7182-4864-af7f-a93352e9ac8b" (UID: "1dd59c0b-7182-4864-af7f-a93352e9ac8b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:27:39.650483 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.650455    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "1dd59c0b-7182-4864-af7f-a93352e9ac8b" (UID: "1dd59c0b-7182-4864-af7f-a93352e9ac8b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:27:39.650730 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.650634    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-token" (OuterVolumeSpecName: "istio-token") pod "1dd59c0b-7182-4864-af7f-a93352e9ac8b" (UID: "1dd59c0b-7182-4864-af7f-a93352e9ac8b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:27:39.650799 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.650739    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dd59c0b-7182-4864-af7f-a93352e9ac8b-kube-api-access-whl4b" (OuterVolumeSpecName: "kube-api-access-whl4b") pod "1dd59c0b-7182-4864-af7f-a93352e9ac8b" (UID: "1dd59c0b-7182-4864-af7f-a93352e9ac8b"). InnerVolumeSpecName "kube-api-access-whl4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:27:39.650944 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.650926    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-cacerts" (OuterVolumeSpecName: "cacerts") pod "1dd59c0b-7182-4864-af7f-a93352e9ac8b" (UID: "1dd59c0b-7182-4864-af7f-a93352e9ac8b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:27:39.749522 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.749485    2569 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-kubeconfig\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:27:39.749522 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.749515    2569 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-csr-ca-configmap\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:27:39.749522 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.749526    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-token\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:27:39.749816 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.749535    2569 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-istio-csr-dns-cert\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:27:39.749816 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.749544    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whl4b\" (UniqueName: \"kubernetes.io/projected/1dd59c0b-7182-4864-af7f-a93352e9ac8b-kube-api-access-whl4b\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:27:39.749816 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.749553    2569 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dd59c0b-7182-4864-af7f-a93352e9ac8b-local-certs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:27:39.749816 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:39.749561    2569 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1dd59c0b-7182-4864-af7f-a93352e9ac8b-cacerts\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:27:40.224913 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:40.224835    2569 generic.go:358] "Generic (PLEG): container finished" podID="1dd59c0b-7182-4864-af7f-a93352e9ac8b" containerID="222e68f5ceefa0525b5eee7c31382c106bcbb531957d318445e2a36cc68317ce" exitCode=0
Apr 28 19:27:40.225369 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:40.224921    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl" event={"ID":"1dd59c0b-7182-4864-af7f-a93352e9ac8b","Type":"ContainerDied","Data":"222e68f5ceefa0525b5eee7c31382c106bcbb531957d318445e2a36cc68317ce"}
Apr 28 19:27:40.225369 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:40.224972    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl" event={"ID":"1dd59c0b-7182-4864-af7f-a93352e9ac8b","Type":"ContainerDied","Data":"e585505b425a01937533b93cf52ceb0f8a542030699bc4b788de51090773f832"}
Apr 28 19:27:40.225369 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:40.224994    2569 scope.go:117] "RemoveContainer" containerID="222e68f5ceefa0525b5eee7c31382c106bcbb531957d318445e2a36cc68317ce"
Apr 28 19:27:40.225369 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:40.224932    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"
Apr 28 19:27:40.234420 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:40.234384    2569 scope.go:117] "RemoveContainer" containerID="222e68f5ceefa0525b5eee7c31382c106bcbb531957d318445e2a36cc68317ce"
Apr 28 19:27:40.234761 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:27:40.234740    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"222e68f5ceefa0525b5eee7c31382c106bcbb531957d318445e2a36cc68317ce\": container with ID starting with 222e68f5ceefa0525b5eee7c31382c106bcbb531957d318445e2a36cc68317ce not found: ID does not exist" containerID="222e68f5ceefa0525b5eee7c31382c106bcbb531957d318445e2a36cc68317ce"
Apr 28 19:27:40.234840 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:40.234781    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"222e68f5ceefa0525b5eee7c31382c106bcbb531957d318445e2a36cc68317ce"} err="failed to get container status \"222e68f5ceefa0525b5eee7c31382c106bcbb531957d318445e2a36cc68317ce\": rpc error: code = NotFound desc = could not find container \"222e68f5ceefa0525b5eee7c31382c106bcbb531957d318445e2a36cc68317ce\": container with ID starting with 222e68f5ceefa0525b5eee7c31382c106bcbb531957d318445e2a36cc68317ce not found: ID does not exist"
Apr 28 19:27:40.246116 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:40.246090    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"]
Apr 28 19:27:40.250408 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:40.250375    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-2hngl"]
Apr 28 19:27:41.943357 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:41.943323    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dd59c0b-7182-4864-af7f-a93352e9ac8b" path="/var/lib/kubelet/pods/1dd59c0b-7182-4864-af7f-a93352e9ac8b/volumes"
Apr 28 19:27:46.352963 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.352932    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-86cc847c5c-8z66h"]
Apr 28 19:27:46.353509 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.353492    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dd59c0b-7182-4864-af7f-a93352e9ac8b" containerName="discovery"
Apr 28 19:27:46.353569 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.353511    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dd59c0b-7182-4864-af7f-a93352e9ac8b" containerName="discovery"
Apr 28 19:27:46.353628 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.353617    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="1dd59c0b-7182-4864-af7f-a93352e9ac8b" containerName="discovery"
Apr 28 19:27:46.356686 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.356666    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-86cc847c5c-8z66h"
Apr 28 19:27:46.360391 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.360361    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Apr 28 19:27:46.360642 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.360626    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Apr 28 19:27:46.361045 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.361028    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Apr 28 19:27:46.361370 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.361353    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-vmkw2\""
Apr 28 19:27:46.366700 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.366679    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-86cc847c5c-8z66h"]
Apr 28 19:27:46.412793 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.412758    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qpj6\" (UniqueName: \"kubernetes.io/projected/0991fd6c-085a-48ea-9214-9d446d11ab68-kube-api-access-2qpj6\") pod \"seaweedfs-86cc847c5c-8z66h\" (UID: \"0991fd6c-085a-48ea-9214-9d446d11ab68\") " pod="kserve/seaweedfs-86cc847c5c-8z66h"
Apr 28 19:27:46.412793 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.412797    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0991fd6c-085a-48ea-9214-9d446d11ab68-data\") pod \"seaweedfs-86cc847c5c-8z66h\" (UID: \"0991fd6c-085a-48ea-9214-9d446d11ab68\") " pod="kserve/seaweedfs-86cc847c5c-8z66h"
Apr 28 19:27:46.513992 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.513954    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2qpj6\" (UniqueName: \"kubernetes.io/projected/0991fd6c-085a-48ea-9214-9d446d11ab68-kube-api-access-2qpj6\") pod \"seaweedfs-86cc847c5c-8z66h\" (UID: \"0991fd6c-085a-48ea-9214-9d446d11ab68\") " pod="kserve/seaweedfs-86cc847c5c-8z66h"
Apr 28 19:27:46.513992 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.513994    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0991fd6c-085a-48ea-9214-9d446d11ab68-data\") pod \"seaweedfs-86cc847c5c-8z66h\" (UID: \"0991fd6c-085a-48ea-9214-9d446d11ab68\") " pod="kserve/seaweedfs-86cc847c5c-8z66h"
Apr 28 19:27:46.514428 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.514381    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0991fd6c-085a-48ea-9214-9d446d11ab68-data\") pod \"seaweedfs-86cc847c5c-8z66h\" (UID: \"0991fd6c-085a-48ea-9214-9d446d11ab68\") " pod="kserve/seaweedfs-86cc847c5c-8z66h"
Apr 28 19:27:46.522075 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.522056    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qpj6\" (UniqueName: \"kubernetes.io/projected/0991fd6c-085a-48ea-9214-9d446d11ab68-kube-api-access-2qpj6\") pod \"seaweedfs-86cc847c5c-8z66h\" (UID: \"0991fd6c-085a-48ea-9214-9d446d11ab68\") " pod="kserve/seaweedfs-86cc847c5c-8z66h"
Apr 28 19:27:46.675047 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.674983    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-86cc847c5c-8z66h"
Apr 28 19:27:46.798756 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:46.798732    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-86cc847c5c-8z66h"]
Apr 28 19:27:46.800269 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:27:46.800243    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0991fd6c_085a_48ea_9214_9d446d11ab68.slice/crio-0719a425d3a355cea8d87a86986a93e244733dd07f44d08f203a5c5472ffa6b8 WatchSource:0}: Error finding container 0719a425d3a355cea8d87a86986a93e244733dd07f44d08f203a5c5472ffa6b8: Status 404 returned error can't find the container with id 0719a425d3a355cea8d87a86986a93e244733dd07f44d08f203a5c5472ffa6b8
Apr 28 19:27:47.257702 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:47.257659    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-86cc847c5c-8z66h" event={"ID":"0991fd6c-085a-48ea-9214-9d446d11ab68","Type":"ContainerStarted","Data":"0719a425d3a355cea8d87a86986a93e244733dd07f44d08f203a5c5472ffa6b8"}
Apr 28 19:27:50.271466 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:50.271430    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-86cc847c5c-8z66h" event={"ID":"0991fd6c-085a-48ea-9214-9d446d11ab68","Type":"ContainerStarted","Data":"9acd86c2993d7449f928b3321a4ffc22235661b393bddb2828bd1d8ad20be493"}
Apr 28 19:27:50.271466 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:50.271472    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-86cc847c5c-8z66h"
Apr 28 19:27:50.289913 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:50.289862    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/seaweedfs-86cc847c5c-8z66h" podStartSLOduration=1.742225891 podStartE2EDuration="4.289846642s" podCreationTimestamp="2026-04-28 19:27:46 +0000 UTC" firstStartedPulling="2026-04-28 19:27:46.801566339 +0000 UTC m=+653.625906425" lastFinishedPulling="2026-04-28 19:27:49.349187087 +0000 UTC m=+656.173527176" observedRunningTime="2026-04-28 19:27:50.28713169 +0000 UTC m=+657.111471799" watchObservedRunningTime="2026-04-28 19:27:50.289846642 +0000 UTC m=+657.114186751"
Apr 28 19:27:56.277922 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:27:56.277889    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-86cc847c5c-8z66h"
Apr 28 19:28:59.203911 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.203831    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-696fc77849-5g6mv"]
Apr 28 19:28:59.207972 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.207952    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-5g6mv"
Apr 28 19:28:59.212440 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.212416    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Apr 28 19:28:59.212623 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.212601    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-gxrxd\""
Apr 28 19:28:59.220667 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.220644    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-5g6mv"]
Apr 28 19:28:59.352239 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.352206    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5222f0d4-ea9f-4485-8ad3-c95a80787aa1-cert\") pod \"odh-model-controller-696fc77849-5g6mv\" (UID: \"5222f0d4-ea9f-4485-8ad3-c95a80787aa1\") " pod="kserve/odh-model-controller-696fc77849-5g6mv"
Apr 28 19:28:59.352389 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.352258    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brtwx\" (UniqueName: \"kubernetes.io/projected/5222f0d4-ea9f-4485-8ad3-c95a80787aa1-kube-api-access-brtwx\") pod \"odh-model-controller-696fc77849-5g6mv\" (UID: \"5222f0d4-ea9f-4485-8ad3-c95a80787aa1\") " pod="kserve/odh-model-controller-696fc77849-5g6mv"
Apr 28 19:28:59.453393 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.453362    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5222f0d4-ea9f-4485-8ad3-c95a80787aa1-cert\") pod \"odh-model-controller-696fc77849-5g6mv\" (UID: \"5222f0d4-ea9f-4485-8ad3-c95a80787aa1\") " pod="kserve/odh-model-controller-696fc77849-5g6mv"
Apr 28 19:28:59.453589 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.453458    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-brtwx\" (UniqueName: \"kubernetes.io/projected/5222f0d4-ea9f-4485-8ad3-c95a80787aa1-kube-api-access-brtwx\") pod \"odh-model-controller-696fc77849-5g6mv\" (UID: \"5222f0d4-ea9f-4485-8ad3-c95a80787aa1\") " pod="kserve/odh-model-controller-696fc77849-5g6mv"
Apr 28 19:28:59.455986 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.455920    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5222f0d4-ea9f-4485-8ad3-c95a80787aa1-cert\") pod \"odh-model-controller-696fc77849-5g6mv\" (UID: \"5222f0d4-ea9f-4485-8ad3-c95a80787aa1\") " pod="kserve/odh-model-controller-696fc77849-5g6mv"
Apr 28 19:28:59.461708 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.461688    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brtwx\" (UniqueName: \"kubernetes.io/projected/5222f0d4-ea9f-4485-8ad3-c95a80787aa1-kube-api-access-brtwx\") pod \"odh-model-controller-696fc77849-5g6mv\" (UID: \"5222f0d4-ea9f-4485-8ad3-c95a80787aa1\") " pod="kserve/odh-model-controller-696fc77849-5g6mv"
Apr 28 19:28:59.521265 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.521240    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-5g6mv"
Apr 28 19:28:59.854520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.851284    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-5g6mv"]
Apr 28 19:28:59.855232 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:28:59.855201    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5222f0d4_ea9f_4485_8ad3_c95a80787aa1.slice/crio-fd325a2cc6c1cef4f859392a46fb472b5ed3e5fa50de1a145ddcc9e523a8eee0 WatchSource:0}: Error finding container fd325a2cc6c1cef4f859392a46fb472b5ed3e5fa50de1a145ddcc9e523a8eee0: Status 404 returned error can't find the container with id fd325a2cc6c1cef4f859392a46fb472b5ed3e5fa50de1a145ddcc9e523a8eee0
Apr 28 19:28:59.856702 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:28:59.856682    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 28 19:29:00.552668 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:29:00.552621    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-5g6mv" event={"ID":"5222f0d4-ea9f-4485-8ad3-c95a80787aa1","Type":"ContainerStarted","Data":"fd325a2cc6c1cef4f859392a46fb472b5ed3e5fa50de1a145ddcc9e523a8eee0"}
Apr 28 19:29:03.571777 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:29:03.571730    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-5g6mv" event={"ID":"5222f0d4-ea9f-4485-8ad3-c95a80787aa1","Type":"ContainerStarted","Data":"46ea30130ef2ffc4c1276d69f2a1bb68b141c04a8c00bde16e7eaf37e7279a13"}
Apr 28 19:29:03.572236 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:29:03.571861    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-696fc77849-5g6mv"
Apr 28 19:29:03.591162 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:29:03.591109    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/odh-model-controller-696fc77849-5g6mv" podStartSLOduration=1.6997611259999998 podStartE2EDuration="4.5910944s" podCreationTimestamp="2026-04-28 19:28:59 +0000 UTC" firstStartedPulling="2026-04-28 19:28:59.856803124 +0000 UTC m=+726.681143210" lastFinishedPulling="2026-04-28 19:29:02.748136393 +0000 UTC m=+729.572476484" observedRunningTime="2026-04-28 19:29:03.589049755 +0000 UTC m=+730.413389861" watchObservedRunningTime="2026-04-28 19:29:03.5910944 +0000 UTC m=+730.415434505"
Apr 28 19:29:14.580655 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:29:14.580624    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-696fc77849-5g6mv"
Apr 28 19:30:03.000938 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.000893    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"]
Apr 28 19:30:03.010647 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.010616    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.014895 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.014675    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Apr 28 19:30:03.014895 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.014738    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 28 19:30:03.015478 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.015458    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vz4n6\""
Apr 28 19:30:03.016058 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.016033    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-epp-sa-dockercfg-xkfpg\""
Apr 28 19:30:03.016300 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.016281    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 28 19:30:03.019170 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.019146    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"]
Apr 28 19:30:03.130319 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.130278    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-tmp\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.130520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.130340    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09eb97e2-0dab-43ca-be47-01aa97e0551c-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.130520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.130381    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mrp6\" (UniqueName: \"kubernetes.io/projected/09eb97e2-0dab-43ca-be47-01aa97e0551c-kube-api-access-5mrp6\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.130520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.130450    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-cache\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.130520 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.130515    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.130760 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.130599    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.231785 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.231746    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.231986 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.231800    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.231986 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.231882    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-tmp\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.231986 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.231931    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09eb97e2-0dab-43ca-be47-01aa97e0551c-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.232171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.231987    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5mrp6\" (UniqueName: \"kubernetes.io/projected/09eb97e2-0dab-43ca-be47-01aa97e0551c-kube-api-access-5mrp6\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.232171 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.232022    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-cache\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.232288 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.232196    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.232288 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.232258    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-tmp\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.232420 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.232340    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-cache\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.232485 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.232395    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.234986 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.234953    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09eb97e2-0dab-43ca-be47-01aa97e0551c-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.240370 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.240350    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mrp6\" (UniqueName: \"kubernetes.io/projected/09eb97e2-0dab-43ca-be47-01aa97e0551c-kube-api-access-5mrp6\") pod \"scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.322664 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.322581    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:03.453787 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.453758    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"]
Apr 28 19:30:03.454815 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:30:03.454788    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09eb97e2_0dab_43ca_be47_01aa97e0551c.slice/crio-9c7a409229a343d89d660b9a48d84151222353f5e55fc6c194d1e53f7ca1136b WatchSource:0}: Error finding container 9c7a409229a343d89d660b9a48d84151222353f5e55fc6c194d1e53f7ca1136b: Status 404 returned error can't find the container with id 9c7a409229a343d89d660b9a48d84151222353f5e55fc6c194d1e53f7ca1136b
Apr 28 19:30:03.814852 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:03.814815    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h" event={"ID":"09eb97e2-0dab-43ca-be47-01aa97e0551c","Type":"ContainerStarted","Data":"9c7a409229a343d89d660b9a48d84151222353f5e55fc6c194d1e53f7ca1136b"}
Apr 28 19:30:07.833167 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:07.833130    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h" event={"ID":"09eb97e2-0dab-43ca-be47-01aa97e0551c","Type":"ContainerStarted","Data":"4b71d763c7c84ae95a7aa4a6fa373d45c420c9f17c4f05f8bc3fa6d50ab8b31a"}
Apr 28 19:30:08.838124 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:08.838088    2569 generic.go:358] "Generic (PLEG): container finished" podID="09eb97e2-0dab-43ca-be47-01aa97e0551c" containerID="4b71d763c7c84ae95a7aa4a6fa373d45c420c9f17c4f05f8bc3fa6d50ab8b31a" exitCode=0
Apr 28 19:30:08.838553 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:08.838176    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h" event={"ID":"09eb97e2-0dab-43ca-be47-01aa97e0551c","Type":"ContainerDied","Data":"4b71d763c7c84ae95a7aa4a6fa373d45c420c9f17c4f05f8bc3fa6d50ab8b31a"}
Apr 28 19:30:10.852325 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:10.852290    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h" event={"ID":"09eb97e2-0dab-43ca-be47-01aa97e0551c","Type":"ContainerStarted","Data":"88fe19212540a304ef009673d5bf8856267727b355cb825463f25197efc2aae7"}
Apr 28 19:30:39.987494 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:39.987454    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h" event={"ID":"09eb97e2-0dab-43ca-be47-01aa97e0551c","Type":"ContainerStarted","Data":"5dd32efff4c5cd933dc40c932d4192e49dc55471027b9d28d8a7ab7f7ecdfaf6"}
Apr 28 19:30:39.987908 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:39.987629    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:39.990365 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:39.990343    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:40.011071 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:40.011025    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h" podStartSLOduration=2.373347208 podStartE2EDuration="38.011011616s" podCreationTimestamp="2026-04-28 19:30:02 +0000 UTC" firstStartedPulling="2026-04-28 19:30:03.456620709 +0000 UTC m=+790.280960796" lastFinishedPulling="2026-04-28 19:30:39.094285104 +0000 UTC m=+825.918625204" observedRunningTime="2026-04-28 19:30:40.008581582 +0000 UTC m=+826.832921713" watchObservedRunningTime="2026-04-28 19:30:40.011011616 +0000 UTC m=+826.835351760"
Apr 28 19:30:43.323263 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:43.323226    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:43.323263 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:43.323270    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:53.324883 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:53.324848    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:30:53.325924 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:30:53.325901    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:31:53.882464 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:31:53.882369    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:31:53.885350 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:31:53.885322    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:31:53.885826 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:31:53.885805    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:31:53.888302 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:31:53.888283    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:36:34.397061 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.397032    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"]
Apr 28 19:36:34.400900 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.400884    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.404010 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.403987    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-sa-dockercfg-ghb55\""
Apr 28 19:36:34.404118 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.404099    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvdde380eaa9fe1facad32d45131f9e34d-kserve-self-signed-certs\""
Apr 28 19:36:34.414243 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.414223    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"]
Apr 28 19:36:34.443951 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.443908    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-kserve-provision-location\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.444108 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.443970    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-cache\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.444108 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.444040    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da1e3b37-3354-4e32-8054-617c5d07b700-tls-certs\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.444108 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.444101    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-uds\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.444215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.444152    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-tmp\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.444215 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.444204    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnhh4\" (UniqueName: \"kubernetes.io/projected/da1e3b37-3354-4e32-8054-617c5d07b700-kube-api-access-cnhh4\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.545234 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.545205    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cnhh4\" (UniqueName: \"kubernetes.io/projected/da1e3b37-3354-4e32-8054-617c5d07b700-kube-api-access-cnhh4\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.545370 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.545258    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-kserve-provision-location\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.545370 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.545301    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-cache\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.545370 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.545332    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da1e3b37-3354-4e32-8054-617c5d07b700-tls-certs\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.545370 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.545366    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-uds\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.545610 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.545433    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-tmp\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.545800 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.545776    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-cache\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.545800 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.545790    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-uds\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.545919 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.545836    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-kserve-provision-location\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.545919 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.545906    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-tmp\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.547853 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.547834    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da1e3b37-3354-4e32-8054-617c5d07b700-tls-certs\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.553193 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.553168    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnhh4\" (UniqueName: \"kubernetes.io/projected/da1e3b37-3354-4e32-8054-617c5d07b700-kube-api-access-cnhh4\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:34.710725 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:34.710690    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:35.056768 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:35.056738    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"]
Apr 28 19:36:35.058756 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:36:35.058727    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1e3b37_3354_4e32_8054_617c5d07b700.slice/crio-25059b85c42f5790b3d0e4dc1d6e75b3e58511aedffbbfe944412ef1f491a195 WatchSource:0}: Error finding container 25059b85c42f5790b3d0e4dc1d6e75b3e58511aedffbbfe944412ef1f491a195: Status 404 returned error can't find the container with id 25059b85c42f5790b3d0e4dc1d6e75b3e58511aedffbbfe944412ef1f491a195
Apr 28 19:36:35.060727 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:35.060706    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 28 19:36:35.396722 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:35.396630    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh" event={"ID":"da1e3b37-3354-4e32-8054-617c5d07b700","Type":"ContainerStarted","Data":"eaeec870df3429daf6eaf576c6f9d0c892e6b1b85f8512abe801420b42d56913"}
Apr 28 19:36:35.396722 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:35.396667    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh" event={"ID":"da1e3b37-3354-4e32-8054-617c5d07b700","Type":"ContainerStarted","Data":"25059b85c42f5790b3d0e4dc1d6e75b3e58511aedffbbfe944412ef1f491a195"}
Apr 28 19:36:36.402513 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:36.402475    2569 generic.go:358] "Generic (PLEG): container finished" podID="da1e3b37-3354-4e32-8054-617c5d07b700" containerID="eaeec870df3429daf6eaf576c6f9d0c892e6b1b85f8512abe801420b42d56913" exitCode=0
Apr 28 19:36:36.402919 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:36.402564    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh" event={"ID":"da1e3b37-3354-4e32-8054-617c5d07b700","Type":"ContainerDied","Data":"eaeec870df3429daf6eaf576c6f9d0c892e6b1b85f8512abe801420b42d56913"}
Apr 28 19:36:37.410484 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:37.410440    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh" event={"ID":"da1e3b37-3354-4e32-8054-617c5d07b700","Type":"ContainerStarted","Data":"043bb74d666bb962286ea26f396cac19a5be3110479f1d5efef227716d1f9310"}
Apr 28 19:36:37.410484 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:37.410488    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh" event={"ID":"da1e3b37-3354-4e32-8054-617c5d07b700","Type":"ContainerStarted","Data":"2f1dff15b313971bfd6a92b36b1a459d0cd18b6cf3e9fb262e2ed51989918f4d"}
Apr 28 19:36:37.411025 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:37.410584    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:37.437131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:37.437076    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh" podStartSLOduration=3.43706292 podStartE2EDuration="3.43706292s" podCreationTimestamp="2026-04-28 19:36:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:36:37.43480696 +0000 UTC m=+1184.259147069" watchObservedRunningTime="2026-04-28 19:36:37.43706292 +0000 UTC m=+1184.261403027"
Apr 28 19:36:44.710928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:44.710885    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:44.711328 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:44.710938    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:44.713689 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:44.713664    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:45.449362 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:45.449327    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:36:53.924224 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:53.924194    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:36:53.925659 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:53.925637    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:36:53.927206 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:53.927172    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:36:53.928586 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:36:53.928566    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:37:06.452925 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:37:06.452898    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:41:53.963026 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:41:53.962999    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:41:53.964719 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:41:53.964698    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:41:53.966091 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:41:53.966071    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:41:53.968008 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:41:53.967986    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:43:40.002266 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:40.002234    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"]
Apr 28 19:43:40.002885 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:40.002680    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh" podUID="da1e3b37-3354-4e32-8054-617c5d07b700" containerName="main" containerID="cri-o://2f1dff15b313971bfd6a92b36b1a459d0cd18b6cf3e9fb262e2ed51989918f4d" gracePeriod=30
Apr 28 19:43:40.002885 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:40.002689    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh" podUID="da1e3b37-3354-4e32-8054-617c5d07b700" containerName="tokenizer" containerID="cri-o://043bb74d666bb962286ea26f396cac19a5be3110479f1d5efef227716d1f9310" gracePeriod=30
Apr 28 19:43:40.122324 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:40.122260    2569 generic.go:358] "Generic (PLEG): container finished" podID="da1e3b37-3354-4e32-8054-617c5d07b700" containerID="2f1dff15b313971bfd6a92b36b1a459d0cd18b6cf3e9fb262e2ed51989918f4d" exitCode=0
Apr 28 19:43:40.122324 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:40.122298    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh" event={"ID":"da1e3b37-3354-4e32-8054-617c5d07b700","Type":"ContainerDied","Data":"2f1dff15b313971bfd6a92b36b1a459d0cd18b6cf3e9fb262e2ed51989918f4d"}
Apr 28 19:43:41.128749 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.128716    2569 generic.go:358] "Generic (PLEG): container finished" podID="da1e3b37-3354-4e32-8054-617c5d07b700" containerID="043bb74d666bb962286ea26f396cac19a5be3110479f1d5efef227716d1f9310" exitCode=0
Apr 28 19:43:41.129082 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.128777    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh" event={"ID":"da1e3b37-3354-4e32-8054-617c5d07b700","Type":"ContainerDied","Data":"043bb74d666bb962286ea26f396cac19a5be3110479f1d5efef227716d1f9310"}
Apr 28 19:43:41.157931 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.157912    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:43:41.250289 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.250202    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-tmp\") pod \"da1e3b37-3354-4e32-8054-617c5d07b700\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") "
Apr 28 19:43:41.250289 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.250275    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-kserve-provision-location\") pod \"da1e3b37-3354-4e32-8054-617c5d07b700\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") "
Apr 28 19:43:41.250516 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.250322    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnhh4\" (UniqueName: \"kubernetes.io/projected/da1e3b37-3354-4e32-8054-617c5d07b700-kube-api-access-cnhh4\") pod \"da1e3b37-3354-4e32-8054-617c5d07b700\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") "
Apr 28 19:43:41.250516 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.250355    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-uds\") pod \"da1e3b37-3354-4e32-8054-617c5d07b700\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") "
Apr 28 19:43:41.250516 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.250380    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-cache\") pod \"da1e3b37-3354-4e32-8054-617c5d07b700\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") "
Apr 28 19:43:41.250516 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.250421    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da1e3b37-3354-4e32-8054-617c5d07b700-tls-certs\") pod \"da1e3b37-3354-4e32-8054-617c5d07b700\" (UID: \"da1e3b37-3354-4e32-8054-617c5d07b700\") "
Apr 28 19:43:41.250726 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.250635    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "da1e3b37-3354-4e32-8054-617c5d07b700" (UID: "da1e3b37-3354-4e32-8054-617c5d07b700"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:43:41.250726 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.250664    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "da1e3b37-3354-4e32-8054-617c5d07b700" (UID: "da1e3b37-3354-4e32-8054-617c5d07b700"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:43:41.250726 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.250713    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "da1e3b37-3354-4e32-8054-617c5d07b700" (UID: "da1e3b37-3354-4e32-8054-617c5d07b700"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:43:41.250883 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.250781    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-cache\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:43:41.250883 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.250796    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-tmp\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:43:41.251028 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.251006    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "da1e3b37-3354-4e32-8054-617c5d07b700" (UID: "da1e3b37-3354-4e32-8054-617c5d07b700"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:43:41.252696 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.252675    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da1e3b37-3354-4e32-8054-617c5d07b700-kube-api-access-cnhh4" (OuterVolumeSpecName: "kube-api-access-cnhh4") pod "da1e3b37-3354-4e32-8054-617c5d07b700" (UID: "da1e3b37-3354-4e32-8054-617c5d07b700"). InnerVolumeSpecName "kube-api-access-cnhh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:43:41.252766 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.252713    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da1e3b37-3354-4e32-8054-617c5d07b700-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "da1e3b37-3354-4e32-8054-617c5d07b700" (UID: "da1e3b37-3354-4e32-8054-617c5d07b700"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:43:41.351504 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.351481    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da1e3b37-3354-4e32-8054-617c5d07b700-tls-certs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:43:41.351504 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.351502    2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-kserve-provision-location\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:43:41.351636 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.351513    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnhh4\" (UniqueName: \"kubernetes.io/projected/da1e3b37-3354-4e32-8054-617c5d07b700-kube-api-access-cnhh4\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:43:41.351636 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:41.351522    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/da1e3b37-3354-4e32-8054-617c5d07b700-tokenizer-uds\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:43:42.134980 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:42.134944    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh" event={"ID":"da1e3b37-3354-4e32-8054-617c5d07b700","Type":"ContainerDied","Data":"25059b85c42f5790b3d0e4dc1d6e75b3e58511aedffbbfe944412ef1f491a195"}
Apr 28 19:43:42.135372 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:42.134997    2569 scope.go:117] "RemoveContainer" containerID="043bb74d666bb962286ea26f396cac19a5be3110479f1d5efef227716d1f9310"
Apr 28 19:43:42.135372 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:42.134964    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"
Apr 28 19:43:42.146659 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:42.146636    2569 scope.go:117] "RemoveContainer" containerID="2f1dff15b313971bfd6a92b36b1a459d0cd18b6cf3e9fb262e2ed51989918f4d"
Apr 28 19:43:42.154952 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:42.154926    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"]
Apr 28 19:43:42.155573 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:42.155557    2569 scope.go:117] "RemoveContainer" containerID="eaeec870df3429daf6eaf576c6f9d0c892e6b1b85f8512abe801420b42d56913"
Apr 28 19:43:42.160306 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:42.160280    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schemnhxh"]
Apr 28 19:43:43.943515 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:43.943483    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da1e3b37-3354-4e32-8054-617c5d07b700" path="/var/lib/kubelet/pods/da1e3b37-3354-4e32-8054-617c5d07b700/volumes"
Apr 28 19:43:49.984312 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:49.984277    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"]
Apr 28 19:43:49.984703 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:49.984686    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da1e3b37-3354-4e32-8054-617c5d07b700" containerName="storage-initializer"
Apr 28 19:43:49.984747 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:49.984705    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="da1e3b37-3354-4e32-8054-617c5d07b700" containerName="storage-initializer"
Apr 28 19:43:49.984747 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:49.984714    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da1e3b37-3354-4e32-8054-617c5d07b700" containerName="main"
Apr 28 19:43:49.984747 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:49.984720    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="da1e3b37-3354-4e32-8054-617c5d07b700" containerName="main"
Apr 28 19:43:49.984747 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:49.984733    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da1e3b37-3354-4e32-8054-617c5d07b700" containerName="tokenizer"
Apr 28 19:43:49.984747 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:49.984738    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="da1e3b37-3354-4e32-8054-617c5d07b700" containerName="tokenizer"
Apr 28 19:43:49.984902 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:49.984808    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="da1e3b37-3354-4e32-8054-617c5d07b700" containerName="main"
Apr 28 19:43:49.984902 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:49.984818    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="da1e3b37-3354-4e32-8054-617c5d07b700" containerName="tokenizer"
Apr 28 19:43:49.990124 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:49.990105    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:49.992841 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:49.992823    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Apr 28 19:43:49.992917 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:49.992829    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-epp-sa-dockercfg-cmrnz\""
Apr 28 19:43:49.999250 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:49.999226    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"]
Apr 28 19:43:50.130517 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.130485    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69976efd-d254-4490-8742-c40b69b6d7f5-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.130674 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.130543    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-tmp\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.130674 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.130575    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.130674 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.130618    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmg97\" (UniqueName: \"kubernetes.io/projected/69976efd-d254-4490-8742-c40b69b6d7f5-kube-api-access-dmg97\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.130674 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.130646    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-cache\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.130674 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.130668    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.231823 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.231786    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-cache\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.231823 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.231822    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.232061 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.231896    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69976efd-d254-4490-8742-c40b69b6d7f5-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.232061 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.231955    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-tmp\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.232061 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.231991    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.232061 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.232045    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dmg97\" (UniqueName: \"kubernetes.io/projected/69976efd-d254-4490-8742-c40b69b6d7f5-kube-api-access-dmg97\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.232269 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.232243    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-cache\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.232335 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.232250    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.232335 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.232322    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-tmp\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.232481 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.232460    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.234391 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.234334    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69976efd-d254-4490-8742-c40b69b6d7f5-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.240209 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.240186    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmg97\" (UniqueName: \"kubernetes.io/projected/69976efd-d254-4490-8742-c40b69b6d7f5-kube-api-access-dmg97\") pod \"custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.301010 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.300982    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:50.441267 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.441233    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"]
Apr 28 19:43:50.442659 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:43:50.442634    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69976efd_d254_4490_8742_c40b69b6d7f5.slice/crio-d5582f985fc26a03ed328044f181514adb1679886b255ff0decb7b37b33470f0 WatchSource:0}: Error finding container d5582f985fc26a03ed328044f181514adb1679886b255ff0decb7b37b33470f0: Status 404 returned error can't find the container with id d5582f985fc26a03ed328044f181514adb1679886b255ff0decb7b37b33470f0
Apr 28 19:43:50.444504 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:50.444488    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 28 19:43:51.171748 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:51.171719    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb" event={"ID":"69976efd-d254-4490-8742-c40b69b6d7f5","Type":"ContainerStarted","Data":"86c19bab6f8e6ed6e996de77cd16261114ee4466ef7816c9d5d183c006072467"}
Apr 28 19:43:51.172057 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:51.171754    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb" event={"ID":"69976efd-d254-4490-8742-c40b69b6d7f5","Type":"ContainerStarted","Data":"d5582f985fc26a03ed328044f181514adb1679886b255ff0decb7b37b33470f0"}
Apr 28 19:43:52.177635 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:52.177597    2569 generic.go:358] "Generic (PLEG): container finished" podID="69976efd-d254-4490-8742-c40b69b6d7f5" containerID="86c19bab6f8e6ed6e996de77cd16261114ee4466ef7816c9d5d183c006072467" exitCode=0
Apr 28 19:43:52.178010 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:52.177684    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb" event={"ID":"69976efd-d254-4490-8742-c40b69b6d7f5","Type":"ContainerDied","Data":"86c19bab6f8e6ed6e996de77cd16261114ee4466ef7816c9d5d183c006072467"}
Apr 28 19:43:53.183516 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:53.183475    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb" event={"ID":"69976efd-d254-4490-8742-c40b69b6d7f5","Type":"ContainerStarted","Data":"3457083dcbf9b76c07cd2d8458570d58ab11a19fb08a9e6412e56e7463354fde"}
Apr 28 19:43:53.183516 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:53.183519    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb" event={"ID":"69976efd-d254-4490-8742-c40b69b6d7f5","Type":"ContainerStarted","Data":"3b3eb6f971b85b0914da32bdf690065c8b0c4625e0e9b98d4310a6ac063e28fb"}
Apr 28 19:43:53.183943 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:53.183621    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:43:53.211333 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:43:53.211281    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb" podStartSLOduration=4.211266216 podStartE2EDuration="4.211266216s" podCreationTimestamp="2026-04-28 19:43:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:43:53.207667469 +0000 UTC m=+1620.032007587" watchObservedRunningTime="2026-04-28 19:43:53.211266216 +0000 UTC m=+1620.035606321"
Apr 28 19:44:00.301768 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:00.301722    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:44:00.301768 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:00.301778    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:44:00.304692 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:00.304669    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:44:01.218961 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:01.218935    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:44:22.223961 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:22.223924    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:44:38.279598 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:38.279560    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"]
Apr 28 19:44:38.280091 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:38.279998    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h" podUID="09eb97e2-0dab-43ca-be47-01aa97e0551c" containerName="main" containerID="cri-o://88fe19212540a304ef009673d5bf8856267727b355cb825463f25197efc2aae7" gracePeriod=30
Apr 28 19:44:38.280225 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:38.280096    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h" podUID="09eb97e2-0dab-43ca-be47-01aa97e0551c" containerName="tokenizer" containerID="cri-o://5dd32efff4c5cd933dc40c932d4192e49dc55471027b9d28d8a7ab7f7ecdfaf6" gracePeriod=30
Apr 28 19:44:39.375754 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.375718    2569 generic.go:358] "Generic (PLEG): container finished" podID="09eb97e2-0dab-43ca-be47-01aa97e0551c" containerID="88fe19212540a304ef009673d5bf8856267727b355cb825463f25197efc2aae7" exitCode=0
Apr 28 19:44:39.376110 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.375798    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h" event={"ID":"09eb97e2-0dab-43ca-be47-01aa97e0551c","Type":"ContainerDied","Data":"88fe19212540a304ef009673d5bf8856267727b355cb825463f25197efc2aae7"}
Apr 28 19:44:39.545063 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.545041    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:44:39.587317 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.587231    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-tmp\") pod \"09eb97e2-0dab-43ca-be47-01aa97e0551c\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") "
Apr 28 19:44:39.587317 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.587287    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09eb97e2-0dab-43ca-be47-01aa97e0551c-tls-certs\") pod \"09eb97e2-0dab-43ca-be47-01aa97e0551c\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") "
Apr 28 19:44:39.587317 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.587308    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-uds\") pod \"09eb97e2-0dab-43ca-be47-01aa97e0551c\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") "
Apr 28 19:44:39.587626 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.587336    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-kserve-provision-location\") pod \"09eb97e2-0dab-43ca-be47-01aa97e0551c\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") "
Apr 28 19:44:39.587626 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.587383    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-cache\") pod \"09eb97e2-0dab-43ca-be47-01aa97e0551c\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") "
Apr 28 19:44:39.587626 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.587449    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mrp6\" (UniqueName: \"kubernetes.io/projected/09eb97e2-0dab-43ca-be47-01aa97e0551c-kube-api-access-5mrp6\") pod \"09eb97e2-0dab-43ca-be47-01aa97e0551c\" (UID: \"09eb97e2-0dab-43ca-be47-01aa97e0551c\") "
Apr 28 19:44:39.587626 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.587610    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "09eb97e2-0dab-43ca-be47-01aa97e0551c" (UID: "09eb97e2-0dab-43ca-be47-01aa97e0551c"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:44:39.587849 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.587646    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "09eb97e2-0dab-43ca-be47-01aa97e0551c" (UID: "09eb97e2-0dab-43ca-be47-01aa97e0551c"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:44:39.587849 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.587685    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "09eb97e2-0dab-43ca-be47-01aa97e0551c" (UID: "09eb97e2-0dab-43ca-be47-01aa97e0551c"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:44:39.587849 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.587800    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-cache\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:44:39.587849 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.587822    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-tmp\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:44:39.587849 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.587836    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-tokenizer-uds\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:44:39.588159 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.588132    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "09eb97e2-0dab-43ca-be47-01aa97e0551c" (UID: "09eb97e2-0dab-43ca-be47-01aa97e0551c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:44:39.589645 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.589612    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09eb97e2-0dab-43ca-be47-01aa97e0551c-kube-api-access-5mrp6" (OuterVolumeSpecName: "kube-api-access-5mrp6") pod "09eb97e2-0dab-43ca-be47-01aa97e0551c" (UID: "09eb97e2-0dab-43ca-be47-01aa97e0551c"). InnerVolumeSpecName "kube-api-access-5mrp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:44:39.589645 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.589628    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09eb97e2-0dab-43ca-be47-01aa97e0551c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09eb97e2-0dab-43ca-be47-01aa97e0551c" (UID: "09eb97e2-0dab-43ca-be47-01aa97e0551c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:44:39.689033 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.689001    2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09eb97e2-0dab-43ca-be47-01aa97e0551c-kserve-provision-location\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:44:39.689033 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.689026    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mrp6\" (UniqueName: \"kubernetes.io/projected/09eb97e2-0dab-43ca-be47-01aa97e0551c-kube-api-access-5mrp6\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:44:39.689033 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:39.689037    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09eb97e2-0dab-43ca-be47-01aa97e0551c-tls-certs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:44:40.381929 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.381851    2569 generic.go:358] "Generic (PLEG): container finished" podID="09eb97e2-0dab-43ca-be47-01aa97e0551c" containerID="5dd32efff4c5cd933dc40c932d4192e49dc55471027b9d28d8a7ab7f7ecdfaf6" exitCode=0
Apr 28 19:44:40.382346 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.381926    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"
Apr 28 19:44:40.382346 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.381931    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h" event={"ID":"09eb97e2-0dab-43ca-be47-01aa97e0551c","Type":"ContainerDied","Data":"5dd32efff4c5cd933dc40c932d4192e49dc55471027b9d28d8a7ab7f7ecdfaf6"}
Apr 28 19:44:40.382346 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.381966    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h" event={"ID":"09eb97e2-0dab-43ca-be47-01aa97e0551c","Type":"ContainerDied","Data":"9c7a409229a343d89d660b9a48d84151222353f5e55fc6c194d1e53f7ca1136b"}
Apr 28 19:44:40.382346 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.381985    2569 scope.go:117] "RemoveContainer" containerID="5dd32efff4c5cd933dc40c932d4192e49dc55471027b9d28d8a7ab7f7ecdfaf6"
Apr 28 19:44:40.391240 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.391218    2569 scope.go:117] "RemoveContainer" containerID="88fe19212540a304ef009673d5bf8856267727b355cb825463f25197efc2aae7"
Apr 28 19:44:40.400374 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.400180    2569 scope.go:117] "RemoveContainer" containerID="4b71d763c7c84ae95a7aa4a6fa373d45c420c9f17c4f05f8bc3fa6d50ab8b31a"
Apr 28 19:44:40.408599 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.408574    2569 scope.go:117] "RemoveContainer" containerID="5dd32efff4c5cd933dc40c932d4192e49dc55471027b9d28d8a7ab7f7ecdfaf6"
Apr 28 19:44:40.408869 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:44:40.408844    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dd32efff4c5cd933dc40c932d4192e49dc55471027b9d28d8a7ab7f7ecdfaf6\": container with ID starting with 5dd32efff4c5cd933dc40c932d4192e49dc55471027b9d28d8a7ab7f7ecdfaf6 not found: ID does not exist" containerID="5dd32efff4c5cd933dc40c932d4192e49dc55471027b9d28d8a7ab7f7ecdfaf6"
Apr 28 19:44:40.408940 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.408883    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dd32efff4c5cd933dc40c932d4192e49dc55471027b9d28d8a7ab7f7ecdfaf6"} err="failed to get container status \"5dd32efff4c5cd933dc40c932d4192e49dc55471027b9d28d8a7ab7f7ecdfaf6\": rpc error: code = NotFound desc = could not find container \"5dd32efff4c5cd933dc40c932d4192e49dc55471027b9d28d8a7ab7f7ecdfaf6\": container with ID starting with 5dd32efff4c5cd933dc40c932d4192e49dc55471027b9d28d8a7ab7f7ecdfaf6 not found: ID does not exist"
Apr 28 19:44:40.408940 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.408907    2569 scope.go:117] "RemoveContainer" containerID="88fe19212540a304ef009673d5bf8856267727b355cb825463f25197efc2aae7"
Apr 28 19:44:40.409636 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:44:40.409177    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88fe19212540a304ef009673d5bf8856267727b355cb825463f25197efc2aae7\": container with ID starting with 88fe19212540a304ef009673d5bf8856267727b355cb825463f25197efc2aae7 not found: ID does not exist" containerID="88fe19212540a304ef009673d5bf8856267727b355cb825463f25197efc2aae7"
Apr 28 19:44:40.409636 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.409230    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88fe19212540a304ef009673d5bf8856267727b355cb825463f25197efc2aae7"} err="failed to get container status \"88fe19212540a304ef009673d5bf8856267727b355cb825463f25197efc2aae7\": rpc error: code = NotFound desc = could not find container \"88fe19212540a304ef009673d5bf8856267727b355cb825463f25197efc2aae7\": container with ID starting with 88fe19212540a304ef009673d5bf8856267727b355cb825463f25197efc2aae7 not found: ID does not exist"
Apr 28 19:44:40.409636 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.409254    2569 scope.go:117] "RemoveContainer" containerID="4b71d763c7c84ae95a7aa4a6fa373d45c420c9f17c4f05f8bc3fa6d50ab8b31a"
Apr 28 19:44:40.409636 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.409515    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"]
Apr 28 19:44:40.409861 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:44:40.409642    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b71d763c7c84ae95a7aa4a6fa373d45c420c9f17c4f05f8bc3fa6d50ab8b31a\": container with ID starting with 4b71d763c7c84ae95a7aa4a6fa373d45c420c9f17c4f05f8bc3fa6d50ab8b31a not found: ID does not exist" containerID="4b71d763c7c84ae95a7aa4a6fa373d45c420c9f17c4f05f8bc3fa6d50ab8b31a"
Apr 28 19:44:40.409861 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.409668    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b71d763c7c84ae95a7aa4a6fa373d45c420c9f17c4f05f8bc3fa6d50ab8b31a"} err="failed to get container status \"4b71d763c7c84ae95a7aa4a6fa373d45c420c9f17c4f05f8bc3fa6d50ab8b31a\": rpc error: code = NotFound desc = could not find container \"4b71d763c7c84ae95a7aa4a6fa373d45c420c9f17c4f05f8bc3fa6d50ab8b31a\": container with ID starting with 4b71d763c7c84ae95a7aa4a6fa373d45c420c9f17c4f05f8bc3fa6d50ab8b31a not found: ID does not exist"
Apr 28 19:44:40.412856 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:40.412837    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-79f75msn4h"]
Apr 28 19:44:41.943087 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:41.943056    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09eb97e2-0dab-43ca-be47-01aa97e0551c" path="/var/lib/kubelet/pods/09eb97e2-0dab-43ca-be47-01aa97e0551c/volumes"
Apr 28 19:44:55.577234 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.577205    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"]
Apr 28 19:44:55.577692 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.577678    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09eb97e2-0dab-43ca-be47-01aa97e0551c" containerName="storage-initializer"
Apr 28 19:44:55.577692 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.577693    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="09eb97e2-0dab-43ca-be47-01aa97e0551c" containerName="storage-initializer"
Apr 28 19:44:55.577830 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.577705    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09eb97e2-0dab-43ca-be47-01aa97e0551c" containerName="tokenizer"
Apr 28 19:44:55.577830 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.577711    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="09eb97e2-0dab-43ca-be47-01aa97e0551c" containerName="tokenizer"
Apr 28 19:44:55.577830 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.577735    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09eb97e2-0dab-43ca-be47-01aa97e0551c" containerName="main"
Apr 28 19:44:55.577830 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.577741    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="09eb97e2-0dab-43ca-be47-01aa97e0551c" containerName="main"
Apr 28 19:44:55.577830 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.577812    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="09eb97e2-0dab-43ca-be47-01aa97e0551c" containerName="main"
Apr 28 19:44:55.577830 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.577824    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="09eb97e2-0dab-43ca-be47-01aa97e0551c" containerName="tokenizer"
Apr 28 19:44:55.582542 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.582522    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.585069 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.585042    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Apr 28 19:44:55.585184 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.585050    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-epp-sa-dockercfg-lcmqz\""
Apr 28 19:44:55.590937 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.590912    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"]
Apr 28 19:44:55.636128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.636093    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-tmp\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.636283 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.636180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-uds\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.636283 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.636234    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx6gf\" (UniqueName: \"kubernetes.io/projected/aa5b9038-d7ed-41a7-a041-b27f649c9420-kube-api-access-bx6gf\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.636443 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.636290    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.636443 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.636358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-cache\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.636443 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.636412    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa5b9038-d7ed-41a7-a041-b27f649c9420-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.737582 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.737543    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-uds\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.737786 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.737599    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bx6gf\" (UniqueName: \"kubernetes.io/projected/aa5b9038-d7ed-41a7-a041-b27f649c9420-kube-api-access-bx6gf\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.737786 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.737640    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.737786 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.737676    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-cache\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.737966 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.737792    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa5b9038-d7ed-41a7-a041-b27f649c9420-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.737966 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.737842    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-tmp\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.738296 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.738005    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-uds\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.738296 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.738015    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-cache\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.738296 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.738096    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.738296 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.738151    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-tmp\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.740308 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.740287    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa5b9038-d7ed-41a7-a041-b27f649c9420-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.745875 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.745852    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx6gf\" (UniqueName: \"kubernetes.io/projected/aa5b9038-d7ed-41a7-a041-b27f649c9420-kube-api-access-bx6gf\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:55.894186 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:55.894106    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:56.050109 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:56.050076    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"]
Apr 28 19:44:56.057341 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:44:56.053212    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa5b9038_d7ed_41a7_a041_b27f649c9420.slice/crio-0f7912dee038f99ab98f2bae3f7eecb79e915dc421897d35800a2f9395850a9f WatchSource:0}: Error finding container 0f7912dee038f99ab98f2bae3f7eecb79e915dc421897d35800a2f9395850a9f: Status 404 returned error can't find the container with id 0f7912dee038f99ab98f2bae3f7eecb79e915dc421897d35800a2f9395850a9f
Apr 28 19:44:56.447755 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:56.447714    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl" event={"ID":"aa5b9038-d7ed-41a7-a041-b27f649c9420","Type":"ContainerStarted","Data":"0087fe90b81244c74cc9b0579aefd43a958307f43909456289fbb83d106dfece"}
Apr 28 19:44:56.448005 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:56.447813    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl" event={"ID":"aa5b9038-d7ed-41a7-a041-b27f649c9420","Type":"ContainerStarted","Data":"0f7912dee038f99ab98f2bae3f7eecb79e915dc421897d35800a2f9395850a9f"}
Apr 28 19:44:57.453473 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:57.453435    2569 generic.go:358] "Generic (PLEG): container finished" podID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerID="0087fe90b81244c74cc9b0579aefd43a958307f43909456289fbb83d106dfece" exitCode=0
Apr 28 19:44:57.453846 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:57.453526    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl" event={"ID":"aa5b9038-d7ed-41a7-a041-b27f649c9420","Type":"ContainerDied","Data":"0087fe90b81244c74cc9b0579aefd43a958307f43909456289fbb83d106dfece"}
Apr 28 19:44:58.459380 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:58.459350    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl" event={"ID":"aa5b9038-d7ed-41a7-a041-b27f649c9420","Type":"ContainerStarted","Data":"6c85ab7d537d73b7ecd5ee21e20edbe23576fadd75ce32465051383c4b03d104"}
Apr 28 19:44:58.459380 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:58.459383    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl" event={"ID":"aa5b9038-d7ed-41a7-a041-b27f649c9420","Type":"ContainerStarted","Data":"0ceb9f5e1ab9cc33876466872af911cb88c1bf4c0e78455462f358874b58a317"}
Apr 28 19:44:58.459778 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:58.459487    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:44:58.482811 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:44:58.482772    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl" podStartSLOduration=3.482759508 podStartE2EDuration="3.482759508s" podCreationTimestamp="2026-04-28 19:44:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:44:58.478971591 +0000 UTC m=+1685.303311722" watchObservedRunningTime="2026-04-28 19:44:58.482759508 +0000 UTC m=+1685.307099615"
Apr 28 19:45:05.894656 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:45:05.894613    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:45:05.895154 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:45:05.894669    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:45:05.897425 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:45:05.897385    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:45:06.493302 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:45:06.493273    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:45:27.498044 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:45:27.498014    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:46:53.996677 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:46:53.996648    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:46:53.999890 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:46:53.999860    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:46:54.000527 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:46:54.000505    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:46:54.003634 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:46:54.003617    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:48:46.789539 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:46.789509    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"]
Apr 28 19:48:46.790027 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:46.789798    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl" podUID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerName="main" containerID="cri-o://0ceb9f5e1ab9cc33876466872af911cb88c1bf4c0e78455462f358874b58a317" gracePeriod=30
Apr 28 19:48:46.790027 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:46.789868    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl" podUID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerName="tokenizer" containerID="cri-o://6c85ab7d537d73b7ecd5ee21e20edbe23576fadd75ce32465051383c4b03d104" gracePeriod=30
Apr 28 19:48:47.369964 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:47.369929    2569 generic.go:358] "Generic (PLEG): container finished" podID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerID="0ceb9f5e1ab9cc33876466872af911cb88c1bf4c0e78455462f358874b58a317" exitCode=0
Apr 28 19:48:47.370142 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:47.370003    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl" event={"ID":"aa5b9038-d7ed-41a7-a041-b27f649c9420","Type":"ContainerDied","Data":"0ceb9f5e1ab9cc33876466872af911cb88c1bf4c0e78455462f358874b58a317"}
Apr 28 19:48:47.497268 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:48:47.497240    2569 logging.go:55] [core] [Channel #1064 SubChannel #1065]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.54:9003", ServerName: "10.133.0.54:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.54:9003: connect: connection refused"
Apr 28 19:48:48.036144 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.036122    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:48:48.166600 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.166532    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-uds\") pod \"aa5b9038-d7ed-41a7-a041-b27f649c9420\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") "
Apr 28 19:48:48.166600 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.166575    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-kserve-provision-location\") pod \"aa5b9038-d7ed-41a7-a041-b27f649c9420\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") "
Apr 28 19:48:48.166824 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.166603    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx6gf\" (UniqueName: \"kubernetes.io/projected/aa5b9038-d7ed-41a7-a041-b27f649c9420-kube-api-access-bx6gf\") pod \"aa5b9038-d7ed-41a7-a041-b27f649c9420\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") "
Apr 28 19:48:48.166824 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.166635    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-tmp\") pod \"aa5b9038-d7ed-41a7-a041-b27f649c9420\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") "
Apr 28 19:48:48.166824 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.166698    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-cache\") pod \"aa5b9038-d7ed-41a7-a041-b27f649c9420\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") "
Apr 28 19:48:48.166824 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.166768    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa5b9038-d7ed-41a7-a041-b27f649c9420-tls-certs\") pod \"aa5b9038-d7ed-41a7-a041-b27f649c9420\" (UID: \"aa5b9038-d7ed-41a7-a041-b27f649c9420\") "
Apr 28 19:48:48.167034 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.166834    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "aa5b9038-d7ed-41a7-a041-b27f649c9420" (UID: "aa5b9038-d7ed-41a7-a041-b27f649c9420"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:48:48.167034 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.166953    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "aa5b9038-d7ed-41a7-a041-b27f649c9420" (UID: "aa5b9038-d7ed-41a7-a041-b27f649c9420"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:48:48.167147 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.167057    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "aa5b9038-d7ed-41a7-a041-b27f649c9420" (UID: "aa5b9038-d7ed-41a7-a041-b27f649c9420"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:48:48.167147 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.167104    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-cache\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:48:48.167147 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.167116    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-uds\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:48:48.167331 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.167307    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "aa5b9038-d7ed-41a7-a041-b27f649c9420" (UID: "aa5b9038-d7ed-41a7-a041-b27f649c9420"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:48:48.168923 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.168900    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa5b9038-d7ed-41a7-a041-b27f649c9420-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aa5b9038-d7ed-41a7-a041-b27f649c9420" (UID: "aa5b9038-d7ed-41a7-a041-b27f649c9420"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:48:48.168923 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.168912    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa5b9038-d7ed-41a7-a041-b27f649c9420-kube-api-access-bx6gf" (OuterVolumeSpecName: "kube-api-access-bx6gf") pod "aa5b9038-d7ed-41a7-a041-b27f649c9420" (UID: "aa5b9038-d7ed-41a7-a041-b27f649c9420"). InnerVolumeSpecName "kube-api-access-bx6gf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:48:48.267952 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.267918    2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-kserve-provision-location\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:48:48.267952 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.267945    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx6gf\" (UniqueName: \"kubernetes.io/projected/aa5b9038-d7ed-41a7-a041-b27f649c9420-kube-api-access-bx6gf\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:48:48.267952 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.267956    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa5b9038-d7ed-41a7-a041-b27f649c9420-tokenizer-tmp\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:48:48.268170 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.267965    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa5b9038-d7ed-41a7-a041-b27f649c9420-tls-certs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:48:48.375450 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.375383    2569 generic.go:358] "Generic (PLEG): container finished" podID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerID="6c85ab7d537d73b7ecd5ee21e20edbe23576fadd75ce32465051383c4b03d104" exitCode=0
Apr 28 19:48:48.375621 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.375467    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl" event={"ID":"aa5b9038-d7ed-41a7-a041-b27f649c9420","Type":"ContainerDied","Data":"6c85ab7d537d73b7ecd5ee21e20edbe23576fadd75ce32465051383c4b03d104"}
Apr 28 19:48:48.375621 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.375511    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl" event={"ID":"aa5b9038-d7ed-41a7-a041-b27f649c9420","Type":"ContainerDied","Data":"0f7912dee038f99ab98f2bae3f7eecb79e915dc421897d35800a2f9395850a9f"}
Apr 28 19:48:48.375621 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.375527    2569 scope.go:117] "RemoveContainer" containerID="6c85ab7d537d73b7ecd5ee21e20edbe23576fadd75ce32465051383c4b03d104"
Apr 28 19:48:48.375621 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.375481    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"
Apr 28 19:48:48.385115 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.385095    2569 scope.go:117] "RemoveContainer" containerID="0ceb9f5e1ab9cc33876466872af911cb88c1bf4c0e78455462f358874b58a317"
Apr 28 19:48:48.393191 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.393175    2569 scope.go:117] "RemoveContainer" containerID="0087fe90b81244c74cc9b0579aefd43a958307f43909456289fbb83d106dfece"
Apr 28 19:48:48.399334 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.399308    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"]
Apr 28 19:48:48.403408 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.403369    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl"]
Apr 28 19:48:48.403928 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.403908    2569 scope.go:117] "RemoveContainer" containerID="6c85ab7d537d73b7ecd5ee21e20edbe23576fadd75ce32465051383c4b03d104"
Apr 28 19:48:48.404181 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:48:48.404162    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c85ab7d537d73b7ecd5ee21e20edbe23576fadd75ce32465051383c4b03d104\": container with ID starting with 6c85ab7d537d73b7ecd5ee21e20edbe23576fadd75ce32465051383c4b03d104 not found: ID does not exist" containerID="6c85ab7d537d73b7ecd5ee21e20edbe23576fadd75ce32465051383c4b03d104"
Apr 28 19:48:48.404269 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.404191    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c85ab7d537d73b7ecd5ee21e20edbe23576fadd75ce32465051383c4b03d104"} err="failed to get container status \"6c85ab7d537d73b7ecd5ee21e20edbe23576fadd75ce32465051383c4b03d104\": rpc error: code = NotFound desc = could not find container \"6c85ab7d537d73b7ecd5ee21e20edbe23576fadd75ce32465051383c4b03d104\": container with ID starting with 6c85ab7d537d73b7ecd5ee21e20edbe23576fadd75ce32465051383c4b03d104 not found: ID does not exist"
Apr 28 19:48:48.404269 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.404221    2569 scope.go:117] "RemoveContainer" containerID="0ceb9f5e1ab9cc33876466872af911cb88c1bf4c0e78455462f358874b58a317"
Apr 28 19:48:48.404510 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:48:48.404490    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ceb9f5e1ab9cc33876466872af911cb88c1bf4c0e78455462f358874b58a317\": container with ID starting with 0ceb9f5e1ab9cc33876466872af911cb88c1bf4c0e78455462f358874b58a317 not found: ID does not exist" containerID="0ceb9f5e1ab9cc33876466872af911cb88c1bf4c0e78455462f358874b58a317"
Apr 28 19:48:48.404556 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.404516    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ceb9f5e1ab9cc33876466872af911cb88c1bf4c0e78455462f358874b58a317"} err="failed to get container status \"0ceb9f5e1ab9cc33876466872af911cb88c1bf4c0e78455462f358874b58a317\": rpc error: code = NotFound desc = could not find container \"0ceb9f5e1ab9cc33876466872af911cb88c1bf4c0e78455462f358874b58a317\": container with ID starting with 0ceb9f5e1ab9cc33876466872af911cb88c1bf4c0e78455462f358874b58a317 not found: ID does not exist"
Apr 28 19:48:48.404556 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.404535    2569 scope.go:117] "RemoveContainer" containerID="0087fe90b81244c74cc9b0579aefd43a958307f43909456289fbb83d106dfece"
Apr 28 19:48:48.404754 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:48:48.404737    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0087fe90b81244c74cc9b0579aefd43a958307f43909456289fbb83d106dfece\": container with ID starting with 0087fe90b81244c74cc9b0579aefd43a958307f43909456289fbb83d106dfece not found: ID does not exist" containerID="0087fe90b81244c74cc9b0579aefd43a958307f43909456289fbb83d106dfece"
Apr 28 19:48:48.404799 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.404760    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0087fe90b81244c74cc9b0579aefd43a958307f43909456289fbb83d106dfece"} err="failed to get container status \"0087fe90b81244c74cc9b0579aefd43a958307f43909456289fbb83d106dfece\": rpc error: code = NotFound desc = could not find container \"0087fe90b81244c74cc9b0579aefd43a958307f43909456289fbb83d106dfece\": container with ID starting with 0087fe90b81244c74cc9b0579aefd43a958307f43909456289fbb83d106dfece not found: ID does not exist"
Apr 28 19:48:48.497572 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:48.497539    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-7d547zwdvl" podUID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.133.0.54:9003\" within 1s: context deadline exceeded"
Apr 28 19:48:48.497723 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:48:48.497617    2569 logging.go:55] [core] [Channel #1064 SubChannel #1065]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.54:9003", ServerName: "10.133.0.54:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.54:9003: operation was canceled"
Apr 28 19:48:49.944016 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:48:49.943982    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa5b9038-d7ed-41a7-a041-b27f649c9420" path="/var/lib/kubelet/pods/aa5b9038-d7ed-41a7-a041-b27f649c9420/volumes"
Apr 28 19:49:00.809180 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.809140    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"]
Apr 28 19:49:00.811616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.809753    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerName="main"
Apr 28 19:49:00.811616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.809770    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerName="main"
Apr 28 19:49:00.811616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.809781    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerName="storage-initializer"
Apr 28 19:49:00.811616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.809787    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerName="storage-initializer"
Apr 28 19:49:00.811616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.809802    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerName="tokenizer"
Apr 28 19:49:00.811616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.809809    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerName="tokenizer"
Apr 28 19:49:00.811616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.809883    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerName="tokenizer"
Apr 28 19:49:00.811616 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.809892    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="aa5b9038-d7ed-41a7-a041-b27f649c9420" containerName="main"
Apr 28 19:49:00.813363 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.813339    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:00.816019 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.815998    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-dxkc8\""
Apr 28 19:49:00.816158 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.815998    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Apr 28 19:49:00.825644 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.825619    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"]
Apr 28 19:49:00.980559 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.980528    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:00.980735 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.980572    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/429e685c-9bc2-4d2f-a571-a0b812a1e264-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:00.980735 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.980629    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v426p\" (UniqueName: \"kubernetes.io/projected/429e685c-9bc2-4d2f-a571-a0b812a1e264-kube-api-access-v426p\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:00.980735 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.980696    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:00.980735 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.980712    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:00.980735 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:00.980733    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.086131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.081975    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.086131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.082035    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/429e685c-9bc2-4d2f-a571-a0b812a1e264-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.086131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.082105    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v426p\" (UniqueName: \"kubernetes.io/projected/429e685c-9bc2-4d2f-a571-a0b812a1e264-kube-api-access-v426p\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.086131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.082160    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.086131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.082186    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.086131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.082217    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.086131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.082683    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.086131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.082917    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.086131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.083559    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.086131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.083786    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.086687 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.086429    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/429e685c-9bc2-4d2f-a571-a0b812a1e264-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.092889 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.092868    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v426p\" (UniqueName: \"kubernetes.io/projected/429e685c-9bc2-4d2f-a571-a0b812a1e264-kube-api-access-v426p\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.126097 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.126073    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:01.269128 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.269099    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"]
Apr 28 19:49:01.270880 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:49:01.270855    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod429e685c_9bc2_4d2f_a571_a0b812a1e264.slice/crio-f74b2eabbe6b02ecd6019cdaad82d543d157c9d56e3069b9b79dc60369a3e9a8 WatchSource:0}: Error finding container f74b2eabbe6b02ecd6019cdaad82d543d157c9d56e3069b9b79dc60369a3e9a8: Status 404 returned error can't find the container with id f74b2eabbe6b02ecd6019cdaad82d543d157c9d56e3069b9b79dc60369a3e9a8
Apr 28 19:49:01.272794 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.272775    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 28 19:49:01.432886 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.432846    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg" event={"ID":"429e685c-9bc2-4d2f-a571-a0b812a1e264","Type":"ContainerStarted","Data":"068067ab335ab01473ae8c9e54fd56ae3541c14e93c2cfc6f37b19e47f054383"}
Apr 28 19:49:01.433058 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:01.432891    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg" event={"ID":"429e685c-9bc2-4d2f-a571-a0b812a1e264","Type":"ContainerStarted","Data":"f74b2eabbe6b02ecd6019cdaad82d543d157c9d56e3069b9b79dc60369a3e9a8"}
Apr 28 19:49:02.437969 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:02.437934    2569 generic.go:358] "Generic (PLEG): container finished" podID="429e685c-9bc2-4d2f-a571-a0b812a1e264" containerID="068067ab335ab01473ae8c9e54fd56ae3541c14e93c2cfc6f37b19e47f054383" exitCode=0
Apr 28 19:49:02.438379 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:02.437974    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg" event={"ID":"429e685c-9bc2-4d2f-a571-a0b812a1e264","Type":"ContainerDied","Data":"068067ab335ab01473ae8c9e54fd56ae3541c14e93c2cfc6f37b19e47f054383"}
Apr 28 19:49:03.443834 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:03.443786    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg" event={"ID":"429e685c-9bc2-4d2f-a571-a0b812a1e264","Type":"ContainerStarted","Data":"e96eca1101b50524a1a414fbf3e85a77b4c7f95fadc0704a20b44475026eab74"}
Apr 28 19:49:03.443834 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:03.443829    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg" event={"ID":"429e685c-9bc2-4d2f-a571-a0b812a1e264","Type":"ContainerStarted","Data":"4c22e6b4fe6a7205676e434132a277a1be10f3991a2134ba7d16e06c52454e7d"}
Apr 28 19:49:03.444384 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:03.443993    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:03.468727 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:03.468674    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg" podStartSLOduration=3.468659081 podStartE2EDuration="3.468659081s" podCreationTimestamp="2026-04-28 19:49:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:49:03.466658212 +0000 UTC m=+1930.290998320" watchObservedRunningTime="2026-04-28 19:49:03.468659081 +0000 UTC m=+1930.292999185"
Apr 28 19:49:11.126759 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:11.126701    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:11.126759 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:11.126765    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:11.129587 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:11.129562    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:11.480294 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:11.480258    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:32.484792 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:32.484761    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:49:45.567120 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:45.566920    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"]
Apr 28 19:49:45.574144 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:45.574093    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb" podUID="69976efd-d254-4490-8742-c40b69b6d7f5" containerName="main" containerID="cri-o://3b3eb6f971b85b0914da32bdf690065c8b0c4625e0e9b98d4310a6ac063e28fb" gracePeriod=30
Apr 28 19:49:45.574306 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:45.574134    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb" podUID="69976efd-d254-4490-8742-c40b69b6d7f5" containerName="tokenizer" containerID="cri-o://3457083dcbf9b76c07cd2d8458570d58ab11a19fb08a9e6412e56e7463354fde" gracePeriod=30
Apr 28 19:49:46.623899 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:46.623867    2569 generic.go:358] "Generic (PLEG): container finished" podID="69976efd-d254-4490-8742-c40b69b6d7f5" containerID="3b3eb6f971b85b0914da32bdf690065c8b0c4625e0e9b98d4310a6ac063e28fb" exitCode=0
Apr 28 19:49:46.624234 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:46.623934    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb" event={"ID":"69976efd-d254-4490-8742-c40b69b6d7f5","Type":"ContainerDied","Data":"3b3eb6f971b85b0914da32bdf690065c8b0c4625e0e9b98d4310a6ac063e28fb"}
Apr 28 19:49:46.837570 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:46.837506    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:49:47.009488 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.009459    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69976efd-d254-4490-8742-c40b69b6d7f5-tls-certs\") pod \"69976efd-d254-4490-8742-c40b69b6d7f5\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") "
Apr 28 19:49:47.009681 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.009508    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-kserve-provision-location\") pod \"69976efd-d254-4490-8742-c40b69b6d7f5\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") "
Apr 28 19:49:47.009681 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.009527    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-cache\") pod \"69976efd-d254-4490-8742-c40b69b6d7f5\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") "
Apr 28 19:49:47.009681 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.009642    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-tmp\") pod \"69976efd-d254-4490-8742-c40b69b6d7f5\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") "
Apr 28 19:49:47.009880 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.009700    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmg97\" (UniqueName: \"kubernetes.io/projected/69976efd-d254-4490-8742-c40b69b6d7f5-kube-api-access-dmg97\") pod \"69976efd-d254-4490-8742-c40b69b6d7f5\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") "
Apr 28 19:49:47.009880 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.009784    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-uds\") pod \"69976efd-d254-4490-8742-c40b69b6d7f5\" (UID: \"69976efd-d254-4490-8742-c40b69b6d7f5\") "
Apr 28 19:49:47.009880 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.009802    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "69976efd-d254-4490-8742-c40b69b6d7f5" (UID: "69976efd-d254-4490-8742-c40b69b6d7f5"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:49:47.010039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.009964    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "69976efd-d254-4490-8742-c40b69b6d7f5" (UID: "69976efd-d254-4490-8742-c40b69b6d7f5"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:49:47.010098 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.010078    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "69976efd-d254-4490-8742-c40b69b6d7f5" (UID: "69976efd-d254-4490-8742-c40b69b6d7f5"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:49:47.010193 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.010168    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-uds\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:49:47.010193 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.010192    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-cache\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:49:47.010386 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.010204    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-tokenizer-tmp\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:49:47.010481 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.010450    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "69976efd-d254-4490-8742-c40b69b6d7f5" (UID: "69976efd-d254-4490-8742-c40b69b6d7f5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:49:47.011729 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.011706    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69976efd-d254-4490-8742-c40b69b6d7f5-kube-api-access-dmg97" (OuterVolumeSpecName: "kube-api-access-dmg97") pod "69976efd-d254-4490-8742-c40b69b6d7f5" (UID: "69976efd-d254-4490-8742-c40b69b6d7f5"). InnerVolumeSpecName "kube-api-access-dmg97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:49:47.011813 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.011794    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69976efd-d254-4490-8742-c40b69b6d7f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "69976efd-d254-4490-8742-c40b69b6d7f5" (UID: "69976efd-d254-4490-8742-c40b69b6d7f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:49:47.111125 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.111047    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmg97\" (UniqueName: \"kubernetes.io/projected/69976efd-d254-4490-8742-c40b69b6d7f5-kube-api-access-dmg97\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:49:47.111125 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.111075    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69976efd-d254-4490-8742-c40b69b6d7f5-tls-certs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:49:47.111125 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.111085    2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/69976efd-d254-4490-8742-c40b69b6d7f5-kserve-provision-location\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:49:47.629735 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.629700    2569 generic.go:358] "Generic (PLEG): container finished" podID="69976efd-d254-4490-8742-c40b69b6d7f5" containerID="3457083dcbf9b76c07cd2d8458570d58ab11a19fb08a9e6412e56e7463354fde" exitCode=0
Apr 28 19:49:47.630106 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.629737    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb" event={"ID":"69976efd-d254-4490-8742-c40b69b6d7f5","Type":"ContainerDied","Data":"3457083dcbf9b76c07cd2d8458570d58ab11a19fb08a9e6412e56e7463354fde"}
Apr 28 19:49:47.630106 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.629764    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb" event={"ID":"69976efd-d254-4490-8742-c40b69b6d7f5","Type":"ContainerDied","Data":"d5582f985fc26a03ed328044f181514adb1679886b255ff0decb7b37b33470f0"}
Apr 28 19:49:47.630106 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.629776    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"
Apr 28 19:49:47.630106 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.629780    2569 scope.go:117] "RemoveContainer" containerID="3457083dcbf9b76c07cd2d8458570d58ab11a19fb08a9e6412e56e7463354fde"
Apr 28 19:49:47.640980 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.640957    2569 scope.go:117] "RemoveContainer" containerID="3b3eb6f971b85b0914da32bdf690065c8b0c4625e0e9b98d4310a6ac063e28fb"
Apr 28 19:49:47.650061 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.650043    2569 scope.go:117] "RemoveContainer" containerID="86c19bab6f8e6ed6e996de77cd16261114ee4466ef7816c9d5d183c006072467"
Apr 28 19:49:47.657293 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.657263    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"]
Apr 28 19:49:47.659099 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.659081    2569 scope.go:117] "RemoveContainer" containerID="3457083dcbf9b76c07cd2d8458570d58ab11a19fb08a9e6412e56e7463354fde"
Apr 28 19:49:47.659420 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:49:47.659381    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3457083dcbf9b76c07cd2d8458570d58ab11a19fb08a9e6412e56e7463354fde\": container with ID starting with 3457083dcbf9b76c07cd2d8458570d58ab11a19fb08a9e6412e56e7463354fde not found: ID does not exist" containerID="3457083dcbf9b76c07cd2d8458570d58ab11a19fb08a9e6412e56e7463354fde"
Apr 28 19:49:47.659562 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.659538    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3457083dcbf9b76c07cd2d8458570d58ab11a19fb08a9e6412e56e7463354fde"} err="failed to get container status \"3457083dcbf9b76c07cd2d8458570d58ab11a19fb08a9e6412e56e7463354fde\": rpc error: code = NotFound desc = could not find container \"3457083dcbf9b76c07cd2d8458570d58ab11a19fb08a9e6412e56e7463354fde\": container with ID starting with 3457083dcbf9b76c07cd2d8458570d58ab11a19fb08a9e6412e56e7463354fde not found: ID does not exist"
Apr 28 19:49:47.659642 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.659567    2569 scope.go:117] "RemoveContainer" containerID="3b3eb6f971b85b0914da32bdf690065c8b0c4625e0e9b98d4310a6ac063e28fb"
Apr 28 19:49:47.659843 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:49:47.659826    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b3eb6f971b85b0914da32bdf690065c8b0c4625e0e9b98d4310a6ac063e28fb\": container with ID starting with 3b3eb6f971b85b0914da32bdf690065c8b0c4625e0e9b98d4310a6ac063e28fb not found: ID does not exist" containerID="3b3eb6f971b85b0914da32bdf690065c8b0c4625e0e9b98d4310a6ac063e28fb"
Apr 28 19:49:47.659924 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.659851    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b3eb6f971b85b0914da32bdf690065c8b0c4625e0e9b98d4310a6ac063e28fb"} err="failed to get container status \"3b3eb6f971b85b0914da32bdf690065c8b0c4625e0e9b98d4310a6ac063e28fb\": rpc error: code = NotFound desc = could not find container \"3b3eb6f971b85b0914da32bdf690065c8b0c4625e0e9b98d4310a6ac063e28fb\": container with ID starting with 3b3eb6f971b85b0914da32bdf690065c8b0c4625e0e9b98d4310a6ac063e28fb not found: ID does not exist"
Apr 28 19:49:47.659924 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.659869    2569 scope.go:117] "RemoveContainer" containerID="86c19bab6f8e6ed6e996de77cd16261114ee4466ef7816c9d5d183c006072467"
Apr 28 19:49:47.660108 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:49:47.660094    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86c19bab6f8e6ed6e996de77cd16261114ee4466ef7816c9d5d183c006072467\": container with ID starting with 86c19bab6f8e6ed6e996de77cd16261114ee4466ef7816c9d5d183c006072467 not found: ID does not exist" containerID="86c19bab6f8e6ed6e996de77cd16261114ee4466ef7816c9d5d183c006072467"
Apr 28 19:49:47.660152 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.660111    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86c19bab6f8e6ed6e996de77cd16261114ee4466ef7816c9d5d183c006072467"} err="failed to get container status \"86c19bab6f8e6ed6e996de77cd16261114ee4466ef7816c9d5d183c006072467\": rpc error: code = NotFound desc = could not find container \"86c19bab6f8e6ed6e996de77cd16261114ee4466ef7816c9d5d183c006072467\": container with ID starting with 86c19bab6f8e6ed6e996de77cd16261114ee4466ef7816c9d5d183c006072467 not found: ID does not exist"
Apr 28 19:49:47.660192 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.660170    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6849fb6fspkqb"]
Apr 28 19:49:47.944209 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:47.944176    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69976efd-d254-4490-8742-c40b69b6d7f5" path="/var/lib/kubelet/pods/69976efd-d254-4490-8742-c40b69b6d7f5/volumes"
Apr 28 19:49:55.064385 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.064345    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"]
Apr 28 19:49:55.064938 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.064920    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69976efd-d254-4490-8742-c40b69b6d7f5" containerName="storage-initializer"
Apr 28 19:49:55.064938 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.064940    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="69976efd-d254-4490-8742-c40b69b6d7f5" containerName="storage-initializer"
Apr 28 19:49:55.065059 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.064962    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69976efd-d254-4490-8742-c40b69b6d7f5" containerName="main"
Apr 28 19:49:55.065059 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.064971    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="69976efd-d254-4490-8742-c40b69b6d7f5" containerName="main"
Apr 28 19:49:55.065059 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.065004    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69976efd-d254-4490-8742-c40b69b6d7f5" containerName="tokenizer"
Apr 28 19:49:55.065059 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.065012    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="69976efd-d254-4490-8742-c40b69b6d7f5" containerName="tokenizer"
Apr 28 19:49:55.065248 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.065092    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="69976efd-d254-4490-8742-c40b69b6d7f5" containerName="main"
Apr 28 19:49:55.065248 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.065103    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="69976efd-d254-4490-8742-c40b69b6d7f5" containerName="tokenizer"
Apr 28 19:49:55.069207 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.069188    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.071880 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.071858    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-test-epp-sa-dockercfg-8djkg\""
Apr 28 19:49:55.072004 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.071886    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-test-kserve-self-signed-certs\""
Apr 28 19:49:55.079018 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.078997    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"]
Apr 28 19:49:55.081063 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.081043    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qskz\" (UniqueName: \"kubernetes.io/projected/f52f1f13-71c0-456e-911b-da69e4994a7e-kube-api-access-2qskz\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.081144 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.081075    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f52f1f13-71c0-456e-911b-da69e4994a7e-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.081144 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.081102    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-cache\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.081273 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.081251    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-tmp\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.081352 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.081334    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.081432 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.081390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.181841 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.181811    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-cache\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.182002 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.181878    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-tmp\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.182002 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.181917    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.182002 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.181942    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.182193 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.182010    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2qskz\" (UniqueName: \"kubernetes.io/projected/f52f1f13-71c0-456e-911b-da69e4994a7e-kube-api-access-2qskz\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.182193 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.182160    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f52f1f13-71c0-456e-911b-da69e4994a7e-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.182303 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.182255    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-cache\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.182303 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.182277    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.182374 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.182333    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.182724 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.182504    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-tmp\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.184702 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.184680    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f52f1f13-71c0-456e-911b-da69e4994a7e-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.190476 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.190456    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qskz\" (UniqueName: \"kubernetes.io/projected/f52f1f13-71c0-456e-911b-da69e4994a7e-kube-api-access-2qskz\") pod \"router-with-refs-test-kserve-router-scheduler-7448f76699-j456s\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.379839 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.379749    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:55.517198 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.517172    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"]
Apr 28 19:49:55.520107 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:49:55.520064    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52f1f13_71c0_456e_911b_da69e4994a7e.slice/crio-326051c4c7c5959f5e92f1220fd67090a5d5b69afe75562732dd8b2db405950e WatchSource:0}: Error finding container 326051c4c7c5959f5e92f1220fd67090a5d5b69afe75562732dd8b2db405950e: Status 404 returned error can't find the container with id 326051c4c7c5959f5e92f1220fd67090a5d5b69afe75562732dd8b2db405950e
Apr 28 19:49:55.669276 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.669190    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s" event={"ID":"f52f1f13-71c0-456e-911b-da69e4994a7e","Type":"ContainerStarted","Data":"55454a1e838e88dc2ebf5d7d75a67f4e752db04a6c793c7dfba04f476278042a"}
Apr 28 19:49:55.669276 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:55.669230    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s" event={"ID":"f52f1f13-71c0-456e-911b-da69e4994a7e","Type":"ContainerStarted","Data":"326051c4c7c5959f5e92f1220fd67090a5d5b69afe75562732dd8b2db405950e"}
Apr 28 19:49:56.674942 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:56.674905    2569 generic.go:358] "Generic (PLEG): container finished" podID="f52f1f13-71c0-456e-911b-da69e4994a7e" containerID="55454a1e838e88dc2ebf5d7d75a67f4e752db04a6c793c7dfba04f476278042a" exitCode=0
Apr 28 19:49:56.675427 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:56.674951    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s" event={"ID":"f52f1f13-71c0-456e-911b-da69e4994a7e","Type":"ContainerDied","Data":"55454a1e838e88dc2ebf5d7d75a67f4e752db04a6c793c7dfba04f476278042a"}
Apr 28 19:49:57.682150 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:57.682116    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s" event={"ID":"f52f1f13-71c0-456e-911b-da69e4994a7e","Type":"ContainerStarted","Data":"e7e19b955e25c287f0971b714a9b4d382e74df71fcedc300f81a724bedf8257f"}
Apr 28 19:49:57.682150 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:57.682148    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s" event={"ID":"f52f1f13-71c0-456e-911b-da69e4994a7e","Type":"ContainerStarted","Data":"716488e9bffa775c7df907bf4c5f0b3ba3bcc04f6c33e28f97c44718123ed277"}
Apr 28 19:49:57.682711 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:57.682182    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:49:57.710034 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:49:57.709989    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s" podStartSLOduration=2.709976589 podStartE2EDuration="2.709976589s" podCreationTimestamp="2026-04-28 19:49:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:49:57.706750774 +0000 UTC m=+1984.531090882" watchObservedRunningTime="2026-04-28 19:49:57.709976589 +0000 UTC m=+1984.534316698"
Apr 28 19:50:05.379896 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:05.379856    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:50:05.379896 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:05.379904    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:50:05.382812 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:05.382788    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:50:05.717063 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:05.717027    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:50:26.722164 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:26.722136    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:50:32.692654 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:32.692623    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"]
Apr 28 19:50:32.693113 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:32.693026    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg" podUID="429e685c-9bc2-4d2f-a571-a0b812a1e264" containerName="main" containerID="cri-o://4c22e6b4fe6a7205676e434132a277a1be10f3991a2134ba7d16e06c52454e7d" gracePeriod=30
Apr 28 19:50:32.693181 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:32.693094    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg" podUID="429e685c-9bc2-4d2f-a571-a0b812a1e264" containerName="tokenizer" containerID="cri-o://e96eca1101b50524a1a414fbf3e85a77b4c7f95fadc0704a20b44475026eab74" gracePeriod=30
Apr 28 19:50:32.824613 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:32.824575    2569 generic.go:358] "Generic (PLEG): container finished" podID="429e685c-9bc2-4d2f-a571-a0b812a1e264" containerID="4c22e6b4fe6a7205676e434132a277a1be10f3991a2134ba7d16e06c52454e7d" exitCode=0
Apr 28 19:50:32.824760 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:32.824643    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg" event={"ID":"429e685c-9bc2-4d2f-a571-a0b812a1e264","Type":"ContainerDied","Data":"4c22e6b4fe6a7205676e434132a277a1be10f3991a2134ba7d16e06c52454e7d"}
Apr 28 19:50:33.830994 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:33.830964    2569 generic.go:358] "Generic (PLEG): container finished" podID="429e685c-9bc2-4d2f-a571-a0b812a1e264" containerID="e96eca1101b50524a1a414fbf3e85a77b4c7f95fadc0704a20b44475026eab74" exitCode=0
Apr 28 19:50:33.831327 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:33.831003    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg" event={"ID":"429e685c-9bc2-4d2f-a571-a0b812a1e264","Type":"ContainerDied","Data":"e96eca1101b50524a1a414fbf3e85a77b4c7f95fadc0704a20b44475026eab74"}
Apr 28 19:50:33.956672 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:33.956614    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:50:34.052046 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.052015    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-tmp\") pod \"429e685c-9bc2-4d2f-a571-a0b812a1e264\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") "
Apr 28 19:50:34.052217 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.052075    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v426p\" (UniqueName: \"kubernetes.io/projected/429e685c-9bc2-4d2f-a571-a0b812a1e264-kube-api-access-v426p\") pod \"429e685c-9bc2-4d2f-a571-a0b812a1e264\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") "
Apr 28 19:50:34.052217 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.052122    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-cache\") pod \"429e685c-9bc2-4d2f-a571-a0b812a1e264\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") "
Apr 28 19:50:34.052217 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.052172    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-kserve-provision-location\") pod \"429e685c-9bc2-4d2f-a571-a0b812a1e264\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") "
Apr 28 19:50:34.052217 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.052196    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-uds\") pod \"429e685c-9bc2-4d2f-a571-a0b812a1e264\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") "
Apr 28 19:50:34.052471 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.052298    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/429e685c-9bc2-4d2f-a571-a0b812a1e264-tls-certs\") pod \"429e685c-9bc2-4d2f-a571-a0b812a1e264\" (UID: \"429e685c-9bc2-4d2f-a571-a0b812a1e264\") "
Apr 28 19:50:34.052471 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.052461    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "429e685c-9bc2-4d2f-a571-a0b812a1e264" (UID: "429e685c-9bc2-4d2f-a571-a0b812a1e264"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:50:34.052581 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.052431    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "429e685c-9bc2-4d2f-a571-a0b812a1e264" (UID: "429e685c-9bc2-4d2f-a571-a0b812a1e264"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:50:34.052581 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.052509    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "429e685c-9bc2-4d2f-a571-a0b812a1e264" (UID: "429e685c-9bc2-4d2f-a571-a0b812a1e264"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:50:34.052705 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.052687    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-tmp\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:50:34.052760 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.052711    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-cache\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:50:34.052760 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.052724    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-tokenizer-uds\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:50:34.053095 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.053064    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "429e685c-9bc2-4d2f-a571-a0b812a1e264" (UID: "429e685c-9bc2-4d2f-a571-a0b812a1e264"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:50:34.054300 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.054279    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/429e685c-9bc2-4d2f-a571-a0b812a1e264-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "429e685c-9bc2-4d2f-a571-a0b812a1e264" (UID: "429e685c-9bc2-4d2f-a571-a0b812a1e264"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:50:34.054300 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.054287    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/429e685c-9bc2-4d2f-a571-a0b812a1e264-kube-api-access-v426p" (OuterVolumeSpecName: "kube-api-access-v426p") pod "429e685c-9bc2-4d2f-a571-a0b812a1e264" (UID: "429e685c-9bc2-4d2f-a571-a0b812a1e264"). InnerVolumeSpecName "kube-api-access-v426p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:50:34.153555 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.153524    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v426p\" (UniqueName: \"kubernetes.io/projected/429e685c-9bc2-4d2f-a571-a0b812a1e264-kube-api-access-v426p\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:50:34.153555 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.153550    2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/429e685c-9bc2-4d2f-a571-a0b812a1e264-kserve-provision-location\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:50:34.153555 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.153560    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/429e685c-9bc2-4d2f-a571-a0b812a1e264-tls-certs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:50:34.836581 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.836550    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"
Apr 28 19:50:34.836965 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.836549    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg" event={"ID":"429e685c-9bc2-4d2f-a571-a0b812a1e264","Type":"ContainerDied","Data":"f74b2eabbe6b02ecd6019cdaad82d543d157c9d56e3069b9b79dc60369a3e9a8"}
Apr 28 19:50:34.836965 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.836680    2569 scope.go:117] "RemoveContainer" containerID="e96eca1101b50524a1a414fbf3e85a77b4c7f95fadc0704a20b44475026eab74"
Apr 28 19:50:34.846080 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.846061    2569 scope.go:117] "RemoveContainer" containerID="4c22e6b4fe6a7205676e434132a277a1be10f3991a2134ba7d16e06c52454e7d"
Apr 28 19:50:34.854125 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.854107    2569 scope.go:117] "RemoveContainer" containerID="068067ab335ab01473ae8c9e54fd56ae3541c14e93c2cfc6f37b19e47f054383"
Apr 28 19:50:34.859757 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.859724    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"]
Apr 28 19:50:34.864926 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:34.864905    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-57875d44fccg"]
Apr 28 19:50:35.942989 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:50:35.942958    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="429e685c-9bc2-4d2f-a571-a0b812a1e264" path="/var/lib/kubelet/pods/429e685c-9bc2-4d2f-a571-a0b812a1e264/volumes"
Apr 28 19:51:54.030933 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:51:54.030909    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:51:54.034145 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:51:54.034125    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:51:54.040574 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:51:54.040557    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:51:54.043453 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:51:54.043435    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:52:30.889791 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:30.889760    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"]
Apr 28 19:52:30.890370 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:30.890185    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s" podUID="f52f1f13-71c0-456e-911b-da69e4994a7e" containerName="main" containerID="cri-o://716488e9bffa775c7df907bf4c5f0b3ba3bcc04f6c33e28f97c44718123ed277" gracePeriod=30
Apr 28 19:52:30.890370 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:30.890229    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s" podUID="f52f1f13-71c0-456e-911b-da69e4994a7e" containerName="tokenizer" containerID="cri-o://e7e19b955e25c287f0971b714a9b4d382e74df71fcedc300f81a724bedf8257f" gracePeriod=30
Apr 28 19:52:31.290874 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:31.290833    2569 generic.go:358] "Generic (PLEG): container finished" podID="f52f1f13-71c0-456e-911b-da69e4994a7e" containerID="716488e9bffa775c7df907bf4c5f0b3ba3bcc04f6c33e28f97c44718123ed277" exitCode=0
Apr 28 19:52:31.291052 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:31.290880    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s" event={"ID":"f52f1f13-71c0-456e-911b-da69e4994a7e","Type":"ContainerDied","Data":"716488e9bffa775c7df907bf4c5f0b3ba3bcc04f6c33e28f97c44718123ed277"}
Apr 28 19:52:32.046144 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.046121    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:52:32.110817 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.110753    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qskz\" (UniqueName: \"kubernetes.io/projected/f52f1f13-71c0-456e-911b-da69e4994a7e-kube-api-access-2qskz\") pod \"f52f1f13-71c0-456e-911b-da69e4994a7e\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") "
Apr 28 19:52:32.110817 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.110803    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f52f1f13-71c0-456e-911b-da69e4994a7e-tls-certs\") pod \"f52f1f13-71c0-456e-911b-da69e4994a7e\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") "
Apr 28 19:52:32.111006 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.110917    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-kserve-provision-location\") pod \"f52f1f13-71c0-456e-911b-da69e4994a7e\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") "
Apr 28 19:52:32.111006 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.110960    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-uds\") pod \"f52f1f13-71c0-456e-911b-da69e4994a7e\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") "
Apr 28 19:52:32.111006 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.110988    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-cache\") pod \"f52f1f13-71c0-456e-911b-da69e4994a7e\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") "
Apr 28 19:52:32.111156 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.111038    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-tmp\") pod \"f52f1f13-71c0-456e-911b-da69e4994a7e\" (UID: \"f52f1f13-71c0-456e-911b-da69e4994a7e\") "
Apr 28 19:52:32.111231 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.111195    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "f52f1f13-71c0-456e-911b-da69e4994a7e" (UID: "f52f1f13-71c0-456e-911b-da69e4994a7e"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:52:32.111361 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.111338    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "f52f1f13-71c0-456e-911b-da69e4994a7e" (UID: "f52f1f13-71c0-456e-911b-da69e4994a7e"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:52:32.111444 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.111423    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-uds\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:52:32.111490 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.111452    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "f52f1f13-71c0-456e-911b-da69e4994a7e" (UID: "f52f1f13-71c0-456e-911b-da69e4994a7e"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:52:32.111704 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.111687    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "f52f1f13-71c0-456e-911b-da69e4994a7e" (UID: "f52f1f13-71c0-456e-911b-da69e4994a7e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 19:52:32.112779 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.112761    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52f1f13-71c0-456e-911b-da69e4994a7e-kube-api-access-2qskz" (OuterVolumeSpecName: "kube-api-access-2qskz") pod "f52f1f13-71c0-456e-911b-da69e4994a7e" (UID: "f52f1f13-71c0-456e-911b-da69e4994a7e"). InnerVolumeSpecName "kube-api-access-2qskz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 19:52:32.112847 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.112832    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f52f1f13-71c0-456e-911b-da69e4994a7e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f52f1f13-71c0-456e-911b-da69e4994a7e" (UID: "f52f1f13-71c0-456e-911b-da69e4994a7e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 19:52:32.212599 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.212575    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f52f1f13-71c0-456e-911b-da69e4994a7e-tls-certs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:52:32.212599 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.212596    2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-kserve-provision-location\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:52:32.212749 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.212620    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-cache\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:52:32.212749 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.212630    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f52f1f13-71c0-456e-911b-da69e4994a7e-tokenizer-tmp\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:52:32.212749 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.212639    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qskz\" (UniqueName: \"kubernetes.io/projected/f52f1f13-71c0-456e-911b-da69e4994a7e-kube-api-access-2qskz\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 19:52:32.296644 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.296612    2569 generic.go:358] "Generic (PLEG): container finished" podID="f52f1f13-71c0-456e-911b-da69e4994a7e" containerID="e7e19b955e25c287f0971b714a9b4d382e74df71fcedc300f81a724bedf8257f" exitCode=0
Apr 28 19:52:32.296789 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.296686    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"
Apr 28 19:52:32.296789 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.296690    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s" event={"ID":"f52f1f13-71c0-456e-911b-da69e4994a7e","Type":"ContainerDied","Data":"e7e19b955e25c287f0971b714a9b4d382e74df71fcedc300f81a724bedf8257f"}
Apr 28 19:52:32.296789 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.296727    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s" event={"ID":"f52f1f13-71c0-456e-911b-da69e4994a7e","Type":"ContainerDied","Data":"326051c4c7c5959f5e92f1220fd67090a5d5b69afe75562732dd8b2db405950e"}
Apr 28 19:52:32.296789 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.296746    2569 scope.go:117] "RemoveContainer" containerID="e7e19b955e25c287f0971b714a9b4d382e74df71fcedc300f81a724bedf8257f"
Apr 28 19:52:32.306105 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.306089    2569 scope.go:117] "RemoveContainer" containerID="716488e9bffa775c7df907bf4c5f0b3ba3bcc04f6c33e28f97c44718123ed277"
Apr 28 19:52:32.314270 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.314250    2569 scope.go:117] "RemoveContainer" containerID="55454a1e838e88dc2ebf5d7d75a67f4e752db04a6c793c7dfba04f476278042a"
Apr 28 19:52:32.323577 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.321228    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"]
Apr 28 19:52:32.326385 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.326362    2569 scope.go:117] "RemoveContainer" containerID="e7e19b955e25c287f0971b714a9b4d382e74df71fcedc300f81a724bedf8257f"
Apr 28 19:52:32.326830 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:52:32.326809    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7e19b955e25c287f0971b714a9b4d382e74df71fcedc300f81a724bedf8257f\": container with ID starting with e7e19b955e25c287f0971b714a9b4d382e74df71fcedc300f81a724bedf8257f not found: ID does not exist" containerID="e7e19b955e25c287f0971b714a9b4d382e74df71fcedc300f81a724bedf8257f"
Apr 28 19:52:32.326897 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.326836    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7e19b955e25c287f0971b714a9b4d382e74df71fcedc300f81a724bedf8257f"} err="failed to get container status \"e7e19b955e25c287f0971b714a9b4d382e74df71fcedc300f81a724bedf8257f\": rpc error: code = NotFound desc = could not find container \"e7e19b955e25c287f0971b714a9b4d382e74df71fcedc300f81a724bedf8257f\": container with ID starting with e7e19b955e25c287f0971b714a9b4d382e74df71fcedc300f81a724bedf8257f not found: ID does not exist"
Apr 28 19:52:32.326897 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.326855    2569 scope.go:117] "RemoveContainer" containerID="716488e9bffa775c7df907bf4c5f0b3ba3bcc04f6c33e28f97c44718123ed277"
Apr 28 19:52:32.327101 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:52:32.327082    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"716488e9bffa775c7df907bf4c5f0b3ba3bcc04f6c33e28f97c44718123ed277\": container with ID starting with 716488e9bffa775c7df907bf4c5f0b3ba3bcc04f6c33e28f97c44718123ed277 not found: ID does not exist" containerID="716488e9bffa775c7df907bf4c5f0b3ba3bcc04f6c33e28f97c44718123ed277"
Apr 28 19:52:32.327179 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.327107    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"716488e9bffa775c7df907bf4c5f0b3ba3bcc04f6c33e28f97c44718123ed277"} err="failed to get container status \"716488e9bffa775c7df907bf4c5f0b3ba3bcc04f6c33e28f97c44718123ed277\": rpc error: code = NotFound desc = could not find container \"716488e9bffa775c7df907bf4c5f0b3ba3bcc04f6c33e28f97c44718123ed277\": container with ID starting with 716488e9bffa775c7df907bf4c5f0b3ba3bcc04f6c33e28f97c44718123ed277 not found: ID does not exist"
Apr 28 19:52:32.327179 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.327126    2569 scope.go:117] "RemoveContainer" containerID="55454a1e838e88dc2ebf5d7d75a67f4e752db04a6c793c7dfba04f476278042a"
Apr 28 19:52:32.327451 ip-10-0-131-5 kubenswrapper[2569]: E0428 19:52:32.327395    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55454a1e838e88dc2ebf5d7d75a67f4e752db04a6c793c7dfba04f476278042a\": container with ID starting with 55454a1e838e88dc2ebf5d7d75a67f4e752db04a6c793c7dfba04f476278042a not found: ID does not exist" containerID="55454a1e838e88dc2ebf5d7d75a67f4e752db04a6c793c7dfba04f476278042a"
Apr 28 19:52:32.327508 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.327460    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55454a1e838e88dc2ebf5d7d75a67f4e752db04a6c793c7dfba04f476278042a"} err="failed to get container status \"55454a1e838e88dc2ebf5d7d75a67f4e752db04a6c793c7dfba04f476278042a\": rpc error: code = NotFound desc = could not find container \"55454a1e838e88dc2ebf5d7d75a67f4e752db04a6c793c7dfba04f476278042a\": container with ID starting with 55454a1e838e88dc2ebf5d7d75a67f4e752db04a6c793c7dfba04f476278042a not found: ID does not exist"
Apr 28 19:52:32.329499 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:32.329479    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7448f76699-j456s"]
Apr 28 19:52:33.943689 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:33.943655    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f52f1f13-71c0-456e-911b-da69e4994a7e" path="/var/lib/kubelet/pods/f52f1f13-71c0-456e-911b-da69e4994a7e/volumes"
Apr 28 19:52:50.002131 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002063    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"]
Apr 28 19:52:50.002523 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002460    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="429e685c-9bc2-4d2f-a571-a0b812a1e264" containerName="storage-initializer"
Apr 28 19:52:50.002523 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002473    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="429e685c-9bc2-4d2f-a571-a0b812a1e264" containerName="storage-initializer"
Apr 28 19:52:50.002523 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002482    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f52f1f13-71c0-456e-911b-da69e4994a7e" containerName="main"
Apr 28 19:52:50.002523 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002488    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f52f1f13-71c0-456e-911b-da69e4994a7e" containerName="main"
Apr 28 19:52:50.002523 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002497    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f52f1f13-71c0-456e-911b-da69e4994a7e" containerName="storage-initializer"
Apr 28 19:52:50.002523 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002504    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f52f1f13-71c0-456e-911b-da69e4994a7e" containerName="storage-initializer"
Apr 28 19:52:50.002523 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002515    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f52f1f13-71c0-456e-911b-da69e4994a7e" containerName="tokenizer"
Apr 28 19:52:50.002523 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002521    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f52f1f13-71c0-456e-911b-da69e4994a7e" containerName="tokenizer"
Apr 28 19:52:50.002785 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002533    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="429e685c-9bc2-4d2f-a571-a0b812a1e264" containerName="main"
Apr 28 19:52:50.002785 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002539    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="429e685c-9bc2-4d2f-a571-a0b812a1e264" containerName="main"
Apr 28 19:52:50.002785 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002547    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="429e685c-9bc2-4d2f-a571-a0b812a1e264" containerName="tokenizer"
Apr 28 19:52:50.002785 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002552    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="429e685c-9bc2-4d2f-a571-a0b812a1e264" containerName="tokenizer"
Apr 28 19:52:50.002785 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002609    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="f52f1f13-71c0-456e-911b-da69e4994a7e" containerName="tokenizer"
Apr 28 19:52:50.002785 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002617    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="429e685c-9bc2-4d2f-a571-a0b812a1e264" containerName="main"
Apr 28 19:52:50.002785 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002623    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="429e685c-9bc2-4d2f-a571-a0b812a1e264" containerName="tokenizer"
Apr 28 19:52:50.002785 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.002630    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="f52f1f13-71c0-456e-911b-da69e4994a7e" containerName="main"
Apr 28 19:52:50.007314 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.007294    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.009882 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.009857    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 28 19:52:50.009996 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.009898    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv8f1a6f044e8c7a4d31a250e0c4861caf-kserve-self-signed-certs\""
Apr 28 19:52:50.011146 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.011127    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-05aa9bba-epp-sa-dockercfg-fnr6p\""
Apr 28 19:52:50.011252 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.011176    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 28 19:52:50.011347 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.011258    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vz4n6\""
Apr 28 19:52:50.016118 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.016095    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"]
Apr 28 19:52:50.075809 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.075774    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-cache\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.075966 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.075830    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-kserve-provision-location\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.075966 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.075878    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzvxr\" (UniqueName: \"kubernetes.io/projected/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-kube-api-access-fzvxr\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.076045 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.075963    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-uds\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.076045 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.075997    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-tmp\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.076045 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.076036    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tls-certs\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.176592 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.176556    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-kserve-provision-location\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.176592 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.176597    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fzvxr\" (UniqueName: \"kubernetes.io/projected/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-kube-api-access-fzvxr\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.176843 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.176642    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-uds\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.176843 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.176664    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-tmp\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.176843 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.176692    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tls-certs\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.176843 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.176715    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-cache\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.177039 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.176990    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-kserve-provision-location\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.177077 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.177032    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-tmp\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.177077 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.177045    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-cache\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.177142 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.177089    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-uds\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.179041 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.179017    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tls-certs\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.184356 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.184335    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzvxr\" (UniqueName: \"kubernetes.io/projected/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-kube-api-access-fzvxr\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.321224 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.320792    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:52:50.462778 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:50.462748    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"]
Apr 28 19:52:50.464155 ip-10-0-131-5 kubenswrapper[2569]: W0428 19:52:50.464125    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ec8b7b_a0c3_44c1_83b2_c7b5890933cc.slice/crio-0287fe53c768c56be7139a9a3254cea95b07c8fa5e3514ef50414ab2b32b6f5b WatchSource:0}: Error finding container 0287fe53c768c56be7139a9a3254cea95b07c8fa5e3514ef50414ab2b32b6f5b: Status 404 returned error can't find the container with id 0287fe53c768c56be7139a9a3254cea95b07c8fa5e3514ef50414ab2b32b6f5b
Apr 28 19:52:51.381108 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:51.381071    2569 generic.go:358] "Generic (PLEG): container finished" podID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" containerID="05363f22a82517e072df1228f3d4e099ea0aca0046254c040e45dd488badd924" exitCode=0
Apr 28 19:52:51.381560 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:51.381162    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf" event={"ID":"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc","Type":"ContainerDied","Data":"05363f22a82517e072df1228f3d4e099ea0aca0046254c040e45dd488badd924"}
Apr 28 19:52:51.381560 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:51.381202    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf" event={"ID":"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc","Type":"ContainerStarted","Data":"0287fe53c768c56be7139a9a3254cea95b07c8fa5e3514ef50414ab2b32b6f5b"}
Apr 28 19:52:52.387927 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:52.387894    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf" event={"ID":"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc","Type":"ContainerStarted","Data":"76eb4e57383ce5cad47c21aecce8e785cceb3072d7d405a4ce0386da81715126"}
Apr 28 19:52:52.387927 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:52.387930    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf" event={"ID":"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc","Type":"ContainerStarted","Data":"f989420f88442b1cf45a295dac70e7033d3c008032b1d6892274b892ccf22401"}
Apr 28 19:52:52.388346 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:52:52.388039    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:53:00.321243 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:53:00.321208    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:53:00.321689 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:53:00.321259    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:53:00.324360 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:53:00.324328    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:53:00.352294 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:53:00.352235    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf" podStartSLOduration=11.352220771 podStartE2EDuration="11.352220771s" podCreationTimestamp="2026-04-28 19:52:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 19:52:52.416307188 +0000 UTC m=+2159.240647298" watchObservedRunningTime="2026-04-28 19:53:00.352220771 +0000 UTC m=+2167.176560916"
Apr 28 19:53:00.422656 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:53:00.422627    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:53:21.426823 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:53:21.426791    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 19:56:54.068715 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:56:54.068691    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:56:54.071895 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:56:54.071870    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 19:56:54.086678 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:56:54.086657    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 19:56:54.089631 ip-10-0-131-5 kubenswrapper[2569]: I0428 19:56:54.089610    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:01:54.109638 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:01:54.109562    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:01:54.112662 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:01:54.112640    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:01:54.122010 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:01:54.121990    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:01:54.124902 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:01:54.124877    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:06:54.142724 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:06:54.142697    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:06:54.145577 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:06:54.145555    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:06:54.156479 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:06:54.156455    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:06:54.159310 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:06:54.159289    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:07:33.889115 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:33.889080    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"]
Apr 28 20:07:33.889739 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:33.889483    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf" podUID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" containerName="main" containerID="cri-o://f989420f88442b1cf45a295dac70e7033d3c008032b1d6892274b892ccf22401" gracePeriod=30
Apr 28 20:07:33.889739 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:33.889515    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf" podUID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" containerName="tokenizer" containerID="cri-o://76eb4e57383ce5cad47c21aecce8e785cceb3072d7d405a4ce0386da81715126" gracePeriod=30
Apr 28 20:07:34.922512 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:34.922484    2569 generic.go:358] "Generic (PLEG): container finished" podID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" containerID="76eb4e57383ce5cad47c21aecce8e785cceb3072d7d405a4ce0386da81715126" exitCode=0
Apr 28 20:07:34.922512 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:34.922508    2569 generic.go:358] "Generic (PLEG): container finished" podID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" containerID="f989420f88442b1cf45a295dac70e7033d3c008032b1d6892274b892ccf22401" exitCode=0
Apr 28 20:07:34.923011 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:34.922529    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf" event={"ID":"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc","Type":"ContainerDied","Data":"76eb4e57383ce5cad47c21aecce8e785cceb3072d7d405a4ce0386da81715126"}
Apr 28 20:07:34.923011 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:34.922564    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf" event={"ID":"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc","Type":"ContainerDied","Data":"f989420f88442b1cf45a295dac70e7033d3c008032b1d6892274b892ccf22401"}
Apr 28 20:07:35.049906 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.049884    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 20:07:35.135805 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.135739    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzvxr\" (UniqueName: \"kubernetes.io/projected/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-kube-api-access-fzvxr\") pod \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") "
Apr 28 20:07:35.135805 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.135791    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-tmp\") pod \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") "
Apr 28 20:07:35.136002 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.135826    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-kserve-provision-location\") pod \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") "
Apr 28 20:07:35.136002 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.135887    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-cache\") pod \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") "
Apr 28 20:07:35.136002 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.135909    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tls-certs\") pod \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") "
Apr 28 20:07:35.136002 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.135931    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-uds\") pod \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\" (UID: \"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc\") "
Apr 28 20:07:35.136219 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.136109    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" (UID: "05ec8b7b-a0c3-44c1-83b2-c7b5890933cc"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:07:35.136219 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.136186    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" (UID: "05ec8b7b-a0c3-44c1-83b2-c7b5890933cc"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:07:35.136335 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.136274    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" (UID: "05ec8b7b-a0c3-44c1-83b2-c7b5890933cc"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:07:35.136666 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.136641    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" (UID: "05ec8b7b-a0c3-44c1-83b2-c7b5890933cc"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:07:35.137813 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.137797    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-kube-api-access-fzvxr" (OuterVolumeSpecName: "kube-api-access-fzvxr") pod "05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" (UID: "05ec8b7b-a0c3-44c1-83b2-c7b5890933cc"). InnerVolumeSpecName "kube-api-access-fzvxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 20:07:35.138009 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.137990    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" (UID: "05ec8b7b-a0c3-44c1-83b2-c7b5890933cc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 20:07:35.236517 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.236492    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-tmp\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:07:35.236517 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.236516    2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-kserve-provision-location\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:07:35.236649 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.236527    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-cache\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:07:35.236649 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.236536    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tls-certs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:07:35.236649 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.236544    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-tokenizer-uds\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:07:35.236649 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.236552    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzvxr\" (UniqueName: \"kubernetes.io/projected/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc-kube-api-access-fzvxr\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:07:35.928708 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.928675    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf" event={"ID":"05ec8b7b-a0c3-44c1-83b2-c7b5890933cc","Type":"ContainerDied","Data":"0287fe53c768c56be7139a9a3254cea95b07c8fa5e3514ef50414ab2b32b6f5b"}
Apr 28 20:07:35.928708 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.928720    2569 scope.go:117] "RemoveContainer" containerID="76eb4e57383ce5cad47c21aecce8e785cceb3072d7d405a4ce0386da81715126"
Apr 28 20:07:35.929202 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.928727    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"
Apr 28 20:07:35.938152 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.938127    2569 scope.go:117] "RemoveContainer" containerID="f989420f88442b1cf45a295dac70e7033d3c008032b1d6892274b892ccf22401"
Apr 28 20:07:35.946229 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.946212    2569 scope.go:117] "RemoveContainer" containerID="05363f22a82517e072df1228f3d4e099ea0aca0046254c040e45dd488badd924"
Apr 28 20:07:35.953526 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.953504    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"]
Apr 28 20:07:35.958614 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:35.958595    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-schez6gxf"]
Apr 28 20:07:37.943080 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:37.943051    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" path="/var/lib/kubelet/pods/05ec8b7b-a0c3-44c1-83b2-c7b5890933cc/volumes"
Apr 28 20:07:49.397631 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.397589    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"]
Apr 28 20:07:49.398170 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.398147    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" containerName="tokenizer"
Apr 28 20:07:49.398252 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.398173    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" containerName="tokenizer"
Apr 28 20:07:49.398252 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.398215    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" containerName="main"
Apr 28 20:07:49.398252 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.398224    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" containerName="main"
Apr 28 20:07:49.398252 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.398236    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" containerName="storage-initializer"
Apr 28 20:07:49.398252 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.398247    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" containerName="storage-initializer"
Apr 28 20:07:49.398551 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.398341    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" containerName="tokenizer"
Apr 28 20:07:49.398551 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.398357    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="05ec8b7b-a0c3-44c1-83b2-c7b5890933cc" containerName="main"
Apr 28 20:07:49.402179 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.402141    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.405835 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.405807    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 28 20:07:49.405971 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.405847    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Apr 28 20:07:49.405971 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.405895    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vz4n6\""
Apr 28 20:07:49.406135 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.406099    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 28 20:07:49.406419 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.406293    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-epp-sa-dockercfg-ft75k\""
Apr 28 20:07:49.414057 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.414035    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"]
Apr 28 20:07:49.461256 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.461222    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-cache\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.461447 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.461278    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.461447 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.461365    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b7624e-a3eb-4343-a733-3a01c6902e4f-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.461447 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.461395    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bw6p\" (UniqueName: \"kubernetes.io/projected/04b7624e-a3eb-4343-a733-3a01c6902e4f-kube-api-access-7bw6p\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.461594 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.461510    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-tmp\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.461594 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.461542    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.562666 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.562566    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.562666 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.562645    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-cache\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.562927 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.562701    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.562927 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.562759    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b7624e-a3eb-4343-a733-3a01c6902e4f-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.562927 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.562788    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7bw6p\" (UniqueName: \"kubernetes.io/projected/04b7624e-a3eb-4343-a733-3a01c6902e4f-kube-api-access-7bw6p\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.562927 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.562853    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-tmp\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.563155 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.562985    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.563155 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.563002    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-cache\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.563155 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.563085    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.563319 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.563182    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-tmp\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.565453 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.565430    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b7624e-a3eb-4343-a733-3a01c6902e4f-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.573233 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.573213    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bw6p\" (UniqueName: \"kubernetes.io/projected/04b7624e-a3eb-4343-a733-3a01c6902e4f-kube-api-access-7bw6p\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.715193 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.715161    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:49.847287 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.847257    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"]
Apr 28 20:07:49.849026 ip-10-0-131-5 kubenswrapper[2569]: W0428 20:07:49.848985    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b7624e_a3eb_4343_a733_3a01c6902e4f.slice/crio-bdf4dc5bd47fb16cc659815a02396ffdebc2a73c4594a77663a8d9d2e569fdaf WatchSource:0}: Error finding container bdf4dc5bd47fb16cc659815a02396ffdebc2a73c4594a77663a8d9d2e569fdaf: Status 404 returned error can't find the container with id bdf4dc5bd47fb16cc659815a02396ffdebc2a73c4594a77663a8d9d2e569fdaf
Apr 28 20:07:49.851055 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.851033    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 28 20:07:49.981940 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.981851    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz" event={"ID":"04b7624e-a3eb-4343-a733-3a01c6902e4f","Type":"ContainerStarted","Data":"711dea03fd74407dfbc02c35b5798379f540bc8c3595b839cc2a60213b19a5dc"}
Apr 28 20:07:49.981940 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:49.981886    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz" event={"ID":"04b7624e-a3eb-4343-a733-3a01c6902e4f","Type":"ContainerStarted","Data":"bdf4dc5bd47fb16cc659815a02396ffdebc2a73c4594a77663a8d9d2e569fdaf"}
Apr 28 20:07:51.989672 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:51.989641    2569 generic.go:358] "Generic (PLEG): container finished" podID="04b7624e-a3eb-4343-a733-3a01c6902e4f" containerID="711dea03fd74407dfbc02c35b5798379f540bc8c3595b839cc2a60213b19a5dc" exitCode=0
Apr 28 20:07:51.990052 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:51.989715    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz" event={"ID":"04b7624e-a3eb-4343-a733-3a01c6902e4f","Type":"ContainerDied","Data":"711dea03fd74407dfbc02c35b5798379f540bc8c3595b839cc2a60213b19a5dc"}
Apr 28 20:07:52.996146 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:52.996111    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz" event={"ID":"04b7624e-a3eb-4343-a733-3a01c6902e4f","Type":"ContainerStarted","Data":"2c3cc1a55b57729430c89e53e6e7eae0e97f51afe55abd9f8b06f367dd7d2ebe"}
Apr 28 20:07:52.996146 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:52.996152    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz" event={"ID":"04b7624e-a3eb-4343-a733-3a01c6902e4f","Type":"ContainerStarted","Data":"816da058268a0d5c8b8b00e568dc85ccca3e9b69ba1227b1f87ddbcc585fe7b6"}
Apr 28 20:07:52.996709 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:52.996333    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:53.021161 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:53.021100    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz" podStartSLOduration=4.021086995 podStartE2EDuration="4.021086995s" podCreationTimestamp="2026-04-28 20:07:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 20:07:53.019379811 +0000 UTC m=+3059.843719920" watchObservedRunningTime="2026-04-28 20:07:53.021086995 +0000 UTC m=+3059.845427102"
Apr 28 20:07:59.715595 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:59.715561    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:59.715991 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:59.715612    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:07:59.718284 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:07:59.718257    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:08:00.025981 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:08:00.025890    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:08:22.033877 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:08:22.033845    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:11:54.175078 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:11:54.175045    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:11:54.177807 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:11:54.177777    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:11:54.190707 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:11:54.190677    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:11:54.193816 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:11:54.193795    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:16:54.207393 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:16:54.207363    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:16:54.210739 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:16:54.210712    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:16:54.235078 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:16:54.235050    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:16:54.238192 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:16:54.238172    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:21:54.239782 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:21:54.239687    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:21:54.243665 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:21:54.242606    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:21:54.273188 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:21:54.273163    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:21:54.276057 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:21:54.276037    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:22:37.076346 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.076267    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"]
Apr 28 20:22:37.076976 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.076815    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz" podUID="04b7624e-a3eb-4343-a733-3a01c6902e4f" containerName="main" containerID="cri-o://816da058268a0d5c8b8b00e568dc85ccca3e9b69ba1227b1f87ddbcc585fe7b6" gracePeriod=30
Apr 28 20:22:37.076976 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.076854    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz" podUID="04b7624e-a3eb-4343-a733-3a01c6902e4f" containerName="tokenizer" containerID="cri-o://2c3cc1a55b57729430c89e53e6e7eae0e97f51afe55abd9f8b06f367dd7d2ebe" gracePeriod=30
Apr 28 20:22:37.161090 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.161055    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"]
Apr 28 20:22:37.165213 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.165165    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.168163 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.168140    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-2-openshift-default-dockercfg-pj4d4\""
Apr 28 20:22:37.168595 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.168369    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"istio-ca-root-cert\""
Apr 28 20:22:37.187826 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.187796    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"]
Apr 28 20:22:37.275048 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.275018    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fddcd380-e919-4cca-9442-953eee906665-istio-podinfo\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.275048 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.275053    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fddcd380-e919-4cca-9442-953eee906665-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.275267 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.275127    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fddcd380-e919-4cca-9442-953eee906665-istio-token\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.275267 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.275144    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khbcg\" (UniqueName: \"kubernetes.io/projected/fddcd380-e919-4cca-9442-953eee906665-kube-api-access-khbcg\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.275267 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.275238    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-credential-socket\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.275420 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.275278    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-workload-socket\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.275420 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.275333    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-istio-envoy\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.275420 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.275376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-istio-data\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.275580 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.275442    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-workload-certs\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.376139 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.376050    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-istio-envoy\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.376139 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.376094    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-istio-data\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.376139 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.376121    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-workload-certs\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.376450 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.376161    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fddcd380-e919-4cca-9442-953eee906665-istio-podinfo\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.376450 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.376190    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fddcd380-e919-4cca-9442-953eee906665-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.376450 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.376255    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fddcd380-e919-4cca-9442-953eee906665-istio-token\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.376450 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.376280    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-khbcg\" (UniqueName: \"kubernetes.io/projected/fddcd380-e919-4cca-9442-953eee906665-kube-api-access-khbcg\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.376450 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.376338    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-credential-socket\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.376450 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.376379    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-workload-socket\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.376764 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.376660    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-istio-data\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.376764 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.376699    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-workload-certs\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.377169 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.377137    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-workload-socket\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.377333 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.377310    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-credential-socket\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.377556 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.377533    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fddcd380-e919-4cca-9442-953eee906665-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.378866 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.378845    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fddcd380-e919-4cca-9442-953eee906665-istio-envoy\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.379205 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.379184    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fddcd380-e919-4cca-9442-953eee906665-istio-podinfo\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.384569 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.384548    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fddcd380-e919-4cca-9442-953eee906665-istio-token\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.384708 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.384686    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khbcg\" (UniqueName: \"kubernetes.io/projected/fddcd380-e919-4cca-9442-953eee906665-kube-api-access-khbcg\") pod \"router-gateway-2-openshift-default-6866b85949-scfv4\" (UID: \"fddcd380-e919-4cca-9442-953eee906665\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.477355 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.477316    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:37.529743 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.529704    2569 generic.go:358] "Generic (PLEG): container finished" podID="04b7624e-a3eb-4343-a733-3a01c6902e4f" containerID="816da058268a0d5c8b8b00e568dc85ccca3e9b69ba1227b1f87ddbcc585fe7b6" exitCode=0
Apr 28 20:22:37.529923 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.529777    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz" event={"ID":"04b7624e-a3eb-4343-a733-3a01c6902e4f","Type":"ContainerDied","Data":"816da058268a0d5c8b8b00e568dc85ccca3e9b69ba1227b1f87ddbcc585fe7b6"}
Apr 28 20:22:37.830278 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.830222    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"]
Apr 28 20:22:37.831764 ip-10-0-131-5 kubenswrapper[2569]: W0428 20:22:37.831726    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfddcd380_e919_4cca_9442_953eee906665.slice/crio-dce1042e360719c96cab06d5963e3f967e8ddc9859a7c82c2f607405475a0972 WatchSource:0}: Error finding container dce1042e360719c96cab06d5963e3f967e8ddc9859a7c82c2f607405475a0972: Status 404 returned error can't find the container with id dce1042e360719c96cab06d5963e3f967e8ddc9859a7c82c2f607405475a0972
Apr 28 20:22:37.834135 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.834112    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 28 20:22:37.834617 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.834582    2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Apr 28 20:22:37.834727 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.834660    2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Apr 28 20:22:37.834727 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:37.834696    2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Apr 28 20:22:38.353048 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.353024    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:22:38.387369 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.387340    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b7624e-a3eb-4343-a733-3a01c6902e4f-tls-certs\") pod \"04b7624e-a3eb-4343-a733-3a01c6902e4f\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") "
Apr 28 20:22:38.387560 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.387377    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-tmp\") pod \"04b7624e-a3eb-4343-a733-3a01c6902e4f\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") "
Apr 28 20:22:38.387560 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.387508    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-cache\") pod \"04b7624e-a3eb-4343-a733-3a01c6902e4f\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") "
Apr 28 20:22:38.387694 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.387571    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-uds\") pod \"04b7624e-a3eb-4343-a733-3a01c6902e4f\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") "
Apr 28 20:22:38.387694 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.387672    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bw6p\" (UniqueName: \"kubernetes.io/projected/04b7624e-a3eb-4343-a733-3a01c6902e4f-kube-api-access-7bw6p\") pod \"04b7624e-a3eb-4343-a733-3a01c6902e4f\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") "
Apr 28 20:22:38.387805 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.387696    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "04b7624e-a3eb-4343-a733-3a01c6902e4f" (UID: "04b7624e-a3eb-4343-a733-3a01c6902e4f"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:22:38.387805 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.387704    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-kserve-provision-location\") pod \"04b7624e-a3eb-4343-a733-3a01c6902e4f\" (UID: \"04b7624e-a3eb-4343-a733-3a01c6902e4f\") "
Apr 28 20:22:38.387917 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.387855    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "04b7624e-a3eb-4343-a733-3a01c6902e4f" (UID: "04b7624e-a3eb-4343-a733-3a01c6902e4f"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:22:38.388015 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.387996    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "04b7624e-a3eb-4343-a733-3a01c6902e4f" (UID: "04b7624e-a3eb-4343-a733-3a01c6902e4f"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:22:38.388098 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.388080    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-tmp\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:22:38.388160 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.388105    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-cache\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:22:38.388160 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.388115    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-tokenizer-uds\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:22:38.388500 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.388472    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "04b7624e-a3eb-4343-a733-3a01c6902e4f" (UID: "04b7624e-a3eb-4343-a733-3a01c6902e4f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:22:38.389660 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.389636    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04b7624e-a3eb-4343-a733-3a01c6902e4f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "04b7624e-a3eb-4343-a733-3a01c6902e4f" (UID: "04b7624e-a3eb-4343-a733-3a01c6902e4f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 20:22:38.389996 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.389975    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b7624e-a3eb-4343-a733-3a01c6902e4f-kube-api-access-7bw6p" (OuterVolumeSpecName: "kube-api-access-7bw6p") pod "04b7624e-a3eb-4343-a733-3a01c6902e4f" (UID: "04b7624e-a3eb-4343-a733-3a01c6902e4f"). InnerVolumeSpecName "kube-api-access-7bw6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 20:22:38.489249 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.489212    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b7624e-a3eb-4343-a733-3a01c6902e4f-tls-certs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:22:38.489249 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.489250    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bw6p\" (UniqueName: \"kubernetes.io/projected/04b7624e-a3eb-4343-a733-3a01c6902e4f-kube-api-access-7bw6p\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:22:38.489491 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.489266    2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b7624e-a3eb-4343-a733-3a01c6902e4f-kserve-provision-location\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:22:38.535366 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.535340    2569 generic.go:358] "Generic (PLEG): container finished" podID="04b7624e-a3eb-4343-a733-3a01c6902e4f" containerID="2c3cc1a55b57729430c89e53e6e7eae0e97f51afe55abd9f8b06f367dd7d2ebe" exitCode=0
Apr 28 20:22:38.535527 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.535439    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"
Apr 28 20:22:38.535527 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.535430    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz" event={"ID":"04b7624e-a3eb-4343-a733-3a01c6902e4f","Type":"ContainerDied","Data":"2c3cc1a55b57729430c89e53e6e7eae0e97f51afe55abd9f8b06f367dd7d2ebe"}
Apr 28 20:22:38.535642 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.535542    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz" event={"ID":"04b7624e-a3eb-4343-a733-3a01c6902e4f","Type":"ContainerDied","Data":"bdf4dc5bd47fb16cc659815a02396ffdebc2a73c4594a77663a8d9d2e569fdaf"}
Apr 28 20:22:38.535642 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.535565    2569 scope.go:117] "RemoveContainer" containerID="2c3cc1a55b57729430c89e53e6e7eae0e97f51afe55abd9f8b06f367dd7d2ebe"
Apr 28 20:22:38.537035 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.537004    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4" event={"ID":"fddcd380-e919-4cca-9442-953eee906665","Type":"ContainerStarted","Data":"0e0d2a213a0ee4824ea4b53228bfad59e26b5c7f3872ba9a02b6715f7ea5749d"}
Apr 28 20:22:38.537035 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.537034    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4" event={"ID":"fddcd380-e919-4cca-9442-953eee906665","Type":"ContainerStarted","Data":"dce1042e360719c96cab06d5963e3f967e8ddc9859a7c82c2f607405475a0972"}
Apr 28 20:22:38.544994 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.544978    2569 scope.go:117] "RemoveContainer" containerID="816da058268a0d5c8b8b00e568dc85ccca3e9b69ba1227b1f87ddbcc585fe7b6"
Apr 28 20:22:38.553978 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.553959    2569 scope.go:117] "RemoveContainer" containerID="711dea03fd74407dfbc02c35b5798379f540bc8c3595b839cc2a60213b19a5dc"
Apr 28 20:22:38.562380 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.562334    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4" podStartSLOduration=1.5623190500000002 podStartE2EDuration="1.56231905s" podCreationTimestamp="2026-04-28 20:22:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 20:22:38.557281155 +0000 UTC m=+3945.381621284" watchObservedRunningTime="2026-04-28 20:22:38.56231905 +0000 UTC m=+3945.386659158"
Apr 28 20:22:38.564157 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.564140    2569 scope.go:117] "RemoveContainer" containerID="2c3cc1a55b57729430c89e53e6e7eae0e97f51afe55abd9f8b06f367dd7d2ebe"
Apr 28 20:22:38.564391 ip-10-0-131-5 kubenswrapper[2569]: E0428 20:22:38.564372    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c3cc1a55b57729430c89e53e6e7eae0e97f51afe55abd9f8b06f367dd7d2ebe\": container with ID starting with 2c3cc1a55b57729430c89e53e6e7eae0e97f51afe55abd9f8b06f367dd7d2ebe not found: ID does not exist" containerID="2c3cc1a55b57729430c89e53e6e7eae0e97f51afe55abd9f8b06f367dd7d2ebe"
Apr 28 20:22:38.564596 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.564415    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c3cc1a55b57729430c89e53e6e7eae0e97f51afe55abd9f8b06f367dd7d2ebe"} err="failed to get container status \"2c3cc1a55b57729430c89e53e6e7eae0e97f51afe55abd9f8b06f367dd7d2ebe\": rpc error: code = NotFound desc = could not find container \"2c3cc1a55b57729430c89e53e6e7eae0e97f51afe55abd9f8b06f367dd7d2ebe\": container with ID starting with 2c3cc1a55b57729430c89e53e6e7eae0e97f51afe55abd9f8b06f367dd7d2ebe not found: ID does not exist"
Apr 28 20:22:38.564596 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.564435    2569 scope.go:117] "RemoveContainer" containerID="816da058268a0d5c8b8b00e568dc85ccca3e9b69ba1227b1f87ddbcc585fe7b6"
Apr 28 20:22:38.564720 ip-10-0-131-5 kubenswrapper[2569]: E0428 20:22:38.564648    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"816da058268a0d5c8b8b00e568dc85ccca3e9b69ba1227b1f87ddbcc585fe7b6\": container with ID starting with 816da058268a0d5c8b8b00e568dc85ccca3e9b69ba1227b1f87ddbcc585fe7b6 not found: ID does not exist" containerID="816da058268a0d5c8b8b00e568dc85ccca3e9b69ba1227b1f87ddbcc585fe7b6"
Apr 28 20:22:38.564720 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.564664    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"816da058268a0d5c8b8b00e568dc85ccca3e9b69ba1227b1f87ddbcc585fe7b6"} err="failed to get container status \"816da058268a0d5c8b8b00e568dc85ccca3e9b69ba1227b1f87ddbcc585fe7b6\": rpc error: code = NotFound desc = could not find container \"816da058268a0d5c8b8b00e568dc85ccca3e9b69ba1227b1f87ddbcc585fe7b6\": container with ID starting with 816da058268a0d5c8b8b00e568dc85ccca3e9b69ba1227b1f87ddbcc585fe7b6 not found: ID does not exist"
Apr 28 20:22:38.564720 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.564676    2569 scope.go:117] "RemoveContainer" containerID="711dea03fd74407dfbc02c35b5798379f540bc8c3595b839cc2a60213b19a5dc"
Apr 28 20:22:38.564887 ip-10-0-131-5 kubenswrapper[2569]: E0428 20:22:38.564873    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"711dea03fd74407dfbc02c35b5798379f540bc8c3595b839cc2a60213b19a5dc\": container with ID starting with 711dea03fd74407dfbc02c35b5798379f540bc8c3595b839cc2a60213b19a5dc not found: ID does not exist" containerID="711dea03fd74407dfbc02c35b5798379f540bc8c3595b839cc2a60213b19a5dc"
Apr 28 20:22:38.564942 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.564895    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711dea03fd74407dfbc02c35b5798379f540bc8c3595b839cc2a60213b19a5dc"} err="failed to get container status \"711dea03fd74407dfbc02c35b5798379f540bc8c3595b839cc2a60213b19a5dc\": rpc error: code = NotFound desc = could not find container \"711dea03fd74407dfbc02c35b5798379f540bc8c3595b839cc2a60213b19a5dc\": container with ID starting with 711dea03fd74407dfbc02c35b5798379f540bc8c3595b839cc2a60213b19a5dc not found: ID does not exist"
Apr 28 20:22:38.574328 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.574307    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"]
Apr 28 20:22:38.576649 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:38.576629    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-5b5cbpw2dz"]
Apr 28 20:22:39.477788 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:39.477749    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:39.946838 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:39.946801    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04b7624e-a3eb-4343-a733-3a01c6902e4f" path="/var/lib/kubelet/pods/04b7624e-a3eb-4343-a733-3a01c6902e4f/volumes"
Apr 28 20:22:40.518551 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:40.518510    2569 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4" podUID="fddcd380-e919-4cca-9442-953eee906665" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Apr 28 20:22:41.538656 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:41.538603    2569 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4" podUID="fddcd380-e919-4cca-9442-953eee906665" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Apr 28 20:22:41.596486 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:41.596456    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:41.596784 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:41.596764    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:41.598546 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:41.598526    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-scfv4"
Apr 28 20:22:48.249470 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.249429    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"]
Apr 28 20:22:48.249891 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.249874    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b7624e-a3eb-4343-a733-3a01c6902e4f" containerName="storage-initializer"
Apr 28 20:22:48.249891 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.249888    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b7624e-a3eb-4343-a733-3a01c6902e4f" containerName="storage-initializer"
Apr 28 20:22:48.249962 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.249900    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b7624e-a3eb-4343-a733-3a01c6902e4f" containerName="tokenizer"
Apr 28 20:22:48.249962 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.249905    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b7624e-a3eb-4343-a733-3a01c6902e4f" containerName="tokenizer"
Apr 28 20:22:48.249962 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.249913    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b7624e-a3eb-4343-a733-3a01c6902e4f" containerName="main"
Apr 28 20:22:48.249962 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.249918    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b7624e-a3eb-4343-a733-3a01c6902e4f" containerName="main"
Apr 28 20:22:48.250091 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.249981    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="04b7624e-a3eb-4343-a733-3a01c6902e4f" containerName="main"
Apr 28 20:22:48.250091 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.249990    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="04b7624e-a3eb-4343-a733-3a01c6902e4f" containerName="tokenizer"
Apr 28 20:22:48.254255 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.254237    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.256819 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.256782    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-epp-sa-dockercfg-2zjdm\""
Apr 28 20:22:48.256819 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.256800    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vz4n6\""
Apr 28 20:22:48.257004 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.256802    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Apr 28 20:22:48.261784 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.261748    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"]
Apr 28 20:22:48.384204 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.384165    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-cache\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.384204 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.384208    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-tmp\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.384455 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.384269    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5707b339-e33c-45c6-a7b7-6ce04b34149b-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.384455 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.384299    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.384455 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.384318    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.384455 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.384357    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zprtj\" (UniqueName: \"kubernetes.io/projected/5707b339-e33c-45c6-a7b7-6ce04b34149b-kube-api-access-zprtj\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.485201 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.485170    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.485409 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.485205    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.485409 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.485262    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zprtj\" (UniqueName: \"kubernetes.io/projected/5707b339-e33c-45c6-a7b7-6ce04b34149b-kube-api-access-zprtj\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.485409 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.485298    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-cache\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.485409 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.485325    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-tmp\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.485409 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.485386    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5707b339-e33c-45c6-a7b7-6ce04b34149b-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.485680 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.485608    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.485680 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.485661    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.485754 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.485691    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-cache\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.485754 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.485709    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-tmp\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.487946 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.487922    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5707b339-e33c-45c6-a7b7-6ce04b34149b-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.493133 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.493115    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zprtj\" (UniqueName: \"kubernetes.io/projected/5707b339-e33c-45c6-a7b7-6ce04b34149b-kube-api-access-zprtj\") pod \"router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.565483 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.565369    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:48.699497 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:48.699473    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"]
Apr 28 20:22:48.700856 ip-10-0-131-5 kubenswrapper[2569]: W0428 20:22:48.700827    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5707b339_e33c_45c6_a7b7_6ce04b34149b.slice/crio-7682671e641c270826316a3f9b301b94e487c67d34c98f776542d45d2ec90bef WatchSource:0}: Error finding container 7682671e641c270826316a3f9b301b94e487c67d34c98f776542d45d2ec90bef: Status 404 returned error can't find the container with id 7682671e641c270826316a3f9b301b94e487c67d34c98f776542d45d2ec90bef
Apr 28 20:22:49.587069 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:49.587034    2569 generic.go:358] "Generic (PLEG): container finished" podID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerID="7706d71bf39fc9212c33d0efedb19fb541490ef34dc5e721e08916f6e9bd0596" exitCode=0
Apr 28 20:22:49.587498 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:49.587081    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2" event={"ID":"5707b339-e33c-45c6-a7b7-6ce04b34149b","Type":"ContainerDied","Data":"7706d71bf39fc9212c33d0efedb19fb541490ef34dc5e721e08916f6e9bd0596"}
Apr 28 20:22:49.587498 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:49.587117    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2" event={"ID":"5707b339-e33c-45c6-a7b7-6ce04b34149b","Type":"ContainerStarted","Data":"7682671e641c270826316a3f9b301b94e487c67d34c98f776542d45d2ec90bef"}
Apr 28 20:22:50.593004 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:50.592966    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2" event={"ID":"5707b339-e33c-45c6-a7b7-6ce04b34149b","Type":"ContainerStarted","Data":"7ca9f79d854b2e298f161f576cd74793926cb44f5e7a7fb9abedf254615049f0"}
Apr 28 20:22:50.593004 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:50.593005    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2" event={"ID":"5707b339-e33c-45c6-a7b7-6ce04b34149b","Type":"ContainerStarted","Data":"27d344d10af4eb6e70fd9cb847b35b0d24a73de70a11deca62f1263bcaac17cd"}
Apr 28 20:22:50.593435 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:50.593087    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:50.614861 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:50.614816    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2" podStartSLOduration=2.6148035590000003 podStartE2EDuration="2.614803559s" podCreationTimestamp="2026-04-28 20:22:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 20:22:50.61194491 +0000 UTC m=+3957.436285017" watchObservedRunningTime="2026-04-28 20:22:50.614803559 +0000 UTC m=+3957.439143667"
Apr 28 20:22:58.566168 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:58.566134    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:58.566674 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:58.566278    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:58.569065 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:58.569045    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:22:58.625845 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:22:58.625819    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:23:20.633785 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:23:20.633751    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:24:54.420891 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.420854    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
Apr 28 20:24:54.425486 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.425461    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.428133 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.428107    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs\""
Apr 28 20:24:54.429240 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.429218    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-f312f5-cb7fb8cf-dockercfg-bvrgh\""
Apr 28 20:24:54.436723 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.436692    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
Apr 28 20:24:54.558568 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.558540    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.558694 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.558598    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.558694 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.558652    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.558694 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.558682    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8gl5\" (UniqueName: \"kubernetes.io/projected/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-kube-api-access-q8gl5\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.558814 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.558734    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.558814 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.558767    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.558814 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.558798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.660027 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.660000    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q8gl5\" (UniqueName: \"kubernetes.io/projected/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-kube-api-access-q8gl5\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.660171 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.660064    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.660171 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.660089    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.660171 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.660129    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.660171 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.660163    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.660361 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.660234    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.660361 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.660269    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.660507 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.660482    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.660658 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.660635    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.660735 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.660666    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.660735 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.660666    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.662531 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.662511    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.662668 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.662648    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.667271 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.667249    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8gl5\" (UniqueName: \"kubernetes.io/projected/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-kube-api-access-q8gl5\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.737220 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.737197    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:24:54.872179 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:54.872150    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
Apr 28 20:24:54.873545 ip-10-0-131-5 kubenswrapper[2569]: W0428 20:24:54.873506    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb67edfe7_5bba_4b86_b378_c71cb7a3b2dc.slice/crio-91e4b058b9524f3d8bffa03c290679c75e33cb613c2e0a942bf38cae89424fb8 WatchSource:0}: Error finding container 91e4b058b9524f3d8bffa03c290679c75e33cb613c2e0a942bf38cae89424fb8: Status 404 returned error can't find the container with id 91e4b058b9524f3d8bffa03c290679c75e33cb613c2e0a942bf38cae89424fb8
Apr 28 20:24:55.113368 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:55.113279    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc","Type":"ContainerStarted","Data":"6e52d0481f5e428346235428d323b02492758957876b48e7c75dd72893502e97"}
Apr 28 20:24:55.113368 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:24:55.113334    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc","Type":"ContainerStarted","Data":"91e4b058b9524f3d8bffa03c290679c75e33cb613c2e0a942bf38cae89424fb8"}
Apr 28 20:25:13.188782 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:25:13.188749    2569 generic.go:358] "Generic (PLEG): container finished" podID="b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" containerID="6e52d0481f5e428346235428d323b02492758957876b48e7c75dd72893502e97" exitCode=0
Apr 28 20:25:13.188782 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:25:13.188790    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc","Type":"ContainerDied","Data":"6e52d0481f5e428346235428d323b02492758957876b48e7c75dd72893502e97"}
Apr 28 20:26:01.406532 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:26:01.406493    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc","Type":"ContainerStarted","Data":"0a27c366fffae8b38fae603309c1d9dafc13caee4be2bdce665e90de97a05a57"}
Apr 28 20:26:01.427128 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:26:01.427063    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" podStartSLOduration=19.789185318 podStartE2EDuration="1m7.427043706s" podCreationTimestamp="2026-04-28 20:24:54 +0000 UTC" firstStartedPulling="2026-04-28 20:25:13.190084966 +0000 UTC m=+4100.014425056" lastFinishedPulling="2026-04-28 20:26:00.827943355 +0000 UTC m=+4147.652283444" observedRunningTime="2026-04-28 20:26:01.426563136 +0000 UTC m=+4148.250903276" watchObservedRunningTime="2026-04-28 20:26:01.427043706 +0000 UTC m=+4148.251383815"
Apr 28 20:26:54.275755 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:26:54.275648    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:26:54.279198 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:26:54.279176    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:26:54.310006 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:26:54.309987    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:26:54.313103 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:26:54.313086    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:27:09.491773 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:09.491736    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"]
Apr 28 20:27:09.492257 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:09.492089    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2" podUID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerName="main" containerID="cri-o://27d344d10af4eb6e70fd9cb847b35b0d24a73de70a11deca62f1263bcaac17cd" gracePeriod=30
Apr 28 20:27:09.492331 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:09.492271    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2" podUID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerName="tokenizer" containerID="cri-o://7ca9f79d854b2e298f161f576cd74793926cb44f5e7a7fb9abedf254615049f0" gracePeriod=30
Apr 28 20:27:09.699320 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:09.699285    2569 generic.go:358] "Generic (PLEG): container finished" podID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerID="27d344d10af4eb6e70fd9cb847b35b0d24a73de70a11deca62f1263bcaac17cd" exitCode=0
Apr 28 20:27:09.699540 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:09.699359    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2" event={"ID":"5707b339-e33c-45c6-a7b7-6ce04b34149b","Type":"ContainerDied","Data":"27d344d10af4eb6e70fd9cb847b35b0d24a73de70a11deca62f1263bcaac17cd"}
Apr 28 20:27:10.633162 ip-10-0-131-5 kubenswrapper[2569]: W0428 20:27:10.633134    2569 logging.go:55] [core] [Channel #2425 SubChannel #2426]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.60:9003", ServerName: "10.133.0.60:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.60:9003: connect: connection refused"
Apr 28 20:27:10.706119 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.706085    2569 generic.go:358] "Generic (PLEG): container finished" podID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerID="7ca9f79d854b2e298f161f576cd74793926cb44f5e7a7fb9abedf254615049f0" exitCode=0
Apr 28 20:27:10.706278 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.706171    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2" event={"ID":"5707b339-e33c-45c6-a7b7-6ce04b34149b","Type":"ContainerDied","Data":"7ca9f79d854b2e298f161f576cd74793926cb44f5e7a7fb9abedf254615049f0"}
Apr 28 20:27:10.738912 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.738892    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:27:10.828861 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.828777    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-cache\") pod \"5707b339-e33c-45c6-a7b7-6ce04b34149b\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") "
Apr 28 20:27:10.829019 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.828859    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-uds\") pod \"5707b339-e33c-45c6-a7b7-6ce04b34149b\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") "
Apr 28 20:27:10.829019 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.828928    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zprtj\" (UniqueName: \"kubernetes.io/projected/5707b339-e33c-45c6-a7b7-6ce04b34149b-kube-api-access-zprtj\") pod \"5707b339-e33c-45c6-a7b7-6ce04b34149b\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") "
Apr 28 20:27:10.829019 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.828958    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5707b339-e33c-45c6-a7b7-6ce04b34149b-tls-certs\") pod \"5707b339-e33c-45c6-a7b7-6ce04b34149b\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") "
Apr 28 20:27:10.829191 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.829036    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-kserve-provision-location\") pod \"5707b339-e33c-45c6-a7b7-6ce04b34149b\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") "
Apr 28 20:27:10.829191 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.829100    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-tmp\") pod \"5707b339-e33c-45c6-a7b7-6ce04b34149b\" (UID: \"5707b339-e33c-45c6-a7b7-6ce04b34149b\") "
Apr 28 20:27:10.829191 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.829101    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "5707b339-e33c-45c6-a7b7-6ce04b34149b" (UID: "5707b339-e33c-45c6-a7b7-6ce04b34149b"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:27:10.829191 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.829125    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "5707b339-e33c-45c6-a7b7-6ce04b34149b" (UID: "5707b339-e33c-45c6-a7b7-6ce04b34149b"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:27:10.829466 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.829443    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-cache\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:27:10.829466 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.829467    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-uds\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:27:10.829610 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.829463    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "5707b339-e33c-45c6-a7b7-6ce04b34149b" (UID: "5707b339-e33c-45c6-a7b7-6ce04b34149b"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:27:10.829757 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.829740    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "5707b339-e33c-45c6-a7b7-6ce04b34149b" (UID: "5707b339-e33c-45c6-a7b7-6ce04b34149b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:27:10.831144 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.831118    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5707b339-e33c-45c6-a7b7-6ce04b34149b-kube-api-access-zprtj" (OuterVolumeSpecName: "kube-api-access-zprtj") pod "5707b339-e33c-45c6-a7b7-6ce04b34149b" (UID: "5707b339-e33c-45c6-a7b7-6ce04b34149b"). InnerVolumeSpecName "kube-api-access-zprtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 20:27:10.831241 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.831118    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5707b339-e33c-45c6-a7b7-6ce04b34149b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5707b339-e33c-45c6-a7b7-6ce04b34149b" (UID: "5707b339-e33c-45c6-a7b7-6ce04b34149b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 20:27:10.930879 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.930834    2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-kserve-provision-location\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:27:10.930879 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.930882    2569 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/5707b339-e33c-45c6-a7b7-6ce04b34149b-tokenizer-tmp\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:27:10.931075 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.930899    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zprtj\" (UniqueName: \"kubernetes.io/projected/5707b339-e33c-45c6-a7b7-6ce04b34149b-kube-api-access-zprtj\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:27:10.931075 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:10.930916    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5707b339-e33c-45c6-a7b7-6ce04b34149b-tls-certs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:27:11.633559 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:11.633521    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2" podUID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.133.0.60:9003\" within 1s: context deadline exceeded"
Apr 28 20:27:11.633559 ip-10-0-131-5 kubenswrapper[2569]: W0428 20:27:11.633546    2569 logging.go:55] [core] [Channel #2425 SubChannel #2426]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.60:9003", ServerName: "10.133.0.60:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.60:9003: operation was canceled"
Apr 28 20:27:11.712193 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:11.712160    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2" event={"ID":"5707b339-e33c-45c6-a7b7-6ce04b34149b","Type":"ContainerDied","Data":"7682671e641c270826316a3f9b301b94e487c67d34c98f776542d45d2ec90bef"}
Apr 28 20:27:11.712193 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:11.712199    2569 scope.go:117] "RemoveContainer" containerID="7ca9f79d854b2e298f161f576cd74793926cb44f5e7a7fb9abedf254615049f0"
Apr 28 20:27:11.712430 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:11.712237    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"
Apr 28 20:27:11.721303 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:11.721281    2569 scope.go:117] "RemoveContainer" containerID="27d344d10af4eb6e70fd9cb847b35b0d24a73de70a11deca62f1263bcaac17cd"
Apr 28 20:27:11.729521 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:11.729501    2569 scope.go:117] "RemoveContainer" containerID="7706d71bf39fc9212c33d0efedb19fb541490ef34dc5e721e08916f6e9bd0596"
Apr 28 20:27:11.741016 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:11.739016    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"]
Apr 28 20:27:11.744206 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:11.744183    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-65cd765c9m7hx2"]
Apr 28 20:27:11.943313 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:11.943272    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5707b339-e33c-45c6-a7b7-6ce04b34149b" path="/var/lib/kubelet/pods/5707b339-e33c-45c6-a7b7-6ce04b34149b/volumes"
Apr 28 20:27:23.340625 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.340592    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"]
Apr 28 20:27:23.341018 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.341001    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerName="storage-initializer"
Apr 28 20:27:23.341018 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.341012    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerName="storage-initializer"
Apr 28 20:27:23.341115 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.341032    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerName="tokenizer"
Apr 28 20:27:23.341115 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.341038    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerName="tokenizer"
Apr 28 20:27:23.341115 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.341049    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerName="main"
Apr 28 20:27:23.341115 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.341055    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerName="main"
Apr 28 20:27:23.341283 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.341117    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerName="tokenizer"
Apr 28 20:27:23.341283 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.341134    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="5707b339-e33c-45c6-a7b7-6ce04b34149b" containerName="main"
Apr 28 20:27:23.343539 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.343509    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.346391 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.346337    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv8de1d74aab16d9cabd8b5aafeb5248e8-kserve-self-signed-certs\""
Apr 28 20:27:23.355849 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.355823    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"]
Apr 28 20:27:23.461019 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.460973    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.461297 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.461272    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.461544 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.461518    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-home\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.461662 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.461576    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgkvj\" (UniqueName: \"kubernetes.io/projected/705c0924-9aa7-41d5-91ec-e94cdec5a095-kube-api-access-tgkvj\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.461662 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.461600    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/705c0924-9aa7-41d5-91ec-e94cdec5a095-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.461662 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.461633    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-dshm\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.461662 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.461659    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.562742 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.562702    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-home\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.562925 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.562761    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tgkvj\" (UniqueName: \"kubernetes.io/projected/705c0924-9aa7-41d5-91ec-e94cdec5a095-kube-api-access-tgkvj\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.562925 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.562780    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/705c0924-9aa7-41d5-91ec-e94cdec5a095-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.562925 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.562805    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-dshm\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.562925 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.562824    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.562925 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.562869    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.562925 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.562907    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.563305 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.563276    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.563394 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.563341    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.563394 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.563366    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.563600 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.563561    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-home\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.565265 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.565244    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-dshm\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.565744 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.565720    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/705c0924-9aa7-41d5-91ec-e94cdec5a095-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.571364 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.571337    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgkvj\" (UniqueName: \"kubernetes.io/projected/705c0924-9aa7-41d5-91ec-e94cdec5a095-kube-api-access-tgkvj\") pod \"llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.656581 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.656484    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:23.799804 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:23.799779    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"]
Apr 28 20:27:23.802347 ip-10-0-131-5 kubenswrapper[2569]: W0428 20:27:23.802313    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod705c0924_9aa7_41d5_91ec_e94cdec5a095.slice/crio-2af7495199378382f09513c2276b30c3aca8a4bd74143116af7a0afc675b807d WatchSource:0}: Error finding container 2af7495199378382f09513c2276b30c3aca8a4bd74143116af7a0afc675b807d: Status 404 returned error can't find the container with id 2af7495199378382f09513c2276b30c3aca8a4bd74143116af7a0afc675b807d
Apr 28 20:27:24.772072 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:24.772033    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" event={"ID":"705c0924-9aa7-41d5-91ec-e94cdec5a095","Type":"ContainerStarted","Data":"6b514d75ab60c58d2f79b9c8def488783b19f92b7646c2305e533cdccfad9e08"}
Apr 28 20:27:24.772550 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:24.772080    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" event={"ID":"705c0924-9aa7-41d5-91ec-e94cdec5a095","Type":"ContainerStarted","Data":"2af7495199378382f09513c2276b30c3aca8a4bd74143116af7a0afc675b807d"}
Apr 28 20:27:28.790927 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:28.790891    2569 generic.go:358] "Generic (PLEG): container finished" podID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerID="6b514d75ab60c58d2f79b9c8def488783b19f92b7646c2305e533cdccfad9e08" exitCode=0
Apr 28 20:27:28.791337 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:28.790967    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" event={"ID":"705c0924-9aa7-41d5-91ec-e94cdec5a095","Type":"ContainerDied","Data":"6b514d75ab60c58d2f79b9c8def488783b19f92b7646c2305e533cdccfad9e08"}
Apr 28 20:27:29.798793 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:29.798763    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" event={"ID":"705c0924-9aa7-41d5-91ec-e94cdec5a095","Type":"ContainerStarted","Data":"fc5efd50ac7af8df4aa7f2e51eaa863bb987c8742965978a30b1e93e114fd4bd"}
Apr 28 20:27:29.821056 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:29.821012    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" podStartSLOduration=6.820999522 podStartE2EDuration="6.820999522s" podCreationTimestamp="2026-04-28 20:27:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 20:27:29.818728846 +0000 UTC m=+4236.643068937" watchObservedRunningTime="2026-04-28 20:27:29.820999522 +0000 UTC m=+4236.645339630"
Apr 28 20:27:33.657272 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:33.657233    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:33.657272 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:33.657269    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:27:33.658880 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:33.658850    2569 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Apr 28 20:27:43.657585 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:43.657539    2569 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Apr 28 20:27:53.657884 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:27:53.657840    2569 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Apr 28 20:28:03.657152 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:28:03.657097    2569 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Apr 28 20:28:13.657917 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:28:13.657866    2569 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Apr 28 20:28:23.657248 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:28:23.657206    2569 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Apr 28 20:28:33.657049 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:28:33.656994    2569 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Apr 28 20:28:43.657420 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:28:43.657302    2569 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Apr 28 20:28:53.657727 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:28:53.657672    2569 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Apr 28 20:29:03.667432 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:03.667380    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:29:03.675146 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:03.675124    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:29:08.107294 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:08.107256    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"]
Apr 28 20:29:08.107704 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:08.107551    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="main" containerID="cri-o://fc5efd50ac7af8df4aa7f2e51eaa863bb987c8742965978a30b1e93e114fd4bd" gracePeriod=30
Apr 28 20:29:38.331809 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.331787    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd_705c0924-9aa7-41d5-91ec-e94cdec5a095/main/0.log"
Apr 28 20:29:38.332210 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.332167    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:29:38.336423 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.336382    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd_705c0924-9aa7-41d5-91ec-e94cdec5a095/main/0.log"
Apr 28 20:29:38.336755 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.336722    2569 generic.go:358] "Generic (PLEG): container finished" podID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerID="fc5efd50ac7af8df4aa7f2e51eaa863bb987c8742965978a30b1e93e114fd4bd" exitCode=137
Apr 28 20:29:38.336867 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.336793    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"
Apr 28 20:29:38.336867 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.336815    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" event={"ID":"705c0924-9aa7-41d5-91ec-e94cdec5a095","Type":"ContainerDied","Data":"fc5efd50ac7af8df4aa7f2e51eaa863bb987c8742965978a30b1e93e114fd4bd"}
Apr 28 20:29:38.336867 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.336841    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd" event={"ID":"705c0924-9aa7-41d5-91ec-e94cdec5a095","Type":"ContainerDied","Data":"2af7495199378382f09513c2276b30c3aca8a4bd74143116af7a0afc675b807d"}
Apr 28 20:29:38.336867 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.336860    2569 scope.go:117] "RemoveContainer" containerID="fc5efd50ac7af8df4aa7f2e51eaa863bb987c8742965978a30b1e93e114fd4bd"
Apr 28 20:29:38.345449 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.345431    2569 scope.go:117] "RemoveContainer" containerID="6b514d75ab60c58d2f79b9c8def488783b19f92b7646c2305e533cdccfad9e08"
Apr 28 20:29:38.391025 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.390951    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-dshm\") pod \"705c0924-9aa7-41d5-91ec-e94cdec5a095\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") "
Apr 28 20:29:38.391025 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.390995    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-tmp-dir\") pod \"705c0924-9aa7-41d5-91ec-e94cdec5a095\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") "
Apr 28 20:29:38.391234 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.391093    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgkvj\" (UniqueName: \"kubernetes.io/projected/705c0924-9aa7-41d5-91ec-e94cdec5a095-kube-api-access-tgkvj\") pod \"705c0924-9aa7-41d5-91ec-e94cdec5a095\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") "
Apr 28 20:29:38.391234 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.391159    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/705c0924-9aa7-41d5-91ec-e94cdec5a095-tls-certs\") pod \"705c0924-9aa7-41d5-91ec-e94cdec5a095\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") "
Apr 28 20:29:38.391234 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.391186    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-kserve-provision-location\") pod \"705c0924-9aa7-41d5-91ec-e94cdec5a095\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") "
Apr 28 20:29:38.391234 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.391213    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-home\") pod \"705c0924-9aa7-41d5-91ec-e94cdec5a095\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") "
Apr 28 20:29:38.391470 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.391256    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-model-cache\") pod \"705c0924-9aa7-41d5-91ec-e94cdec5a095\" (UID: \"705c0924-9aa7-41d5-91ec-e94cdec5a095\") "
Apr 28 20:29:38.391883 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.391804    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-model-cache" (OuterVolumeSpecName: "model-cache") pod "705c0924-9aa7-41d5-91ec-e94cdec5a095" (UID: "705c0924-9aa7-41d5-91ec-e94cdec5a095"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:29:38.392154 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.392132    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-home" (OuterVolumeSpecName: "home") pod "705c0924-9aa7-41d5-91ec-e94cdec5a095" (UID: "705c0924-9aa7-41d5-91ec-e94cdec5a095"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:29:38.393693 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.393664    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/705c0924-9aa7-41d5-91ec-e94cdec5a095-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "705c0924-9aa7-41d5-91ec-e94cdec5a095" (UID: "705c0924-9aa7-41d5-91ec-e94cdec5a095"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 20:29:38.393791 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.393668    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-dshm" (OuterVolumeSpecName: "dshm") pod "705c0924-9aa7-41d5-91ec-e94cdec5a095" (UID: "705c0924-9aa7-41d5-91ec-e94cdec5a095"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:29:38.393791 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.393772    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/705c0924-9aa7-41d5-91ec-e94cdec5a095-kube-api-access-tgkvj" (OuterVolumeSpecName: "kube-api-access-tgkvj") pod "705c0924-9aa7-41d5-91ec-e94cdec5a095" (UID: "705c0924-9aa7-41d5-91ec-e94cdec5a095"). InnerVolumeSpecName "kube-api-access-tgkvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 20:29:38.403201 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.403176    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "705c0924-9aa7-41d5-91ec-e94cdec5a095" (UID: "705c0924-9aa7-41d5-91ec-e94cdec5a095"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:29:38.415724 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.415707    2569 scope.go:117] "RemoveContainer" containerID="fc5efd50ac7af8df4aa7f2e51eaa863bb987c8742965978a30b1e93e114fd4bd"
Apr 28 20:29:38.416036 ip-10-0-131-5 kubenswrapper[2569]: E0428 20:29:38.415998    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc5efd50ac7af8df4aa7f2e51eaa863bb987c8742965978a30b1e93e114fd4bd\": container with ID starting with fc5efd50ac7af8df4aa7f2e51eaa863bb987c8742965978a30b1e93e114fd4bd not found: ID does not exist" containerID="fc5efd50ac7af8df4aa7f2e51eaa863bb987c8742965978a30b1e93e114fd4bd"
Apr 28 20:29:38.416103 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.416043    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc5efd50ac7af8df4aa7f2e51eaa863bb987c8742965978a30b1e93e114fd4bd"} err="failed to get container status \"fc5efd50ac7af8df4aa7f2e51eaa863bb987c8742965978a30b1e93e114fd4bd\": rpc error: code = NotFound desc = could not find container \"fc5efd50ac7af8df4aa7f2e51eaa863bb987c8742965978a30b1e93e114fd4bd\": container with ID starting with fc5efd50ac7af8df4aa7f2e51eaa863bb987c8742965978a30b1e93e114fd4bd not found: ID does not exist"
Apr 28 20:29:38.416103 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.416064    2569 scope.go:117] "RemoveContainer" containerID="6b514d75ab60c58d2f79b9c8def488783b19f92b7646c2305e533cdccfad9e08"
Apr 28 20:29:38.416306 ip-10-0-131-5 kubenswrapper[2569]: E0428 20:29:38.416289    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b514d75ab60c58d2f79b9c8def488783b19f92b7646c2305e533cdccfad9e08\": container with ID starting with 6b514d75ab60c58d2f79b9c8def488783b19f92b7646c2305e533cdccfad9e08 not found: ID does not exist" containerID="6b514d75ab60c58d2f79b9c8def488783b19f92b7646c2305e533cdccfad9e08"
Apr 28 20:29:38.416365 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.416311    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b514d75ab60c58d2f79b9c8def488783b19f92b7646c2305e533cdccfad9e08"} err="failed to get container status \"6b514d75ab60c58d2f79b9c8def488783b19f92b7646c2305e533cdccfad9e08\": rpc error: code = NotFound desc = could not find container \"6b514d75ab60c58d2f79b9c8def488783b19f92b7646c2305e533cdccfad9e08\": container with ID starting with 6b514d75ab60c58d2f79b9c8def488783b19f92b7646c2305e533cdccfad9e08 not found: ID does not exist"
Apr 28 20:29:38.450250 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.450225    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "705c0924-9aa7-41d5-91ec-e94cdec5a095" (UID: "705c0924-9aa7-41d5-91ec-e94cdec5a095"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:29:38.492916 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.492896    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgkvj\" (UniqueName: \"kubernetes.io/projected/705c0924-9aa7-41d5-91ec-e94cdec5a095-kube-api-access-tgkvj\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:29:38.492916 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.492917    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/705c0924-9aa7-41d5-91ec-e94cdec5a095-tls-certs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:29:38.493041 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.492928    2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-kserve-provision-location\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:29:38.493041 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.492937    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-home\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:29:38.493041 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.492945    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-model-cache\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:29:38.493041 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.492954    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-dshm\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:29:38.493041 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.492965    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/705c0924-9aa7-41d5-91ec-e94cdec5a095-tmp-dir\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:29:38.661457 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.661429    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"]
Apr 28 20:29:38.664328 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:38.664300    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-76c49fccf6bsmgd"]
Apr 28 20:29:39.943325 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:29:39.943288    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" path="/var/lib/kubelet/pods/705c0924-9aa7-41d5-91ec-e94cdec5a095/volumes"
Apr 28 20:31:54.315584 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:31:54.315480    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:31:54.319524 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:31:54.318483    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:31:54.345671 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:31:54.345655    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:31:54.348711 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:31:54.348693    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:36:54.348288 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:36:54.348176    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:36:54.352295 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:36:54.351184    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:36:54.383832 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:36:54.383814    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:36:54.387105 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:36:54.387087    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:37:33.631472 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:33.631436    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
Apr 28 20:37:33.631992 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:33.631716    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" podUID="b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" containerName="main" containerID="cri-o://0a27c366fffae8b38fae603309c1d9dafc13caee4be2bdce665e90de97a05a57" gracePeriod=30
Apr 28 20:37:34.273835 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.273804    2569 generic.go:358] "Generic (PLEG): container finished" podID="b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" containerID="0a27c366fffae8b38fae603309c1d9dafc13caee4be2bdce665e90de97a05a57" exitCode=0
Apr 28 20:37:34.273946 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.273874    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc","Type":"ContainerDied","Data":"0a27c366fffae8b38fae603309c1d9dafc13caee4be2bdce665e90de97a05a57"}
Apr 28 20:37:34.273946 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.273907    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc","Type":"ContainerDied","Data":"91e4b058b9524f3d8bffa03c290679c75e33cb613c2e0a942bf38cae89424fb8"}
Apr 28 20:37:34.273946 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.273918    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91e4b058b9524f3d8bffa03c290679c75e33cb613c2e0a942bf38cae89424fb8"
Apr 28 20:37:34.277264 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.277247    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:37:34.400509 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.400425    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-model-cache\") pod \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") "
Apr 28 20:37:34.400509 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.400459    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-home\") pod \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") "
Apr 28 20:37:34.400509 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.400501    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8gl5\" (UniqueName: \"kubernetes.io/projected/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-kube-api-access-q8gl5\") pod \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") "
Apr 28 20:37:34.400778 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.400554    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-tmp-dir\") pod \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") "
Apr 28 20:37:34.400778 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.400626    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-tls-certs\") pod \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") "
Apr 28 20:37:34.400778 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.400659    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-dshm\") pod \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") "
Apr 28 20:37:34.400778 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.400700    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" (UID: "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:37:34.400778 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.400740    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-kserve-provision-location\") pod \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\" (UID: \"b67edfe7-5bba-4b86-b378-c71cb7a3b2dc\") "
Apr 28 20:37:34.401040 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.401027    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-model-cache\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:37:34.401186 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.401138    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-home" (OuterVolumeSpecName: "home") pod "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" (UID: "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:37:34.402783 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.402753    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-kube-api-access-q8gl5" (OuterVolumeSpecName: "kube-api-access-q8gl5") pod "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" (UID: "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc"). InnerVolumeSpecName "kube-api-access-q8gl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 28 20:37:34.403384 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.403357    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" (UID: "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 28 20:37:34.403384 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.403363    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-dshm" (OuterVolumeSpecName: "dshm") pod "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" (UID: "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:37:34.419483 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.419456    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" (UID: "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:37:34.455530 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.455502    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" (UID: "b67edfe7-5bba-4b86-b378-c71cb7a3b2dc"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 28 20:37:34.502148 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.502123    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-tmp-dir\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:37:34.502148 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.502146    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-tls-certs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:37:34.502278 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.502155    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-dshm\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:37:34.502278 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.502165    2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-kserve-provision-location\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:37:34.502278 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.502174    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-home\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:37:34.502278 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:34.502183    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8gl5\" (UniqueName: \"kubernetes.io/projected/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc-kube-api-access-q8gl5\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\""
Apr 28 20:37:35.277468 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:35.277437    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 28 20:37:35.300395 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:35.300371    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
Apr 28 20:37:35.304023 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:35.304002    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
Apr 28 20:37:35.943361 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:35.943326    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" path="/var/lib/kubelet/pods/b67edfe7-5bba-4b86-b378-c71cb7a3b2dc/volumes"
Apr 28 20:37:48.356439 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:48.356347    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:37:49.391938 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:49.391905    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:37:50.383886 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:50.383844    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:37:51.373946 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:51.373917    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:37:52.346716 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:52.346680    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:37:53.345245 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:53.345214    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:37:54.321218 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:54.321175    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:37:54.368310 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:54.368284    2569 scope.go:117] "RemoveContainer" containerID="0a27c366fffae8b38fae603309c1d9dafc13caee4be2bdce665e90de97a05a57"
Apr 28 20:37:54.376919 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:54.376893    2569 scope.go:117] "RemoveContainer" containerID="6e52d0481f5e428346235428d323b02492758957876b48e7c75dd72893502e97"
Apr 28 20:37:55.304742 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:55.304712    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:37:56.282575 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:56.282547    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:37:57.280465 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:57.280436    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:37:58.256394 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:58.256363    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:37:59.266992 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:37:59.266952    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:38:00.328727 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:00.328677    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:38:01.365272 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:01.365243    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-scfv4_fddcd380-e919-4cca-9442-953eee906665/istio-proxy/0.log"
Apr 28 20:38:02.374488 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:02.374454    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-55ff986f96-98lk2_5138a970-8b27-4382-a1e4-d2e459940e30/discovery/0.log"
Apr 28 20:38:02.404037 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:02.404005    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-7c5447bb76-2dmrp_10308d5b-7938-424c-8136-95e5bac92f07/istio-proxy/0.log"
Apr 28 20:38:03.168084 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:03.168053    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-55ff986f96-98lk2_5138a970-8b27-4382-a1e4-d2e459940e30/discovery/0.log"
Apr 28 20:38:03.196728 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:03.196693    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-7c5447bb76-2dmrp_10308d5b-7938-424c-8136-95e5bac92f07/istio-proxy/0.log"
Apr 28 20:38:03.968164 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:03.968125    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-55ff986f96-98lk2_5138a970-8b27-4382-a1e4-d2e459940e30/discovery/0.log"
Apr 28 20:38:04.000781 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:04.000751    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-7c5447bb76-2dmrp_10308d5b-7938-424c-8136-95e5bac92f07/istio-proxy/0.log"
Apr 28 20:38:04.730634 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:04.730603    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-gslhk_b1410653-ac40-400a-82c8-c26c0ce1ea45/authorino/0.log"
Apr 28 20:38:04.744350 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:04.744329    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-qrpft_21d8f832-2d09-4067-af3c-56fcf6e4d485/manager/0.log"
Apr 28 20:38:04.797472 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:04.797450    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6ddf9554fc-6qqhg_c314cdb3-b328-4177-9e72-6910aaeeed25/manager/0.log"
Apr 28 20:38:05.591416 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:05.591373    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-gslhk_b1410653-ac40-400a-82c8-c26c0ce1ea45/authorino/0.log"
Apr 28 20:38:05.605123 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:05.605081    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-qrpft_21d8f832-2d09-4067-af3c-56fcf6e4d485/manager/0.log"
Apr 28 20:38:05.662315 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:05.662292    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6ddf9554fc-6qqhg_c314cdb3-b328-4177-9e72-6910aaeeed25/manager/0.log"
Apr 28 20:38:06.465556 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:06.465516    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-gslhk_b1410653-ac40-400a-82c8-c26c0ce1ea45/authorino/0.log"
Apr 28 20:38:06.476266 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:06.476245    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-qrpft_21d8f832-2d09-4067-af3c-56fcf6e4d485/manager/0.log"
Apr 28 20:38:06.534082 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:06.534048    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6ddf9554fc-6qqhg_c314cdb3-b328-4177-9e72-6910aaeeed25/manager/0.log"
Apr 28 20:38:07.357204 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:07.357175    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-gslhk_b1410653-ac40-400a-82c8-c26c0ce1ea45/authorino/0.log"
Apr 28 20:38:07.368045 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:07.368022    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-qrpft_21d8f832-2d09-4067-af3c-56fcf6e4d485/manager/0.log"
Apr 28 20:38:07.427577 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:07.427550    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6ddf9554fc-6qqhg_c314cdb3-b328-4177-9e72-6910aaeeed25/manager/0.log"
Apr 28 20:38:08.214672 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:08.214637    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-gslhk_b1410653-ac40-400a-82c8-c26c0ce1ea45/authorino/0.log"
Apr 28 20:38:08.227584 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:08.227561    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-qrpft_21d8f832-2d09-4067-af3c-56fcf6e4d485/manager/0.log"
Apr 28 20:38:08.285291 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:08.285255    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6ddf9554fc-6qqhg_c314cdb3-b328-4177-9e72-6910aaeeed25/manager/0.log"
Apr 28 20:38:10.579485 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.579422    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hjc6s/must-gather-jhwcn"]
Apr 28 20:38:10.579966 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.579947    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="storage-initializer"
Apr 28 20:38:10.580018 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.579968    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="storage-initializer"
Apr 28 20:38:10.580018 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.579985    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" containerName="main"
Apr 28 20:38:10.580018 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.579991    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" containerName="main"
Apr 28 20:38:10.580018 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.580000    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" containerName="storage-initializer"
Apr 28 20:38:10.580018 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.580006    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" containerName="storage-initializer"
Apr 28 20:38:10.580200 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.580021    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="main"
Apr 28 20:38:10.580200 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.580027    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="main"
Apr 28 20:38:10.580200 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.580089    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="b67edfe7-5bba-4b86-b378-c71cb7a3b2dc" containerName="main"
Apr 28 20:38:10.580200 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.580098    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="705c0924-9aa7-41d5-91ec-e94cdec5a095" containerName="main"
Apr 28 20:38:10.585059 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.585041    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hjc6s/must-gather-jhwcn"
Apr 28 20:38:10.589083 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.589054    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hjc6s\"/\"default-dockercfg-nkncc\""
Apr 28 20:38:10.589228 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.589157    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hjc6s\"/\"openshift-service-ca.crt\""
Apr 28 20:38:10.589491 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.589473    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hjc6s\"/\"kube-root-ca.crt\""
Apr 28 20:38:10.593439 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.593382    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hjc6s/must-gather-jhwcn"]
Apr 28 20:38:10.623901 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.623872    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2fq8\" (UniqueName: \"kubernetes.io/projected/6d797b8a-217f-45fc-9b1a-8cdfc386e903-kube-api-access-l2fq8\") pod \"must-gather-jhwcn\" (UID: \"6d797b8a-217f-45fc-9b1a-8cdfc386e903\") " pod="openshift-must-gather-hjc6s/must-gather-jhwcn"
Apr 28 20:38:10.624051 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.623922    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6d797b8a-217f-45fc-9b1a-8cdfc386e903-must-gather-output\") pod \"must-gather-jhwcn\" (UID: \"6d797b8a-217f-45fc-9b1a-8cdfc386e903\") " pod="openshift-must-gather-hjc6s/must-gather-jhwcn"
Apr 28 20:38:10.724475 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.724446    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6d797b8a-217f-45fc-9b1a-8cdfc386e903-must-gather-output\") pod \"must-gather-jhwcn\" (UID: \"6d797b8a-217f-45fc-9b1a-8cdfc386e903\") " pod="openshift-must-gather-hjc6s/must-gather-jhwcn"
Apr 28 20:38:10.724698 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.724581    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l2fq8\" (UniqueName: \"kubernetes.io/projected/6d797b8a-217f-45fc-9b1a-8cdfc386e903-kube-api-access-l2fq8\") pod \"must-gather-jhwcn\" (UID: \"6d797b8a-217f-45fc-9b1a-8cdfc386e903\") " pod="openshift-must-gather-hjc6s/must-gather-jhwcn"
Apr 28 20:38:10.724775 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.724755    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6d797b8a-217f-45fc-9b1a-8cdfc386e903-must-gather-output\") pod \"must-gather-jhwcn\" (UID: \"6d797b8a-217f-45fc-9b1a-8cdfc386e903\") " pod="openshift-must-gather-hjc6s/must-gather-jhwcn"
Apr 28 20:38:10.733300 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.733277    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2fq8\" (UniqueName: \"kubernetes.io/projected/6d797b8a-217f-45fc-9b1a-8cdfc386e903-kube-api-access-l2fq8\") pod \"must-gather-jhwcn\" (UID: \"6d797b8a-217f-45fc-9b1a-8cdfc386e903\") " pod="openshift-must-gather-hjc6s/must-gather-jhwcn"
Apr 28 20:38:10.895161 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:10.895070    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hjc6s/must-gather-jhwcn"
Apr 28 20:38:11.020500 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:11.020471    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hjc6s/must-gather-jhwcn"]
Apr 28 20:38:11.021969 ip-10-0-131-5 kubenswrapper[2569]: W0428 20:38:11.021941    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d797b8a_217f_45fc_9b1a_8cdfc386e903.slice/crio-bc6ac3c09b4b841acf652442efa7bb5c2a3305ad9b851a21c29c3c5c47718173 WatchSource:0}: Error finding container bc6ac3c09b4b841acf652442efa7bb5c2a3305ad9b851a21c29c3c5c47718173: Status 404 returned error can't find the container with id bc6ac3c09b4b841acf652442efa7bb5c2a3305ad9b851a21c29c3c5c47718173
Apr 28 20:38:11.023759 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:11.023742    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 28 20:38:11.421230 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:11.421196    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hjc6s/must-gather-jhwcn" event={"ID":"6d797b8a-217f-45fc-9b1a-8cdfc386e903","Type":"ContainerStarted","Data":"bc6ac3c09b4b841acf652442efa7bb5c2a3305ad9b851a21c29c3c5c47718173"}
Apr 28 20:38:12.430116 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:12.430020    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hjc6s/must-gather-jhwcn" event={"ID":"6d797b8a-217f-45fc-9b1a-8cdfc386e903","Type":"ContainerStarted","Data":"79839173cd504fb097685a4e679fea48b039e4758b65c628ad2c132c376a43df"}
Apr 28 20:38:12.430116 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:12.430070    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hjc6s/must-gather-jhwcn" event={"ID":"6d797b8a-217f-45fc-9b1a-8cdfc386e903","Type":"ContainerStarted","Data":"6a141f327a3c9c10652e475ee3468fac216c6d96568aaa4f473068595688af06"}
Apr 28 20:38:12.448023 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:12.447976    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-hjc6s/must-gather-jhwcn" podStartSLOduration=1.6844835200000001 podStartE2EDuration="2.447961014s" podCreationTimestamp="2026-04-28 20:38:10 +0000 UTC" firstStartedPulling="2026-04-28 20:38:11.02386727 +0000 UTC m=+4877.848207356" lastFinishedPulling="2026-04-28 20:38:11.787344749 +0000 UTC m=+4878.611684850" observedRunningTime="2026-04-28 20:38:12.446482743 +0000 UTC m=+4879.270822853" watchObservedRunningTime="2026-04-28 20:38:12.447961014 +0000 UTC m=+4879.272301122"
Apr 28 20:38:13.349188 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:13.349152    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rw72g_a444a3ec-874d-4d0f-a9f8-4cc5a9c3846e/global-pull-secret-syncer/0.log"
Apr 28 20:38:13.393855 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:13.393826    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jxpx9_2a07090c-27b9-4995-8133-9976a9919809/konnectivity-agent/0.log"
Apr 28 20:38:13.530890 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:13.530856    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-5.ec2.internal_55d83b29c984d704f2c407ca2173be08/haproxy/0.log"
Apr 28 20:38:17.279270 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:17.279230    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-gslhk_b1410653-ac40-400a-82c8-c26c0ce1ea45/authorino/0.log"
Apr 28 20:38:17.304823 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:17.304796    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-qrpft_21d8f832-2d09-4067-af3c-56fcf6e4d485/manager/0.log"
Apr 28 20:38:17.406595 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:17.406543    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6ddf9554fc-6qqhg_c314cdb3-b328-4177-9e72-6910aaeeed25/manager/0.log"
Apr 28 20:38:18.436387 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.436352    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_52f98b6c-1246-4518-b036-c837710fa2cf/alertmanager/0.log"
Apr 28 20:38:18.458702 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.458629    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_52f98b6c-1246-4518-b036-c837710fa2cf/config-reloader/0.log"
Apr 28 20:38:18.472654 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.472627    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_52f98b6c-1246-4518-b036-c837710fa2cf/kube-rbac-proxy-web/0.log"
Apr 28 20:38:18.489596 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.489530    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_52f98b6c-1246-4518-b036-c837710fa2cf/kube-rbac-proxy/0.log"
Apr 28 20:38:18.507159 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.507130    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_52f98b6c-1246-4518-b036-c837710fa2cf/kube-rbac-proxy-metric/0.log"
Apr 28 20:38:18.526550 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.526518    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_52f98b6c-1246-4518-b036-c837710fa2cf/prom-label-proxy/0.log"
Apr 28 20:38:18.551765 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.551731    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_52f98b6c-1246-4518-b036-c837710fa2cf/init-config-reloader/0.log"
Apr 28 20:38:18.692957 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.692889    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-8677d4bd78-9jx6t_a20ecd11-5310-4c8d-ae67-90b4bf922cb7/metrics-server/0.log"
Apr 28 20:38:18.721483 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.721451    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7dccd58f55-lpxtz_de9f763c-a931-402d-abd1-2f0b4f2971ff/monitoring-plugin/0.log"
Apr 28 20:38:18.746783 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.746747    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8d5rz_f6205f07-25d7-4a09-8c48-c01358a1e07c/node-exporter/0.log"
Apr 28 20:38:18.763821 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.763797    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8d5rz_f6205f07-25d7-4a09-8c48-c01358a1e07c/kube-rbac-proxy/0.log"
Apr 28 20:38:18.791311 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.791281    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8d5rz_f6205f07-25d7-4a09-8c48-c01358a1e07c/init-textfile/0.log"
Apr 28 20:38:18.961525 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.961436    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9d44df66c-ljn9c_6e7e390c-5479-46d7-af9f-3bcdc26f7a64/kube-rbac-proxy-main/0.log"
Apr 28 20:38:18.979635 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:18.979550    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9d44df66c-ljn9c_6e7e390c-5479-46d7-af9f-3bcdc26f7a64/kube-rbac-proxy-self/0.log"
Apr 28 20:38:19.016108 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:19.016066    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9d44df66c-ljn9c_6e7e390c-5479-46d7-af9f-3bcdc26f7a64/openshift-state-metrics/0.log"
Apr 28 20:38:19.232644 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:19.232554    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-57cf98b594-jrfsp_ba388df3-0277-4c59-8dda-8c29928a0112/prometheus-operator-admission-webhook/0.log"
Apr 28 20:38:19.356910 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:19.356839    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7896b685b8-qtmcb_a9eb99a8-0b0b-48a5-9edc-69806958ec99/thanos-query/0.log"
Apr 28 20:38:19.373230 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:19.373205    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7896b685b8-qtmcb_a9eb99a8-0b0b-48a5-9edc-69806958ec99/kube-rbac-proxy-web/0.log"
Apr 28 20:38:19.393945 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:19.393921    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7896b685b8-qtmcb_a9eb99a8-0b0b-48a5-9edc-69806958ec99/kube-rbac-proxy/0.log"
Apr 28 20:38:19.410928 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:19.410904    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7896b685b8-qtmcb_a9eb99a8-0b0b-48a5-9edc-69806958ec99/prom-label-proxy/0.log"
Apr 28 20:38:19.428676 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:19.428640    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7896b685b8-qtmcb_a9eb99a8-0b0b-48a5-9edc-69806958ec99/kube-rbac-proxy-rules/0.log"
Apr 28 20:38:19.450693 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:19.450664    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7896b685b8-qtmcb_a9eb99a8-0b0b-48a5-9edc-69806958ec99/kube-rbac-proxy-metrics/0.log"
Apr 28 20:38:21.291528 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:21.291496    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/1.log"
Apr 28 20:38:21.299326 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:21.299293    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-6svf6_034e2b42-803b-45fd-a6df-4371bec965ff/console-operator/2.log"
Apr 28 20:38:21.719492 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:21.719461    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-778b87f87-v9zfd_f8243766-5ac7-495c-af6d-85b6df87fae2/console/0.log"
Apr 28 20:38:22.107988 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.107895    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"]
Apr 28 20:38:22.115377 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.115349    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.121201 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.121172    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"]
Apr 28 20:38:22.254064 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.254032    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/70a1f61f-2363-4590-8f79-63b849f0fbc2-podres\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.254211 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.254076    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70a1f61f-2363-4590-8f79-63b849f0fbc2-sys\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.254270 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.254207    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv84k\" (UniqueName: \"kubernetes.io/projected/70a1f61f-2363-4590-8f79-63b849f0fbc2-kube-api-access-xv84k\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.254270 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.254260    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/70a1f61f-2363-4590-8f79-63b849f0fbc2-proc\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.254350 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.254288    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70a1f61f-2363-4590-8f79-63b849f0fbc2-lib-modules\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.355825 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.355791    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xv84k\" (UniqueName: \"kubernetes.io/projected/70a1f61f-2363-4590-8f79-63b849f0fbc2-kube-api-access-xv84k\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.356456 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.355837    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/70a1f61f-2363-4590-8f79-63b849f0fbc2-proc\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.356456 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.355869    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70a1f61f-2363-4590-8f79-63b849f0fbc2-lib-modules\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.356456 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.355920    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/70a1f61f-2363-4590-8f79-63b849f0fbc2-podres\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.356456 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.355953    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70a1f61f-2363-4590-8f79-63b849f0fbc2-sys\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.356456 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.356036    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70a1f61f-2363-4590-8f79-63b849f0fbc2-lib-modules\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.356456 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.356051    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70a1f61f-2363-4590-8f79-63b849f0fbc2-sys\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.356456 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.356071    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/70a1f61f-2363-4590-8f79-63b849f0fbc2-proc\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.356456 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.356115    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/70a1f61f-2363-4590-8f79-63b849f0fbc2-podres\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.363549 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.363491    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv84k\" (UniqueName: \"kubernetes.io/projected/70a1f61f-2363-4590-8f79-63b849f0fbc2-kube-api-access-xv84k\") pod \"perf-node-gather-daemonset-2dt8n\" (UID: \"70a1f61f-2363-4590-8f79-63b849f0fbc2\") " pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.438156 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.434273    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:22.586688 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:22.586618    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"]
Apr 28 20:38:23.021432 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:23.021387    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lt6tx_35caf455-585a-430d-be29-50d6ad53fcd7/dns/0.log"
Apr 28 20:38:23.035906 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:23.035878    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lt6tx_35caf455-585a-430d-be29-50d6ad53fcd7/kube-rbac-proxy/0.log"
Apr 28 20:38:23.088525 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:23.088497    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qpmnk_989021b3-c3c6-43a9-8d1e-c2a8660c0d89/dns-node-resolver/0.log"
Apr 28 20:38:23.497051 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:23.497013    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n" event={"ID":"70a1f61f-2363-4590-8f79-63b849f0fbc2","Type":"ContainerStarted","Data":"9c88b972681314959d0ea7889efe1303dae7b3501ccf909d468370cde9ee9e02"}
Apr 28 20:38:23.497051 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:23.497050    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n" event={"ID":"70a1f61f-2363-4590-8f79-63b849f0fbc2","Type":"ContainerStarted","Data":"cd3c5210e0cfd71930b10c925eeef45f52ee661dc8c7a03870442fe4a74281fb"}
Apr 28 20:38:23.497513 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:23.497141    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:23.517350 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:23.517294    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n" podStartSLOduration=1.517276681 podStartE2EDuration="1.517276681s" podCreationTimestamp="2026-04-28 20:38:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-28 20:38:23.514541743 +0000 UTC m=+4890.338881853" watchObservedRunningTime="2026-04-28 20:38:23.517276681 +0000 UTC m=+4890.341616790"
Apr 28 20:38:23.528346 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:23.528318    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-549b6c67db-42h9s_fc141dec-64d4-4235-bae0-ec7e3e98edc4/registry/0.log"
Apr 28 20:38:23.599457 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:23.599423    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tzfn9_8c8595e0-ea1f-4107-b1b4-f9f55cdaf8ad/node-ca/0.log"
Apr 28 20:38:24.354212 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:24.354187    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-55ff986f96-98lk2_5138a970-8b27-4382-a1e4-d2e459940e30/discovery/0.log"
Apr 28 20:38:24.396705 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:24.396676    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-7c5447bb76-2dmrp_10308d5b-7938-424c-8136-95e5bac92f07/istio-proxy/0.log"
Apr 28 20:38:24.835198 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:24.835171    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g4hd4_f38d24be-b797-499f-8593-7b9ec113f103/serve-healthcheck-canary/0.log"
Apr 28 20:38:25.265447 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:25.265417    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-585dfdc468-x7kx7_10e853da-a692-43bb-b339-3bf2059fd151/insights-operator/0.log"
Apr 28 20:38:25.266451 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:25.266422    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-585dfdc468-x7kx7_10e853da-a692-43bb-b339-3bf2059fd151/insights-operator/1.log"
Apr 28 20:38:25.334704 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:25.334676    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fjrcg_aeee4d80-c824-4f03-a757-515670c34252/kube-rbac-proxy/0.log"
Apr 28 20:38:25.354159 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:25.354138    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fjrcg_aeee4d80-c824-4f03-a757-515670c34252/exporter/0.log"
Apr 28 20:38:25.370714 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:25.370686    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fjrcg_aeee4d80-c824-4f03-a757-515670c34252/extractor/0.log"
Apr 28 20:38:27.943851 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:27.943799    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-jvdxl_f370d3f9-0a7e-4402-8a6c-e94fbf56705c/openshift-lws-operator/0.log"
Apr 28 20:38:28.714634 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:28.714602    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-696fc77849-5g6mv_5222f0d4-ea9f-4485-8ad3-c95a80787aa1/manager/0.log"
Apr 28 20:38:28.752294 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:28.752272    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-86cc847c5c-8z66h_0991fd6c-085a-48ea-9214-9d446d11ab68/seaweedfs/0.log"
Apr 28 20:38:29.513746 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:29.513719    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hjc6s/perf-node-gather-daemonset-2dt8n"
Apr 28 20:38:33.475045 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:33.474963    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6769c5d45-x8wk8_abc28c3c-a5be-45cd-8095-f9d59dd936dc/kube-storage-version-migrator-operator/1.log"
Apr 28 20:38:33.476165 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:33.476144    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6769c5d45-x8wk8_abc28c3c-a5be-45cd-8095-f9d59dd936dc/kube-storage-version-migrator-operator/0.log"
Apr 28 20:38:34.526762 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:34.526726    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvd4r_1fd16e46-3b64-4b7f-9d06-6ed637847678/kube-multus-additional-cni-plugins/0.log"
Apr 28 20:38:34.542766 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:34.542739    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvd4r_1fd16e46-3b64-4b7f-9d06-6ed637847678/egress-router-binary-copy/0.log"
Apr 28 20:38:34.559704 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:34.559680    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvd4r_1fd16e46-3b64-4b7f-9d06-6ed637847678/cni-plugins/0.log"
Apr 28 20:38:34.575458 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:34.575427    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvd4r_1fd16e46-3b64-4b7f-9d06-6ed637847678/bond-cni-plugin/0.log"
Apr 28 20:38:34.596002 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:34.595980    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvd4r_1fd16e46-3b64-4b7f-9d06-6ed637847678/routeoverride-cni/0.log"
Apr 28 20:38:34.618668 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:34.618644    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvd4r_1fd16e46-3b64-4b7f-9d06-6ed637847678/whereabouts-cni-bincopy/0.log"
Apr 28 20:38:34.636730 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:34.636706    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvd4r_1fd16e46-3b64-4b7f-9d06-6ed637847678/whereabouts-cni/0.log"
Apr 28 20:38:34.800967 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:34.800904    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bp2ht_25d9e6b9-b643-471e-ae03-5d308920bbbb/kube-multus/0.log"
Apr 28 20:38:34.886316 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:34.886289    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hgm2f_e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33/network-metrics-daemon/0.log"
Apr 28 20:38:34.900768 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:34.900747    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hgm2f_e09c6f20-b816-4ac6-a0f3-37fd5d8a2d33/kube-rbac-proxy/0.log"
Apr 28 20:38:36.031523 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:36.031496    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-controller/0.log"
Apr 28 20:38:36.047194 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:36.047170    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/0.log"
Apr 28 20:38:36.070143 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:36.070115    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovn-acl-logging/1.log"
Apr 28 20:38:36.086088 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:36.086065    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/kube-rbac-proxy-node/0.log"
Apr 28 20:38:36.105030 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:36.104997    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/kube-rbac-proxy-ovn-metrics/0.log"
Apr 28 20:38:36.119048 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:36.119021    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/northd/0.log"
Apr 28 20:38:36.134613 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:36.134591    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/nbdb/0.log"
Apr 28 20:38:36.152102 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:36.152082    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/sbdb/0.log"
Apr 28 20:38:36.270924 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:36.270897    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ptjf5_578ad6cd-a560-4a18-8806-846b76fc9338/ovnkube-controller/0.log"
Apr 28 20:38:37.660549 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:37.660516    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-khqq9_ad86af47-f5e9-4e6f-8640-b3771e91074e/network-check-target-container/0.log"
Apr 28 20:38:38.574056 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:38.574030    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-fbwbg_88298985-0c2c-4787-9fa7-539c47c429e2/iptables-alerter/0.log"
Apr 28 20:38:39.337797 ip-10-0-131-5 kubenswrapper[2569]: I0428 20:38:39.337769    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-xkhgf_6a7302f5-8d7a-4665-af2a-81b3fe58a0c4/tuned/0.log"