Apr 17 07:52:08.831732 ip-10-0-143-45 systemd[1]: Starting Kubernetes Kubelet...
Apr 17 07:52:09.290197 ip-10-0-143-45 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Apr 17 07:52:09.290197 ip-10-0-143-45 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Apr 17 07:52:09.290197 ip-10-0-143-45 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Apr 17 07:52:09.290197 ip-10-0-143-45 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Apr 17 07:52:09.290197 ip-10-0-143-45 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Apr 17 07:52:09.291207 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.291056    2571 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Apr 17 07:52:09.296321 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296295    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 17 07:52:09.296321 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296315    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 17 07:52:09.296321 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296321    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 17 07:52:09.296321 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296326    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296331    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296335    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296342    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296345    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296350    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296354    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296357    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296361    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296364    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296368    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296372    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296376    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296382    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296387    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296391    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296395    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296398    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296402    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 17 07:52:09.296603 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296406    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296409    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296413    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296417    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296421    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296450    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296455    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296459    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296464    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296468    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296472    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296476    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296480    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296484    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296488    2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296493    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296497    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296502    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296507    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296511    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 17 07:52:09.297363 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296515    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296519    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296523    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296528    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296531    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296535    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296540    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296544    2571 feature_gate.go:328] unrecognized feature gate: Example2
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296548    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296555    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296562    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296568    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296573    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296579    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296583    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296588    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296592    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296597    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296601    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296605    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 17 07:52:09.298258 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296609    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296615    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296619    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296623    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296628    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296632    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296636    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296641    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296646    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296650    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296654    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296658    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296663    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296667    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296671    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296675    2571 feature_gate.go:328] unrecognized feature gate: Example
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296681    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296685    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296690    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296694    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 17 07:52:09.299272 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296698    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296702    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296706    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.296711    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297347    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297357    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297362    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297366    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297370    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297374    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297379    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297383    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297387    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297391    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297396    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297400    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297404    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297408    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297412    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297418    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 17 07:52:09.299795 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297423    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297427    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297453    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297458    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297462    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297466    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297470    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297474    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297480    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297484    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297489    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297493    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297497    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297501    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297506    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297510    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297516    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297520    2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297524    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297529    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 17 07:52:09.300369 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297533    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297537    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297541    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297545    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297549    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297553    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297558    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297562    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297566    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297570    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297574    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297579    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297583    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297588    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297592    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297596    2571 feature_gate.go:328] unrecognized feature gate: Example2
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297600    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297604    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297609    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 17 07:52:09.301100 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297613    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297619    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297628    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297633    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297637    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297641    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297645    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297649    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297653    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297657    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297662    2571 feature_gate.go:328] unrecognized feature gate: Example
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297666    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297669    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297674    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297678    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297683    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297691    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297695    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297699    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 17 07:52:09.301702 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297703    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297708    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297712    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297716    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297720    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297725    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297729    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297734    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297738    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297742    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297746    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.297750    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297857    2571 flags.go:64] FLAG: --address="0.0.0.0"
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297875    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297885    2571 flags.go:64] FLAG: --anonymous-auth="true"
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297891    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297899    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297906    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297913    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297920    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297925    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Apr 17 07:52:09.302181 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297930    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297936    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297941    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297946    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297951    2571 flags.go:64] FLAG: --cgroup-root=""
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297956    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297961    2571 flags.go:64] FLAG: --client-ca-file=""
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297966    2571 flags.go:64] FLAG: --cloud-config=""
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297970    2571 flags.go:64] FLAG: --cloud-provider="external"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297975    2571 flags.go:64] FLAG: --cluster-dns="[]"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297981    2571 flags.go:64] FLAG: --cluster-domain=""
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297986    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297992    2571 flags.go:64] FLAG: --config-dir=""
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.297996    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298001    2571 flags.go:64] FLAG: --container-log-max-files="5"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298008    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298013    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298018    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298023    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298028    2571 flags.go:64] FLAG: --contention-profiling="false"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298033    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298038    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298043    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298048    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298055    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Apr 17 07:52:09.302790 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298060    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298065    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298070    2571 flags.go:64] FLAG: --enable-load-reader="false"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298075    2571 flags.go:64] FLAG: --enable-server="true"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298080    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298089    2571 flags.go:64] FLAG: --event-burst="100"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298094    2571 flags.go:64] FLAG: --event-qps="50"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298099    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298104    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298109    2571 flags.go:64] FLAG: --eviction-hard=""
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298115    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298120    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298124    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298129    2571 flags.go:64] FLAG: --eviction-soft=""
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298134    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298139    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298144    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298149    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298154    2571 flags.go:64] FLAG: --fail-cgroupv1="false"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298158    2571 flags.go:64] FLAG: --fail-swap-on="true"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298163    2571 flags.go:64] FLAG: --feature-gates=""
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298169    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298174    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298179    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298185    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298190    2571 flags.go:64] FLAG: --healthz-port="10248"
Apr 17 07:52:09.303400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298196    2571 flags.go:64] FLAG: --help="false"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298201    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298205    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298210    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298215    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298221    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298227    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298232    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298236    2571 flags.go:64] FLAG: --image-service-endpoint=""
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298241    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298246    2571 flags.go:64] FLAG: --kube-api-burst="100"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298251    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298256    2571 flags.go:64] FLAG: --kube-api-qps="50"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298260    2571 flags.go:64] FLAG: --kube-reserved=""
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298265    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298270    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298275    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298279    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298283    2571 flags.go:64] FLAG: --lock-file=""
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298288    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298293    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298298    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298307    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Apr 17 07:52:09.304049 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298312    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298321    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298326    2571 flags.go:64] FLAG: --logging-format="text"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298331    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298337    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298342    2571 flags.go:64] FLAG: --manifest-url=""
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298347    2571 flags.go:64] FLAG: --manifest-url-header=""
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298354    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298359    2571 flags.go:64] FLAG: --max-open-files="1000000"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298366    2571 flags.go:64] FLAG: --max-pods="110"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298371    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298376    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298381    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298385    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298390    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298395    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298400    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298412    2571 flags.go:64] FLAG: --node-status-max-images="50"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298417    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298422    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298445    2571 flags.go:64] FLAG: --pod-cidr=""
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298450    2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c8cfe89231412ff3ee8cb6207fa0be33cad0f08e88c9c0f1e9f7e8c6f14d6715"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298460    2571 flags.go:64] FLAG: --pod-manifest-path=""
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298464    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Apr 17 07:52:09.304621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298470    2571 flags.go:64] FLAG: --pods-per-core="0"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298474    2571 flags.go:64] FLAG: --port="10250"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298479    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298484    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-079263282dc904958"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298489    2571 flags.go:64] FLAG: --qos-reserved=""
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298494    2571 flags.go:64] FLAG: --read-only-port="10255"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298501    2571 flags.go:64] FLAG: --register-node="true"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298506    2571 flags.go:64] FLAG: --register-schedulable="true"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298511    2571 flags.go:64] FLAG: --register-with-taints=""
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298517    2571 flags.go:64] FLAG: --registry-burst="10"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298524    2571 flags.go:64] FLAG: --registry-qps="5"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298528    2571 flags.go:64] FLAG: --reserved-cpus=""
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298533    2571 flags.go:64] FLAG: --reserved-memory=""
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298539    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298544    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298549    2571 flags.go:64] FLAG: --rotate-certificates="false"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298554    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298559    2571 flags.go:64] FLAG: --runonce="false"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298564    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298569    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298575    2571 flags.go:64] FLAG: --seccomp-default="false"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298579    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298584    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298589    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298594    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298599    2571 flags.go:64] FLAG: --storage-driver-password="root"
Apr 17 07:52:09.305201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298605    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298610    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298614    2571 flags.go:64] FLAG: --storage-driver-user="root"
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298620    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298625    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298630    2571 flags.go:64] FLAG: --system-cgroups=""
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298634    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298643    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298647    2571 flags.go:64] FLAG: --tls-cert-file=""
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298652    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298659    2571 flags.go:64] FLAG: --tls-min-version=""
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298663    2571 flags.go:64] FLAG: --tls-private-key-file=""
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298670    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298674    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298679    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298684    2571 flags.go:64] FLAG: --v="2"
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298691    2571 flags.go:64] FLAG: --version="false"
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298697    2571 flags.go:64] FLAG: --vmodule=""
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298704    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.298709    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298884    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298892    2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298898    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298903    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 17 07:52:09.305894 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298907    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298912    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298916    2571 feature_gate.go:328] unrecognized feature gate: Example
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298921    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298925    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298930    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298934    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298938    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298943    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298947    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298951    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298955    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298963    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298969    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298974    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298979    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298983    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298987    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298991    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 17 07:52:09.306479 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.298995    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299002    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299006    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299011    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299015    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299022    2571 feature_gate.go:328] unrecognized feature gate: Example2
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299026    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299030    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299034    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299038    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299043    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299047    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299051    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299056    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299060    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299064    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299069    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299073    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299077    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299082    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299086    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 17 07:52:09.306991 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299090    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299094    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299098    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299103    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299107    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299112    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299117    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299121    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299126    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299130    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299135    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299139    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299145    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299149    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299153    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299158    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299164    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299169    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299173    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 17 07:52:09.307521 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299177    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299181    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299185    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299190    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299194    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299198    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299203    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299207    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299211    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299216    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299220    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299224    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299230    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299236    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299240    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299244    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299249    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299253    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299257    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299262    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 17 07:52:09.307984 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299267    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299270    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.299274    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.299980    2571 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.306805    2571 server.go:530] "Kubelet version" kubeletVersion="v1.33.9"
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.306821    2571 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306870    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306875    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306878    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306882    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306886    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306889    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306892    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306895    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306898    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306900    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 17 07:52:09.308504 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306903    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306906    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306909    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306911    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306914    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306917    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306920    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306922    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306925    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306927    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306930    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306932    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306935    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306938    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306941    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306944    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306947    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306950    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306953    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306956    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 17 07:52:09.308903 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306959    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306961    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306963    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306966    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306969    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306971    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306973    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306976    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306979    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306982    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306986    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306988    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306991    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306994    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306997    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.306999    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307002    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307005    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307008    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 17 07:52:09.309393 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307010    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307013    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307015    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307018    2571 feature_gate.go:328] unrecognized feature gate: Example
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307021    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307024    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307027    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307029    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307032    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307035    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307038    2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307041    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307044    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307046    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307049    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307051    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307054    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307057    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307060    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 17 07:52:09.309878 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307062    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307065    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307068    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307071    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307075    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307079    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307081    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307084    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307087    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307090    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307093    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307095    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307098    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307100    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307103    2571 feature_gate.go:328] unrecognized feature gate: Example2
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307105    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307108    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 17 07:52:09.310347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307110    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.307115    2571 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307219    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307224    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307227    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307231    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307234    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307237    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307239    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307242    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307245    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307247    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307250    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307252    2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307255    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307257    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Apr 17 07:52:09.310775 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307260    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307262    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307264    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307267    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307269    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307273    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307276    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307278    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307281    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307283    2571 feature_gate.go:328] unrecognized feature gate: Example2
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307286    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307288    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307291    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307293    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307296    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307298    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307300    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307304    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307307    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Apr 17 07:52:09.311207 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307309    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307313    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307315    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307318    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307321    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307325    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307328    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307331    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307334    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307337    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307340    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307343    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307345    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307348    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307350    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307353    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307355    2571 feature_gate.go:328] unrecognized feature gate: Example
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307358    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307360    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307363    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Apr 17 07:52:09.311690 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307366    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307368    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307371    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307373    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307376    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307378    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307381    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307383    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307386    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307388    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307391    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307394    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307396    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307399    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307401    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307404    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307406    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307409    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307412    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Apr 17 07:52:09.312165 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307414    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307417    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307419    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307422    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307424    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307427    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307450    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307454    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307458    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307461    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307463    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307466    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307468    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:09.307471    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.307476    2571 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Apr 17 07:52:09.312658 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.307600    2571 server.go:962] "Client rotation is on, will bootstrap in background"
Apr 17 07:52:09.313031 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.310573    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Apr 17 07:52:09.313031 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.311590    2571 server.go:1019] "Starting client certificate rotation"
Apr 17 07:52:09.313031 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.311681    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Apr 17 07:52:09.313031 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.311721    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Apr 17 07:52:09.338780 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.338758    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Apr 17 07:52:09.341375 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.341351    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Apr 17 07:52:09.356998 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.356969    2571 log.go:25] "Validated CRI v1 runtime API"
Apr 17 07:52:09.363133 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.363112    2571 log.go:25] "Validated CRI v1 image API"
Apr 17 07:52:09.364416 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.364393    2571 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Apr 17 07:52:09.367306 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.367289    2571 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Apr 17 07:52:09.370369 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.370347    2571 fs.go:135] Filesystem UUIDs: map[4d09b83d-7aeb-4d66-98ec-10b0ae679f4b:/dev/nvme0n1p4 5ee61c3b-eeb4-43de-ba6f-c164a2094e1e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Apr 17 07:52:09.370446 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.370369    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Apr 17 07:52:09.376337 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.376230    2571 manager.go:217] Machine: {Timestamp:2026-04-17 07:52:09.374030362 +0000 UTC m=+0.427633584 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3088254 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b3979342abc4c980583ecefdb3642 SystemUUID:ec2b3979-342a-bc4c-9805-83ecefdb3642 BootID:5320201a-cdcf-4ad8-b273-b7778bdf84b8 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2c:ea:89:3a:31 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2c:ea:89:3a:31 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:85:22:b3:2a:c9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Apr 17 07:52:09.376337 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.376330    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Apr 17 07:52:09.376460 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.376415    2571 manager.go:233] Version: {KernelVersion:5.14.0-570.107.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260414-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Apr 17 07:52:09.377556 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.377533    2571 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Apr 17 07:52:09.377699 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.377558    2571 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-45.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Apr 17 07:52:09.377748 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.377708    2571 topology_manager.go:138] "Creating topology manager with none policy"
Apr 17 07:52:09.377748 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.377717    2571 container_manager_linux.go:306] "Creating device plugin manager"
Apr 17 07:52:09.377748 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.377730    2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Apr 17 07:52:09.378549 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.378538    2571 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Apr 17 07:52:09.379891 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.379881    2571 state_mem.go:36] "Initialized new in-memory state store"
Apr 17 07:52:09.380005 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.379991    2571 server.go:1267] "Using root directory" path="/var/lib/kubelet"
Apr 17 07:52:09.382521 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.382512    2571 kubelet.go:491] "Attempting to sync node with API server"
Apr 17 07:52:09.382562 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.382529    2571 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
Apr 17 07:52:09.382562 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.382542    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Apr 17 07:52:09.382562 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.382551    2571 kubelet.go:397] "Adding apiserver pod source"
Apr 17 07:52:09.382562 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.382561    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Apr 17 07:52:09.383640 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.383629    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Apr 17 07:52:09.383683 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.383648    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Apr 17 07:52:09.384930 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.384910    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h8g2q"
Apr 17 07:52:09.386803 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.386790    2571 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.10-2.rhaos4.20.gita4d0894.el9" apiVersion="v1"
Apr 17 07:52:09.388264 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.388251    2571 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Apr 17 07:52:09.390096 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.390081    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Apr 17 07:52:09.390149 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.390118    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Apr 17 07:52:09.390149 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.390128    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Apr 17 07:52:09.390212 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.390137    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Apr 17 07:52:09.390212 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.390190    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Apr 17 07:52:09.390212 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.390200    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Apr 17 07:52:09.390212 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.390208    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Apr 17 07:52:09.390317 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.390217    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Apr 17 07:52:09.390317 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.390228    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Apr 17 07:52:09.390317 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.390237    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Apr 17 07:52:09.390317 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.390250    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Apr 17 07:52:09.390317 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.390263    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Apr 17 07:52:09.391344 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.391332    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Apr 17 07:52:09.391391 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.391345    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
Apr 17 07:52:09.391546 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.391524    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h8g2q"
Apr 17 07:52:09.395521 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.395506    2571 watchdog_linux.go:99] "Systemd watchdog is not enabled"
Apr 17 07:52:09.395620 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.395542    2571 server.go:1295] "Started kubelet"
Apr 17 07:52:09.395733 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.395667    2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Apr 17 07:52:09.396119 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.395694    2571 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Apr 17 07:52:09.396379 ip-10-0-143-45 systemd[1]: Started Kubernetes Kubelet.
Apr 17 07:52:09.399036 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.399009    2571 server_v1.go:47] "podresources" method="list" useActivePods=true
Apr 17 07:52:09.399936 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.399915    2571 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 17 07:52:09.400969 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.400950    2571 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Apr 17 07:52:09.405760 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.405673    2571 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 17 07:52:09.405760 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.405706    2571 server.go:317] "Adding debug handlers to kubelet server"
Apr 17 07:52:09.405974 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.405958    2571 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-45.ec2.internal" not found
Apr 17 07:52:09.410325 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:09.410305    2571 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Apr 17 07:52:09.416032 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.416014    2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Apr 17 07:52:09.416032 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.416023    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Apr 17 07:52:09.416951 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.416833    2571 volume_manager.go:295] "The desired_state_of_world populator starts"
Apr 17 07:52:09.416951 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.416866    2571 volume_manager.go:297] "Starting Kubelet Volume Manager"
Apr 17 07:52:09.417054 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.416964    2571 reconstruct.go:97] "Volume reconstruction finished"
Apr 17 07:52:09.417054 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.416975    2571 reconciler.go:26] "Reconciler: start to sync state"
Apr 17 07:52:09.417054 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.417018    2571 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
Apr 17 07:52:09.417255 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:09.417158    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-45.ec2.internal\" not found"
Apr 17 07:52:09.417535 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.417512    2571 factory.go:153] Registering CRI-O factory
Apr 17 07:52:09.417535 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.417536    2571 factory.go:223] Registration of the crio container factory successfully
Apr 17 07:52:09.417642 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.417590    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Apr 17 07:52:09.417642 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.417599    2571 factory.go:55] Registering systemd factory
Apr 17 07:52:09.417642 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.417607    2571 factory.go:223] Registration of the systemd container factory successfully
Apr 17 07:52:09.417642 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.417631    2571 factory.go:103] Registering Raw factory
Apr 17 07:52:09.417797 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.417645    2571 manager.go:1196] Started watching for new ooms in manager
Apr 17 07:52:09.418280 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.418264    2571 manager.go:319] Starting recovery of all containers
Apr 17 07:52:09.418742 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.418722    2571 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 17 07:52:09.420742 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:09.420710    2571 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-45.ec2.internal\" not found" node="ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.421114 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.421098    2571 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-45.ec2.internal" not found
Apr 17 07:52:09.428398 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.428248    2571 manager.go:324] Recovery completed
Apr 17 07:52:09.433869 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.433853    2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 17 07:52:09.435892 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.435876    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-45.ec2.internal" event="NodeHasSufficientMemory"
Apr 17 07:52:09.435976 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.435905    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-45.ec2.internal" event="NodeHasNoDiskPressure"
Apr 17 07:52:09.435976 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.435919    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-45.ec2.internal" event="NodeHasSufficientPID"
Apr 17 07:52:09.436455 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.436443    2571 cpu_manager.go:222] "Starting CPU manager" policy="none"
Apr 17 07:52:09.436508 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.436455    2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Apr 17 07:52:09.436508 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.436475    2571 state_mem.go:36] "Initialized new in-memory state store"
Apr 17 07:52:09.438646 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.438634    2571 policy_none.go:49] "None policy: Start"
Apr 17 07:52:09.438698 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.438650    2571 memory_manager.go:186] "Starting memorymanager" policy="None"
Apr 17 07:52:09.438698 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.438660    2571 state_mem.go:35] "Initializing new in-memory state store"
Apr 17 07:52:09.488384 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.476987    2571 manager.go:341] "Starting Device Plugin manager"
Apr 17 07:52:09.488384 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:09.477017    2571 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Apr 17 07:52:09.488384 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.477026    2571 server.go:85] "Starting device plugin registration server"
Apr 17 07:52:09.488384 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.477261    2571 eviction_manager.go:189] "Eviction manager: starting control loop"
Apr 17 07:52:09.488384 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.477271    2571 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Apr 17 07:52:09.488384 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.477382    2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Apr 17 07:52:09.488384 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.477486    2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Apr 17 07:52:09.488384 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.477494    2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Apr 17 07:52:09.488384 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:09.477993    2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Apr 17 07:52:09.488384 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:09.478029    2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-45.ec2.internal\" not found"
Apr 17 07:52:09.488384 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.482416    2571 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-45.ec2.internal" not found
Apr 17 07:52:09.536742 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.536704    2571 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
Apr 17 07:52:09.537938 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.537917    2571 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
Apr 17 07:52:09.538041 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.537945    2571 status_manager.go:230] "Starting to sync pod status with apiserver"
Apr 17 07:52:09.538041 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.537965    2571 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Apr 17 07:52:09.538041 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.537972    2571 kubelet.go:2451] "Starting kubelet main sync loop"
Apr 17 07:52:09.538041 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:09.538004    2571 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Apr 17 07:52:09.540153 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.540123    2571 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 17 07:52:09.578058 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.577972    2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Apr 17 07:52:09.578977 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.578957    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-45.ec2.internal" event="NodeHasSufficientMemory"
Apr 17 07:52:09.579075 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.578992    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-45.ec2.internal" event="NodeHasNoDiskPressure"
Apr 17 07:52:09.579075 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.579007    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-45.ec2.internal" event="NodeHasSufficientPID"
Apr 17 07:52:09.579075 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.579038    2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.591020 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.590994    2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.638678 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.638645    2571 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-45.ec2.internal"]
Apr 17 07:52:09.643301 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.643271    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.643301 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.643286    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.661012 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.660990    2571 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.664919 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.664899    2571 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.718018 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.717992    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4b6643937e58b8d1200745e1d581977f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal\" (UID: \"4b6643937e58b8d1200745e1d581977f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.766177 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.766151    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Apr 17 07:52:09.766316 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.766155    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Apr 17 07:52:09.818532 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.818496    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b6643937e58b8d1200745e1d581977f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal\" (UID: \"4b6643937e58b8d1200745e1d581977f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.818532 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.818536    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd84846842e0dd9650476cd580a9ac6c-config\") pod \"kube-apiserver-proxy-ip-10-0-143-45.ec2.internal\" (UID: \"bd84846842e0dd9650476cd580a9ac6c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.818673 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.818595    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4b6643937e58b8d1200745e1d581977f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal\" (UID: \"4b6643937e58b8d1200745e1d581977f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.818673 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.818635    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4b6643937e58b8d1200745e1d581977f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal\" (UID: \"4b6643937e58b8d1200745e1d581977f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.918953 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.918924    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b6643937e58b8d1200745e1d581977f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal\" (UID: \"4b6643937e58b8d1200745e1d581977f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.919027 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.918963    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd84846842e0dd9650476cd580a9ac6c-config\") pod \"kube-apiserver-proxy-ip-10-0-143-45.ec2.internal\" (UID: \"bd84846842e0dd9650476cd580a9ac6c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.919027 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.919003    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd84846842e0dd9650476cd580a9ac6c-config\") pod \"kube-apiserver-proxy-ip-10-0-143-45.ec2.internal\" (UID: \"bd84846842e0dd9650476cd580a9ac6c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:09.919092 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:09.919041    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b6643937e58b8d1200745e1d581977f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal\" (UID: \"4b6643937e58b8d1200745e1d581977f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:10.069382 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.069340    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:10.069543 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.069459    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal"
Apr 17 07:52:10.311881 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.311849    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Apr 17 07:52:10.312525 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.312050    2571 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Apr 17 07:52:10.312525 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.312053    2571 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Apr 17 07:52:10.312525 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.312056    2571 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Apr 17 07:52:10.382758 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.382727    2571 apiserver.go:52] "Watching apiserver"
Apr 17 07:52:10.390037 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.390014    2571 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Apr 17 07:52:10.391006 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.390986    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-45.ec2.internal","openshift-dns/node-resolver-vcnrb","openshift-image-registry/node-ca-27pc9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal","openshift-multus/network-metrics-daemon-wxkt9","openshift-network-diagnostics/network-check-target-r78gk","openshift-network-operator/iptables-alerter-8qgq7","openshift-ovn-kubernetes/ovnkube-node-krj9p","kube-system/konnectivity-agent-v2g9s","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c","openshift-cluster-node-tuning-operator/tuned-s5skc","openshift-multus/multus-additional-cni-plugins-8jsdp","openshift-multus/multus-v5cdv"]
Apr 17 07:52:10.393047 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.393019    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-04-16 07:47:09 +0000 UTC" deadline="2027-11-13 03:42:36.226160964 +0000 UTC"
Apr 17 07:52:10.393102 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.393046    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13795h50m25.833117149s"
Apr 17 07:52:10.396163 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.395978    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vcnrb"
Apr 17 07:52:10.396163 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.396018    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-27pc9"
Apr 17 07:52:10.398556 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.398534    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:10.398556 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.398550    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Apr 17 07:52:10.398707 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:10.398596    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:10.398707 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.398535    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hsx9w\""
Apr 17 07:52:10.398707 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.398654    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tmwwc\""
Apr 17 07:52:10.398923 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.398900    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Apr 17 07:52:10.398923 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.398915    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Apr 17 07:52:10.399189 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.399176    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Apr 17 07:52:10.399189 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.399183    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Apr 17 07:52:10.402903 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.402878    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:10.402994 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.402940    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8qgq7"
Apr 17 07:52:10.402994 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:10.402959    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:10.405119 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.405102    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Apr 17 07:52:10.405191 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.405123    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Apr 17 07:52:10.405287 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.405273    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.405374 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.405359    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xmqmw\""
Apr 17 07:52:10.405499 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.405478    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Apr 17 07:52:10.407418 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.407402    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Apr 17 07:52:10.407541 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.407522    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v2g9s"
Apr 17 07:52:10.407609 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.407538    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Apr 17 07:52:10.407609 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.407564    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Apr 17 07:52:10.407609 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.407534    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Apr 17 07:52:10.407760 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.407745    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Apr 17 07:52:10.408338 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.408323    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Apr 17 07:52:10.408417 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.408392    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c9jv7\""
Apr 17 07:52:10.409766 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.409743    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Apr 17 07:52:10.409847 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.409770    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Apr 17 07:52:10.409847 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.409745    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mxk9b\""
Apr 17 07:52:10.409847 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.409811    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.411800 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.411762    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Apr 17 07:52:10.411930 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.411913    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Apr 17 07:52:10.412000 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.411942    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Apr 17 07:52:10.412000 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.411997    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qxnxl\""
Apr 17 07:52:10.412216 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.412201    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.414196 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.414180    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sdr4g\""
Apr 17 07:52:10.414580 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.414559    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Apr 17 07:52:10.414677 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.414573    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.414730 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.414712    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Apr 17 07:52:10.416184 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.416142    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Apr 17 07:52:10.417249 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.416936    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Apr 17 07:52:10.417249 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.417005    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Apr 17 07:52:10.417249 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.416937    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Apr 17 07:52:10.417249 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.417138    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pqf5d\""
Apr 17 07:52:10.417682 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.417526    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.417682 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.417612    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Apr 17 07:52:10.417950 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.417935    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Apr 17 07:52:10.419661 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.419642    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Apr 17 07:52:10.419758 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.419740    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cqlwn\""
Apr 17 07:52:10.422011 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.421970    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-run-ovn-kubernetes\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.422120 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422014    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-run-openvswitch\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.422120 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422045    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-sys\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.422120 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422069    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4txqz\" (UniqueName: \"kubernetes.io/projected/accd983c-5d20-4f58-9453-9dd0a312b13b-kube-api-access-4txqz\") pod \"node-resolver-vcnrb\" (UID: \"accd983c-5d20-4f58-9453-9dd0a312b13b\") " pod="openshift-dns/node-resolver-vcnrb"
Apr 17 07:52:10.422120 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422101    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/99ac4cf1-ceb9-44bd-9ab5-ad7588da1738-konnectivity-ca\") pod \"konnectivity-agent-v2g9s\" (UID: \"99ac4cf1-ceb9-44bd-9ab5-ad7588da1738\") " pod="kube-system/konnectivity-agent-v2g9s"
Apr 17 07:52:10.422288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422130    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-slash\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.422288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422159    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-run-systemd\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.422288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422189    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-run-ovn\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.422288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422218    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-run\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.422288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422265    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff540440-d950-4564-be29-fd2a0bdc83d8-iptables-alerter-script\") pod \"iptables-alerter-8qgq7\" (UID: \"ff540440-d950-4564-be29-fd2a0bdc83d8\") " pod="openshift-network-operator/iptables-alerter-8qgq7"
Apr 17 07:52:10.422523 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422293    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkqpn\" (UniqueName: \"kubernetes.io/projected/268375f5-3f20-4e84-9af0-286587a445c7-kube-api-access-lkqpn\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.422523 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422321    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/161f0c24-f3ae-495e-a9db-213c81687526-system-cni-dir\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.422523 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422410    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-cnibin\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.422703 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422541    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-run-k8s-cni-cncf-io\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.422703 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422621    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/268375f5-3f20-4e84-9af0-286587a445c7-multus-daemon-config\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.422774 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422708    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8tqn\" (UniqueName: \"kubernetes.io/projected/e65a3e99-bd0c-4035-b41f-db6915c67e9a-kube-api-access-x8tqn\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.422809 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.422777    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-run-multus-certs\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.423145 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423124    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-etc-openvswitch\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.423184 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423166    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e65a3e99-bd0c-4035-b41f-db6915c67e9a-ovnkube-script-lib\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.423219 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423184    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-sysctl-d\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.423219 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423198    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-lib-modules\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.423219 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423213    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3263f5a7-519d-490a-8444-599d2f8159d0-etc-tuned\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.423332 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423228    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/99ac4cf1-ceb9-44bd-9ab5-ad7588da1738-agent-certs\") pod \"konnectivity-agent-v2g9s\" (UID: \"99ac4cf1-ceb9-44bd-9ab5-ad7588da1738\") " pod="kube-system/konnectivity-agent-v2g9s"
Apr 17 07:52:10.423332 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423260    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-node-log\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.423332 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423284    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e65a3e99-bd0c-4035-b41f-db6915c67e9a-ovn-node-metrics-cert\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.423332 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423314    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-sys-fs\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.423484 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423332    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff540440-d950-4564-be29-fd2a0bdc83d8-host-slash\") pod \"iptables-alerter-8qgq7\" (UID: \"ff540440-d950-4564-be29-fd2a0bdc83d8\") " pod="openshift-network-operator/iptables-alerter-8qgq7"
Apr 17 07:52:10.423484 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423347    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/161f0c24-f3ae-495e-a9db-213c81687526-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.423484 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423379    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cpxk\" (UniqueName: \"kubernetes.io/projected/6f91ec65-f7dd-4657-838b-318538310ead-kube-api-access-6cpxk\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:10.423484 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423398    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjmc4\" (UniqueName: \"kubernetes.io/projected/ff540440-d950-4564-be29-fd2a0bdc83d8-kube-api-access-hjmc4\") pod \"iptables-alerter-8qgq7\" (UID: \"ff540440-d950-4564-be29-fd2a0bdc83d8\") " pod="openshift-network-operator/iptables-alerter-8qgq7"
Apr 17 07:52:10.423484 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423413    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-system-cni-dir\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.423484 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423427    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-cni-netd\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.423484 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423478    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-var-lib-kubelet\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.423701 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423511    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/161f0c24-f3ae-495e-a9db-213c81687526-cni-binary-copy\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.423701 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423532    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/268375f5-3f20-4e84-9af0-286587a445c7-cni-binary-copy\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.423701 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423549    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-var-lib-kubelet\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.423701 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423562    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-etc-kubernetes\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.423701 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423580    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-kubernetes\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.423701 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423606    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-sysctl-conf\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.423701 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423627    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3263f5a7-519d-490a-8444-599d2f8159d0-tmp\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.423701 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423648    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-os-release\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.423701 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423666    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-log-socket\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.423701 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423688    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-cni-bin\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423706    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-registration-dir\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423720    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wqkp\" (UniqueName: \"kubernetes.io/projected/41c98b12-f6cc-4300-ba04-7c5a891f74eb-kube-api-access-9wqkp\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423733    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-sysconfig\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423752    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/accd983c-5d20-4f58-9453-9dd0a312b13b-tmp-dir\") pod \"node-resolver-vcnrb\" (UID: \"accd983c-5d20-4f58-9453-9dd0a312b13b\") " pod="openshift-dns/node-resolver-vcnrb"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423782    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/161f0c24-f3ae-495e-a9db-213c81687526-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423808    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-multus-socket-dir-parent\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423832    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-var-lib-cni-bin\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423847    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-kubelet\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423860    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmcks\" (UniqueName: \"kubernetes.io/projected/07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32-kube-api-access-wmcks\") pod \"node-ca-27pc9\" (UID: \"07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32\") " pod="openshift-image-registry/node-ca-27pc9"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423874    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-hostroot\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423890    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423905    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32-host\") pod \"node-ca-27pc9\" (UID: \"07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32\") " pod="openshift-image-registry/node-ca-27pc9"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423919    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32-serviceca\") pod \"node-ca-27pc9\" (UID: \"07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32\") " pod="openshift-image-registry/node-ca-27pc9"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423932    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/161f0c24-f3ae-495e-a9db-213c81687526-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423946    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-multus-cni-dir\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.423992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423958    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-multus-conf-dir\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.423990    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2nwx\" (UniqueName: \"kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx\") pod \"network-check-target-r78gk\" (UID: \"bde75c6d-e1a4-4848-bef6-94161be7ed61\") " pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424013    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-run-netns\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424029    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e65a3e99-bd0c-4035-b41f-db6915c67e9a-ovnkube-config\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424043    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-socket-dir\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424057    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-modprobe-d\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424079    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-systemd-units\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424103    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e65a3e99-bd0c-4035-b41f-db6915c67e9a-env-overrides\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424117    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-systemd\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424142    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-host\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424169    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh6sq\" (UniqueName: \"kubernetes.io/projected/3263f5a7-519d-490a-8444-599d2f8159d0-kube-api-access-mh6sq\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424183    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/accd983c-5d20-4f58-9453-9dd0a312b13b-hosts-file\") pod \"node-resolver-vcnrb\" (UID: \"accd983c-5d20-4f58-9453-9dd0a312b13b\") " pod="openshift-dns/node-resolver-vcnrb"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424196    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424215    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-device-dir\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424230    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-etc-selinux\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424246    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/161f0c24-f3ae-495e-a9db-213c81687526-os-release\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.424483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424260    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c89ld\" (UniqueName: \"kubernetes.io/projected/161f0c24-f3ae-495e-a9db-213c81687526-kube-api-access-c89ld\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.424922 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424275    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-run-netns\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.424922 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424303    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-var-lib-openvswitch\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.424922 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424324    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/161f0c24-f3ae-495e-a9db-213c81687526-cnibin\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.424922 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424339    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-var-lib-cni-multus\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.424922 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.424374    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:10.428767 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.428752    2571 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Apr 17 07:52:10.447740 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.447718    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-d6gt6"
Apr 17 07:52:10.456196 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.456176    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-d6gt6"
Apr 17 07:52:10.517716 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.517698    2571 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
Apr 17 07:52:10.525377 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525357    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:10.525560 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525384    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-run-ovn-kubernetes\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.525560 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525400    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-run-openvswitch\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.525560 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525414    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-sys\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.525560 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525474    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-run-openvswitch\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.525560 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525501    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-run-ovn-kubernetes\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.525560 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525510    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4txqz\" (UniqueName: \"kubernetes.io/projected/accd983c-5d20-4f58-9453-9dd0a312b13b-kube-api-access-4txqz\") pod \"node-resolver-vcnrb\" (UID: \"accd983c-5d20-4f58-9453-9dd0a312b13b\") " pod="openshift-dns/node-resolver-vcnrb"
Apr 17 07:52:10.525560 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:10.525539    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:10.525560 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525551    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-sys\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.525883 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525599    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/99ac4cf1-ceb9-44bd-9ab5-ad7588da1738-konnectivity-ca\") pod \"konnectivity-agent-v2g9s\" (UID: \"99ac4cf1-ceb9-44bd-9ab5-ad7588da1738\") " pod="kube-system/konnectivity-agent-v2g9s"
Apr 17 07:52:10.525883 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:10.525642    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs podName:6f91ec65-f7dd-4657-838b-318538310ead nodeName:}" failed. No retries permitted until 2026-04-17 07:52:11.025591672 +0000 UTC m=+2.079194875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs") pod "network-metrics-daemon-wxkt9" (UID: "6f91ec65-f7dd-4657-838b-318538310ead") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:10.525883 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525675    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-slash\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.525883 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525710    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-run-systemd\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.525883 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525736    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-run-ovn\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.525883 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525734    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-slash\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.525883 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525757    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-run\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.525883 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525768    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-run-systemd\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.525883 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525801    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff540440-d950-4564-be29-fd2a0bdc83d8-iptables-alerter-script\") pod \"iptables-alerter-8qgq7\" (UID: \"ff540440-d950-4564-be29-fd2a0bdc83d8\") " pod="openshift-network-operator/iptables-alerter-8qgq7"
Apr 17 07:52:10.525883 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525834    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lkqpn\" (UniqueName: \"kubernetes.io/projected/268375f5-3f20-4e84-9af0-286587a445c7-kube-api-access-lkqpn\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.525883 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525854    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/161f0c24-f3ae-495e-a9db-213c81687526-system-cni-dir\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.525883 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525870    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-cnibin\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.525883 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525843    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-run-ovn\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525891    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-run-k8s-cni-cncf-io\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525951    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/161f0c24-f3ae-495e-a9db-213c81687526-system-cni-dir\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525959    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-run\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.525979    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-run-k8s-cni-cncf-io\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526002    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-cnibin\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526027    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/268375f5-3f20-4e84-9af0-286587a445c7-multus-daemon-config\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526071    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x8tqn\" (UniqueName: \"kubernetes.io/projected/e65a3e99-bd0c-4035-b41f-db6915c67e9a-kube-api-access-x8tqn\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526088    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-run-multus-certs\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526108    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-etc-openvswitch\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526124    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-run-multus-certs\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526132    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e65a3e99-bd0c-4035-b41f-db6915c67e9a-ovnkube-script-lib\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526148    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-sysctl-d\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526157    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-etc-openvswitch\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526163    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-lib-modules\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526183    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3263f5a7-519d-490a-8444-599d2f8159d0-etc-tuned\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526213    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/99ac4cf1-ceb9-44bd-9ab5-ad7588da1738-agent-certs\") pod \"konnectivity-agent-v2g9s\" (UID: \"99ac4cf1-ceb9-44bd-9ab5-ad7588da1738\") " pod="kube-system/konnectivity-agent-v2g9s"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526238    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-node-log\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.526493 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526261    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e65a3e99-bd0c-4035-b41f-db6915c67e9a-ovn-node-metrics-cert\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526286    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-sys-fs\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526297    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-sysctl-d\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526315    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff540440-d950-4564-be29-fd2a0bdc83d8-host-slash\") pod \"iptables-alerter-8qgq7\" (UID: \"ff540440-d950-4564-be29-fd2a0bdc83d8\") " pod="openshift-network-operator/iptables-alerter-8qgq7"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526321    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-lib-modules\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526359    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/161f0c24-f3ae-495e-a9db-213c81687526-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526373    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff540440-d950-4564-be29-fd2a0bdc83d8-host-slash\") pod \"iptables-alerter-8qgq7\" (UID: \"ff540440-d950-4564-be29-fd2a0bdc83d8\") " pod="openshift-network-operator/iptables-alerter-8qgq7"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526388    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6cpxk\" (UniqueName: \"kubernetes.io/projected/6f91ec65-f7dd-4657-838b-318538310ead-kube-api-access-6cpxk\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526417    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hjmc4\" (UniqueName: \"kubernetes.io/projected/ff540440-d950-4564-be29-fd2a0bdc83d8-kube-api-access-hjmc4\") pod \"iptables-alerter-8qgq7\" (UID: \"ff540440-d950-4564-be29-fd2a0bdc83d8\") " pod="openshift-network-operator/iptables-alerter-8qgq7"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526413    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-node-log\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526457    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-system-cni-dir\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526498    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-cni-netd\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526523    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-var-lib-kubelet\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526557    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/161f0c24-f3ae-495e-a9db-213c81687526-cni-binary-copy\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526585    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/268375f5-3f20-4e84-9af0-286587a445c7-cni-binary-copy\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526633    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-var-lib-kubelet\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526702    2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526721    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-sys-fs\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.527252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526770    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-var-lib-kubelet\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526948    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-system-cni-dir\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.526976    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff540440-d950-4564-be29-fd2a0bdc83d8-iptables-alerter-script\") pod \"iptables-alerter-8qgq7\" (UID: \"ff540440-d950-4564-be29-fd2a0bdc83d8\") " pod="openshift-network-operator/iptables-alerter-8qgq7"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527012    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-cni-netd\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527050    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-etc-kubernetes\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527082    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-var-lib-kubelet\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527086    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/99ac4cf1-ceb9-44bd-9ab5-ad7588da1738-konnectivity-ca\") pod \"konnectivity-agent-v2g9s\" (UID: \"99ac4cf1-ceb9-44bd-9ab5-ad7588da1738\") " pod="kube-system/konnectivity-agent-v2g9s"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527084    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-kubernetes\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527178    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-etc-kubernetes\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527270    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/161f0c24-f3ae-495e-a9db-213c81687526-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527337    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-sysctl-conf\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527382    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-kubernetes\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527459    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3263f5a7-519d-490a-8444-599d2f8159d0-tmp\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527477    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-sysctl-conf\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527512    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-os-release\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527542    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-log-socket\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527581    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-cni-bin\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527605    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-registration-dir\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.528097 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527629    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9wqkp\" (UniqueName: \"kubernetes.io/projected/41c98b12-f6cc-4300-ba04-7c5a891f74eb-kube-api-access-9wqkp\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527652    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-sysconfig\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527690    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/accd983c-5d20-4f58-9453-9dd0a312b13b-tmp-dir\") pod \"node-resolver-vcnrb\" (UID: \"accd983c-5d20-4f58-9453-9dd0a312b13b\") " pod="openshift-dns/node-resolver-vcnrb"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527715    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/161f0c24-f3ae-495e-a9db-213c81687526-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527749    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-multus-socket-dir-parent\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527815    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-var-lib-cni-bin\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527856    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e65a3e99-bd0c-4035-b41f-db6915c67e9a-ovnkube-script-lib\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527838    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-kubelet\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527923    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/161f0c24-f3ae-495e-a9db-213c81687526-cni-binary-copy\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527953    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wmcks\" (UniqueName: \"kubernetes.io/projected/07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32-kube-api-access-wmcks\") pod \"node-ca-27pc9\" (UID: \"07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32\") " pod="openshift-image-registry/node-ca-27pc9"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.527999    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-hostroot\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528034    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-kubelet\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528028    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528110    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32-host\") pod \"node-ca-27pc9\" (UID: \"07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32\") " pod="openshift-image-registry/node-ca-27pc9"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528138    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32-serviceca\") pod \"node-ca-27pc9\" (UID: \"07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32\") " pod="openshift-image-registry/node-ca-27pc9"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528148    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-sysconfig\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528170    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/161f0c24-f3ae-495e-a9db-213c81687526-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.529030 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528210    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-multus-cni-dir\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528233    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-multus-conf-dir\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528259    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k2nwx\" (UniqueName: \"kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx\") pod \"network-check-target-r78gk\" (UID: \"bde75c6d-e1a4-4848-bef6-94161be7ed61\") " pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528284    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-run-netns\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528310    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e65a3e99-bd0c-4035-b41f-db6915c67e9a-ovnkube-config\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528359    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-socket-dir\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528493    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-modprobe-d\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528531    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-systemd-units\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528530    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528558    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e65a3e99-bd0c-4035-b41f-db6915c67e9a-env-overrides\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528593    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-systemd\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528611    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-hostroot\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528619    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-host\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528652    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/accd983c-5d20-4f58-9453-9dd0a312b13b-tmp-dir\") pod \"node-resolver-vcnrb\" (UID: \"accd983c-5d20-4f58-9453-9dd0a312b13b\") " pod="openshift-dns/node-resolver-vcnrb"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528655    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mh6sq\" (UniqueName: \"kubernetes.io/projected/3263f5a7-519d-490a-8444-599d2f8159d0-kube-api-access-mh6sq\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528731    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/accd983c-5d20-4f58-9453-9dd0a312b13b-hosts-file\") pod \"node-resolver-vcnrb\" (UID: \"accd983c-5d20-4f58-9453-9dd0a312b13b\") " pod="openshift-dns/node-resolver-vcnrb"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528761    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.529871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528786    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-device-dir\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528907    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-etc-selinux\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528946    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/161f0c24-f3ae-495e-a9db-213c81687526-os-release\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.528982    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c89ld\" (UniqueName: \"kubernetes.io/projected/161f0c24-f3ae-495e-a9db-213c81687526-kube-api-access-c89ld\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529008    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-run-netns\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529066    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-var-lib-openvswitch\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529093    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/161f0c24-f3ae-495e-a9db-213c81687526-cnibin\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529120    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-var-lib-cni-multus\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529188    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/268375f5-3f20-4e84-9af0-286587a445c7-multus-daemon-config\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529234    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-var-lib-cni-multus\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529296    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32-serviceca\") pod \"node-ca-27pc9\" (UID: \"07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32\") " pod="openshift-image-registry/node-ca-27pc9"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529321    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/accd983c-5d20-4f58-9453-9dd0a312b13b-hosts-file\") pod \"node-resolver-vcnrb\" (UID: \"accd983c-5d20-4f58-9453-9dd0a312b13b\") " pod="openshift-dns/node-resolver-vcnrb"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529366    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529400    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-multus-conf-dir\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529414    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-device-dir\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529489    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e65a3e99-bd0c-4035-b41f-db6915c67e9a-ovnkube-config\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529624    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-multus-cni-dir\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529674    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32-host\") pod \"node-ca-27pc9\" (UID: \"07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32\") " pod="openshift-image-registry/node-ca-27pc9"
Apr 17 07:52:10.530650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529719    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-run-netns\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529790    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/161f0c24-f3ae-495e-a9db-213c81687526-os-release\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529799    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-multus-socket-dir-parent\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529842    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/161f0c24-f3ae-495e-a9db-213c81687526-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529852    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-host-cni-bin\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529845    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-systemd-units\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529903    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-registration-dir\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529947    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-log-socket\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529959    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-systemd\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529984    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e65a3e99-bd0c-4035-b41f-db6915c67e9a-var-lib-openvswitch\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.530035    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/161f0c24-f3ae-495e-a9db-213c81687526-cnibin\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.530112    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-etc-selinux\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.530188    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-os-release\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.530273    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/41c98b12-f6cc-4300-ba04-7c5a891f74eb-socket-dir\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.529904    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-run-netns\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.530297    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-etc-modprobe-d\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.530343    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3263f5a7-519d-490a-8444-599d2f8159d0-host\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.530373    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e65a3e99-bd0c-4035-b41f-db6915c67e9a-env-overrides\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.531373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.530515    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/268375f5-3f20-4e84-9af0-286587a445c7-host-var-lib-cni-bin\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.532180 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.531471    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3263f5a7-519d-490a-8444-599d2f8159d0-tmp\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.532180 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.531474    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e65a3e99-bd0c-4035-b41f-db6915c67e9a-ovn-node-metrics-cert\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.532180 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.531556    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/161f0c24-f3ae-495e-a9db-213c81687526-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.532180 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.531565    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/99ac4cf1-ceb9-44bd-9ab5-ad7588da1738-agent-certs\") pod \"konnectivity-agent-v2g9s\" (UID: \"99ac4cf1-ceb9-44bd-9ab5-ad7588da1738\") " pod="kube-system/konnectivity-agent-v2g9s"
Apr 17 07:52:10.532180 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.531770    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3263f5a7-519d-490a-8444-599d2f8159d0-etc-tuned\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.534141 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.532492    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/268375f5-3f20-4e84-9af0-286587a445c7-cni-binary-copy\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.539289 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:10.539261    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 17 07:52:10.539378 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:10.539292    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 17 07:52:10.539378 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:10.539304    2571 projected.go:194] Error preparing data for projected volume kube-api-access-k2nwx for pod openshift-network-diagnostics/network-check-target-r78gk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:10.539521 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.539426    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjmc4\" (UniqueName: \"kubernetes.io/projected/ff540440-d950-4564-be29-fd2a0bdc83d8-kube-api-access-hjmc4\") pod \"iptables-alerter-8qgq7\" (UID: \"ff540440-d950-4564-be29-fd2a0bdc83d8\") " pod="openshift-network-operator/iptables-alerter-8qgq7"
Apr 17 07:52:10.539583 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.539525    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4txqz\" (UniqueName: \"kubernetes.io/projected/accd983c-5d20-4f58-9453-9dd0a312b13b-kube-api-access-4txqz\") pod \"node-resolver-vcnrb\" (UID: \"accd983c-5d20-4f58-9453-9dd0a312b13b\") " pod="openshift-dns/node-resolver-vcnrb"
Apr 17 07:52:10.539750 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:10.539733    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx podName:bde75c6d-e1a4-4848-bef6-94161be7ed61 nodeName:}" failed. No retries permitted until 2026-04-17 07:52:11.039709048 +0000 UTC m=+2.093312270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k2nwx" (UniqueName: "kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx") pod "network-check-target-r78gk" (UID: "bde75c6d-e1a4-4848-bef6-94161be7ed61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:10.539939 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.539917    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkqpn\" (UniqueName: \"kubernetes.io/projected/268375f5-3f20-4e84-9af0-286587a445c7-kube-api-access-lkqpn\") pod \"multus-v5cdv\" (UID: \"268375f5-3f20-4e84-9af0-286587a445c7\") " pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.540140 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.540116    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8tqn\" (UniqueName: \"kubernetes.io/projected/e65a3e99-bd0c-4035-b41f-db6915c67e9a-kube-api-access-x8tqn\") pod \"ovnkube-node-krj9p\" (UID: \"e65a3e99-bd0c-4035-b41f-db6915c67e9a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.540520 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.540497    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh6sq\" (UniqueName: \"kubernetes.io/projected/3263f5a7-519d-490a-8444-599d2f8159d0-kube-api-access-mh6sq\") pod \"tuned-s5skc\" (UID: \"3263f5a7-519d-490a-8444-599d2f8159d0\") " pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.541544 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.541525    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cpxk\" (UniqueName: \"kubernetes.io/projected/6f91ec65-f7dd-4657-838b-318538310ead-kube-api-access-6cpxk\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:10.542123 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.542099    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c89ld\" (UniqueName: \"kubernetes.io/projected/161f0c24-f3ae-495e-a9db-213c81687526-kube-api-access-c89ld\") pod \"multus-additional-cni-plugins-8jsdp\" (UID: \"161f0c24-f3ae-495e-a9db-213c81687526\") " pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.542190 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.542126    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wqkp\" (UniqueName: \"kubernetes.io/projected/41c98b12-f6cc-4300-ba04-7c5a891f74eb-kube-api-access-9wqkp\") pod \"aws-ebs-csi-driver-node-4w87c\" (UID: \"41c98b12-f6cc-4300-ba04-7c5a891f74eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.542503 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.542485    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmcks\" (UniqueName: \"kubernetes.io/projected/07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32-kube-api-access-wmcks\") pod \"node-ca-27pc9\" (UID: \"07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32\") " pod="openshift-image-registry/node-ca-27pc9"
Apr 17 07:52:10.563294 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.563228    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8jsdp"
Apr 17 07:52:10.568799 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.568781    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v5cdv"
Apr 17 07:52:10.655698 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:10.655657    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd84846842e0dd9650476cd580a9ac6c.slice/crio-b27a087687d37c18a647cf50d3c080ddfad1c471ab58c7d8a7fe10fd88d2cdbe WatchSource:0}: Error finding container b27a087687d37c18a647cf50d3c080ddfad1c471ab58c7d8a7fe10fd88d2cdbe: Status 404 returned error can't find the container with id b27a087687d37c18a647cf50d3c080ddfad1c471ab58c7d8a7fe10fd88d2cdbe
Apr 17 07:52:10.655940 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:10.655915    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6643937e58b8d1200745e1d581977f.slice/crio-8a1f36b7d65f2b0a23e8dd3f32af736f02fd0490a753f478d8546b1aa4c7600c WatchSource:0}: Error finding container 8a1f36b7d65f2b0a23e8dd3f32af736f02fd0490a753f478d8546b1aa4c7600c: Status 404 returned error can't find the container with id 8a1f36b7d65f2b0a23e8dd3f32af736f02fd0490a753f478d8546b1aa4c7600c
Apr 17 07:52:10.659817 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.659802    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 17 07:52:10.734638 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.734608    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vcnrb"
Apr 17 07:52:10.740678 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:10.740640    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaccd983c_5d20_4f58_9453_9dd0a312b13b.slice/crio-25991b2d020718f81465f4058cc5c0a918337236212ef4148b5b4ae0838c2ec9 WatchSource:0}: Error finding container 25991b2d020718f81465f4058cc5c0a918337236212ef4148b5b4ae0838c2ec9: Status 404 returned error can't find the container with id 25991b2d020718f81465f4058cc5c0a918337236212ef4148b5b4ae0838c2ec9
Apr 17 07:52:10.752784 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.752766    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-27pc9"
Apr 17 07:52:10.758832 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:10.758802    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b9a2c8_ac6f_44a3_96e9_995ed5bf5d32.slice/crio-66eb6e1d1cefcc024c2718583c4435172b2e1bd132953a73dd3d94927318e8b4 WatchSource:0}: Error finding container 66eb6e1d1cefcc024c2718583c4435172b2e1bd132953a73dd3d94927318e8b4: Status 404 returned error can't find the container with id 66eb6e1d1cefcc024c2718583c4435172b2e1bd132953a73dd3d94927318e8b4
Apr 17 07:52:10.766109 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.766084    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8qgq7"
Apr 17 07:52:10.772002 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:10.771976    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff540440_d950_4564_be29_fd2a0bdc83d8.slice/crio-ee07f644e38c56ea2268e8097583fdf803885b356073e61a61be79d747ac66c8 WatchSource:0}: Error finding container ee07f644e38c56ea2268e8097583fdf803885b356073e61a61be79d747ac66c8: Status 404 returned error can't find the container with id ee07f644e38c56ea2268e8097583fdf803885b356073e61a61be79d747ac66c8
Apr 17 07:52:10.789057 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.789033    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:10.794857 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:10.794836    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode65a3e99_bd0c_4035_b41f_db6915c67e9a.slice/crio-d2ea71eabfb6cd7218cac3c6b0e4e36a1887dc9e1453c68df6c4f3724368a810 WatchSource:0}: Error finding container d2ea71eabfb6cd7218cac3c6b0e4e36a1887dc9e1453c68df6c4f3724368a810: Status 404 returned error can't find the container with id d2ea71eabfb6cd7218cac3c6b0e4e36a1887dc9e1453c68df6c4f3724368a810
Apr 17 07:52:10.803481 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.803463    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v2g9s"
Apr 17 07:52:10.808967 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.808950    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c"
Apr 17 07:52:10.815344 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:10.815323    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c98b12_f6cc_4300_ba04_7c5a891f74eb.slice/crio-3536146ca8b3705ad0582ed232d8782a4f86afc8f39b1a66f86c15af15635365 WatchSource:0}: Error finding container 3536146ca8b3705ad0582ed232d8782a4f86afc8f39b1a66f86c15af15635365: Status 404 returned error can't find the container with id 3536146ca8b3705ad0582ed232d8782a4f86afc8f39b1a66f86c15af15635365
Apr 17 07:52:10.832417 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:10.832389    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s5skc"
Apr 17 07:52:10.837925 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:10.837896    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3263f5a7_519d_490a_8444_599d2f8159d0.slice/crio-6af7ab78e77a55e6decd350e9d012589bf1cf558335c5d909dd494ba9c5d2df5 WatchSource:0}: Error finding container 6af7ab78e77a55e6decd350e9d012589bf1cf558335c5d909dd494ba9c5d2df5: Status 404 returned error can't find the container with id 6af7ab78e77a55e6decd350e9d012589bf1cf558335c5d909dd494ba9c5d2df5
Apr 17 07:52:10.874146 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:10.874107    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod161f0c24_f3ae_495e_a9db_213c81687526.slice/crio-5d8b3612e6cd15723508ad83f13c3f208d6ddb4ee9ab7433400cd7fdba7d2c3a WatchSource:0}: Error finding container 5d8b3612e6cd15723508ad83f13c3f208d6ddb4ee9ab7433400cd7fdba7d2c3a: Status 404 returned error can't find the container with id 5d8b3612e6cd15723508ad83f13c3f208d6ddb4ee9ab7433400cd7fdba7d2c3a
Apr 17 07:52:11.032684 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.032638    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:11.032899 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:11.032798    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:11.032899 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:11.032879    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs podName:6f91ec65-f7dd-4657-838b-318538310ead nodeName:}" failed. No retries permitted until 2026-04-17 07:52:12.0328577 +0000 UTC m=+3.086460889 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs") pod "network-metrics-daemon-wxkt9" (UID: "6f91ec65-f7dd-4657-838b-318538310ead") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:11.133418 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.133328    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k2nwx\" (UniqueName: \"kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx\") pod \"network-check-target-r78gk\" (UID: \"bde75c6d-e1a4-4848-bef6-94161be7ed61\") " pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:11.133598 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:11.133503    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 17 07:52:11.133598 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:11.133523    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 17 07:52:11.133598 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:11.133536    2571 projected.go:194] Error preparing data for projected volume kube-api-access-k2nwx for pod openshift-network-diagnostics/network-check-target-r78gk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:11.133598 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:11.133593    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx podName:bde75c6d-e1a4-4848-bef6-94161be7ed61 nodeName:}" failed. No retries permitted until 2026-04-17 07:52:12.133574286 +0000 UTC m=+3.187177494 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2nwx" (UniqueName: "kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx") pod "network-check-target-r78gk" (UID: "bde75c6d-e1a4-4848-bef6-94161be7ed61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:11.457947 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.457855    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-16 07:47:10 +0000 UTC" deadline="2027-11-13 09:51:33.267986442 +0000 UTC"
Apr 17 07:52:11.457947 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.457892    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13801h59m21.810097484s"
Apr 17 07:52:11.498375 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.498152    2571 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 17 07:52:11.527401 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.527369    2571 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 17 07:52:11.549834 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.549806    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:11.549989 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:11.549929    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:11.564370 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.564241    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v2g9s" event={"ID":"99ac4cf1-ceb9-44bd-9ab5-ad7588da1738","Type":"ContainerStarted","Data":"4844ec798e732de7839bca788756cf73d10880aa36f265a05a48ec8edd3a280b"}
Apr 17 07:52:11.567726 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.567649    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8qgq7" event={"ID":"ff540440-d950-4564-be29-fd2a0bdc83d8","Type":"ContainerStarted","Data":"ee07f644e38c56ea2268e8097583fdf803885b356073e61a61be79d747ac66c8"}
Apr 17 07:52:11.577128 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.577093    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-27pc9" event={"ID":"07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32","Type":"ContainerStarted","Data":"66eb6e1d1cefcc024c2718583c4435172b2e1bd132953a73dd3d94927318e8b4"}
Apr 17 07:52:11.585387 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.585359    2571 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Apr 17 07:52:11.589324 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.589249    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vcnrb" event={"ID":"accd983c-5d20-4f58-9453-9dd0a312b13b","Type":"ContainerStarted","Data":"25991b2d020718f81465f4058cc5c0a918337236212ef4148b5b4ae0838c2ec9"}
Apr 17 07:52:11.609658 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.609624    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-45.ec2.internal" event={"ID":"bd84846842e0dd9650476cd580a9ac6c","Type":"ContainerStarted","Data":"b27a087687d37c18a647cf50d3c080ddfad1c471ab58c7d8a7fe10fd88d2cdbe"}
Apr 17 07:52:11.615761 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.615686    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal" event={"ID":"4b6643937e58b8d1200745e1d581977f","Type":"ContainerStarted","Data":"8a1f36b7d65f2b0a23e8dd3f32af736f02fd0490a753f478d8546b1aa4c7600c"}
Apr 17 07:52:11.622646 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.622503    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jsdp" event={"ID":"161f0c24-f3ae-495e-a9db-213c81687526","Type":"ContainerStarted","Data":"5d8b3612e6cd15723508ad83f13c3f208d6ddb4ee9ab7433400cd7fdba7d2c3a"}
Apr 17 07:52:11.634557 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.634527    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s5skc" event={"ID":"3263f5a7-519d-490a-8444-599d2f8159d0","Type":"ContainerStarted","Data":"6af7ab78e77a55e6decd350e9d012589bf1cf558335c5d909dd494ba9c5d2df5"}
Apr 17 07:52:11.646745 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.646684    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c" event={"ID":"41c98b12-f6cc-4300-ba04-7c5a891f74eb","Type":"ContainerStarted","Data":"3536146ca8b3705ad0582ed232d8782a4f86afc8f39b1a66f86c15af15635365"}
Apr 17 07:52:11.659925 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.659882    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p" event={"ID":"e65a3e99-bd0c-4035-b41f-db6915c67e9a","Type":"ContainerStarted","Data":"d2ea71eabfb6cd7218cac3c6b0e4e36a1887dc9e1453c68df6c4f3724368a810"}
Apr 17 07:52:11.676871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:11.676789    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v5cdv" event={"ID":"268375f5-3f20-4e84-9af0-286587a445c7","Type":"ContainerStarted","Data":"b8be44c6a718a1ac8a45a49dc9cc70e1d693a11b0c7628ba2bb7301c554b8590"}
Apr 17 07:52:12.040227 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:12.040159    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:12.040397 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:12.040301    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:12.040397 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:12.040378    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs podName:6f91ec65-f7dd-4657-838b-318538310ead nodeName:}" failed. No retries permitted until 2026-04-17 07:52:14.040356955 +0000 UTC m=+5.093960161 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs") pod "network-metrics-daemon-wxkt9" (UID: "6f91ec65-f7dd-4657-838b-318538310ead") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:12.141011 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:12.140950    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k2nwx\" (UniqueName: \"kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx\") pod \"network-check-target-r78gk\" (UID: \"bde75c6d-e1a4-4848-bef6-94161be7ed61\") " pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:12.141212 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:12.141172    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 17 07:52:12.141212 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:12.141194    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 17 07:52:12.141212 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:12.141209    2571 projected.go:194] Error preparing data for projected volume kube-api-access-k2nwx for pod openshift-network-diagnostics/network-check-target-r78gk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:12.141378 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:12.141271    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx podName:bde75c6d-e1a4-4848-bef6-94161be7ed61 nodeName:}" failed. No retries permitted until 2026-04-17 07:52:14.141249389 +0000 UTC m=+5.194852584 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2nwx" (UniqueName: "kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx") pod "network-check-target-r78gk" (UID: "bde75c6d-e1a4-4848-bef6-94161be7ed61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:12.459165 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:12.459028    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-16 07:47:10 +0000 UTC" deadline="2027-10-24 03:59:09.031864255 +0000 UTC"
Apr 17 07:52:12.459165 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:12.459064    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13316h6m56.572803566s"
Apr 17 07:52:12.538700 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:12.538167    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:12.538700 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:12.538301    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:13.538458 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:13.538382    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:13.538912 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:13.538516    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:14.058288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:14.058251    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:14.058504 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:14.058408    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:14.058504 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:14.058488    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs podName:6f91ec65-f7dd-4657-838b-318538310ead nodeName:}" failed. No retries permitted until 2026-04-17 07:52:18.058469587 +0000 UTC m=+9.112072778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs") pod "network-metrics-daemon-wxkt9" (UID: "6f91ec65-f7dd-4657-838b-318538310ead") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:14.158917 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:14.158874    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k2nwx\" (UniqueName: \"kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx\") pod \"network-check-target-r78gk\" (UID: \"bde75c6d-e1a4-4848-bef6-94161be7ed61\") " pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:14.159153 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:14.159130    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 17 07:52:14.159203 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:14.159159    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 17 07:52:14.159203 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:14.159173    2571 projected.go:194] Error preparing data for projected volume kube-api-access-k2nwx for pod openshift-network-diagnostics/network-check-target-r78gk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:14.159272 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:14.159235    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx podName:bde75c6d-e1a4-4848-bef6-94161be7ed61 nodeName:}" failed. No retries permitted until 2026-04-17 07:52:18.159215698 +0000 UTC m=+9.212818901 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2nwx" (UniqueName: "kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx") pod "network-check-target-r78gk" (UID: "bde75c6d-e1a4-4848-bef6-94161be7ed61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:14.538748 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:14.538577    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:14.538748 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:14.538718    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:15.542201 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:15.542175    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:15.542697 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:15.542288    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:16.538426 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:16.538383    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:16.538661 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:16.538626    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:17.538582 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:17.538544    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:17.539026 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:17.538708    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:18.092697 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:18.092607    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:18.092889 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:18.092745    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:18.092889 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:18.092812    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs podName:6f91ec65-f7dd-4657-838b-318538310ead nodeName:}" failed. No retries permitted until 2026-04-17 07:52:26.092793123 +0000 UTC m=+17.146396316 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs") pod "network-metrics-daemon-wxkt9" (UID: "6f91ec65-f7dd-4657-838b-318538310ead") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:18.193585 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:18.193462    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k2nwx\" (UniqueName: \"kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx\") pod \"network-check-target-r78gk\" (UID: \"bde75c6d-e1a4-4848-bef6-94161be7ed61\") " pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:18.193770 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:18.193665    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 17 07:52:18.193770 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:18.193688    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 17 07:52:18.193770 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:18.193700    2571 projected.go:194] Error preparing data for projected volume kube-api-access-k2nwx for pod openshift-network-diagnostics/network-check-target-r78gk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:18.193770 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:18.193767    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx podName:bde75c6d-e1a4-4848-bef6-94161be7ed61 nodeName:}" failed. No retries permitted until 2026-04-17 07:52:26.193749446 +0000 UTC m=+17.247352648 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2nwx" (UniqueName: "kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx") pod "network-check-target-r78gk" (UID: "bde75c6d-e1a4-4848-bef6-94161be7ed61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:18.539411 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:18.538891    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:18.539411 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:18.539048    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:19.540176 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:19.540140    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:19.540608 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:19.540261    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:20.538995 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:20.538951    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:20.539166 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:20.539092    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:21.539560 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:21.539264    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:21.540008 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:21.539653    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:22.538514 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:22.538479    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:22.538681 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:22.538622    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:23.538217 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:23.538179    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:23.538710 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:23.538292    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:24.538541 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:24.538509    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:24.539013 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:24.538656    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:25.538996 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:25.538961    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:25.539461 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:25.539106    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:26.149747 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:26.149704    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:26.149970 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:26.149893    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:26.150042 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:26.149977    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs podName:6f91ec65-f7dd-4657-838b-318538310ead nodeName:}" failed. No retries permitted until 2026-04-17 07:52:42.149954476 +0000 UTC m=+33.203557664 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs") pod "network-metrics-daemon-wxkt9" (UID: "6f91ec65-f7dd-4657-838b-318538310ead") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:26.250973 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:26.250936    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k2nwx\" (UniqueName: \"kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx\") pod \"network-check-target-r78gk\" (UID: \"bde75c6d-e1a4-4848-bef6-94161be7ed61\") " pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:26.251155 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:26.251134    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 17 07:52:26.251213 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:26.251162    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 17 07:52:26.251213 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:26.251177    2571 projected.go:194] Error preparing data for projected volume kube-api-access-k2nwx for pod openshift-network-diagnostics/network-check-target-r78gk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:26.251301 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:26.251245    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx podName:bde75c6d-e1a4-4848-bef6-94161be7ed61 nodeName:}" failed. No retries permitted until 2026-04-17 07:52:42.25122633 +0000 UTC m=+33.304829533 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2nwx" (UniqueName: "kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx") pod "network-check-target-r78gk" (UID: "bde75c6d-e1a4-4848-bef6-94161be7ed61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:26.538973 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:26.538939    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:26.539131 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:26.539065    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:27.538853 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:27.538812    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:27.539041 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:27.538949    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:28.538798 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:28.538761    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:28.539237 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:28.538890    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:29.539116 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.539084    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:29.540005 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:29.539204    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:29.712398 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.712343    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-45.ec2.internal" event={"ID":"bd84846842e0dd9650476cd580a9ac6c","Type":"ContainerStarted","Data":"ec04efa1683fd89607c4afcf822dd9a31f1197fcd902524d9288563a479d6a46"}
Apr 17 07:52:29.714757 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.714727    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s5skc" event={"ID":"3263f5a7-519d-490a-8444-599d2f8159d0","Type":"ContainerStarted","Data":"1a1ed431bf2c569e720557f70d5b2d4b2650ac0e8b1e5e33664211a6a54b94b0"}
Apr 17 07:52:29.719455 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.719400    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 07:52:29.719869 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.719810    2571 generic.go:358] "Generic (PLEG): container finished" podID="e65a3e99-bd0c-4035-b41f-db6915c67e9a" containerID="39078473d6c7a14e0a40e23ebba387afbdb207d88568ae6de0bdc018abb13293" exitCode=1
Apr 17 07:52:29.719975 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.719885    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p" event={"ID":"e65a3e99-bd0c-4035-b41f-db6915c67e9a","Type":"ContainerStarted","Data":"bd965cea51f6dc1e98911e8f48b79683ce6137f7e7a49fa69cb3a9ea0952c7f2"}
Apr 17 07:52:29.719975 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.719910    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p" event={"ID":"e65a3e99-bd0c-4035-b41f-db6915c67e9a","Type":"ContainerStarted","Data":"c40ac46d4987fcdef110be52c2decd48f9a9025244573cd0e21a41eabd910253"}
Apr 17 07:52:29.719975 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.719924    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p" event={"ID":"e65a3e99-bd0c-4035-b41f-db6915c67e9a","Type":"ContainerStarted","Data":"f926b5bb20c4ce75efae3d30184eddd55f21b9f5493242ab88d108847827c712"}
Apr 17 07:52:29.719975 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.719936    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p" event={"ID":"e65a3e99-bd0c-4035-b41f-db6915c67e9a","Type":"ContainerStarted","Data":"58bd188b125cc778f0a11c5b474c805dffb41ce0ec0073d194e2698a29d202ff"}
Apr 17 07:52:29.719975 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.719948    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p" event={"ID":"e65a3e99-bd0c-4035-b41f-db6915c67e9a","Type":"ContainerDied","Data":"39078473d6c7a14e0a40e23ebba387afbdb207d88568ae6de0bdc018abb13293"}
Apr 17 07:52:29.719975 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.719965    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p" event={"ID":"e65a3e99-bd0c-4035-b41f-db6915c67e9a","Type":"ContainerStarted","Data":"dc36738097a8797b444947fc038ea1cc65d6b9b8f26828b62979ebb3246feb86"}
Apr 17 07:52:29.722076 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.721979    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v5cdv" event={"ID":"268375f5-3f20-4e84-9af0-286587a445c7","Type":"ContainerStarted","Data":"37539c38b70e30023ff49418f465114461fc25d7b4d59c54ecffc9c83e3b4465"}
Apr 17 07:52:29.729725 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.729673    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-45.ec2.internal" podStartSLOduration=20.729654772 podStartE2EDuration="20.729654772s" podCreationTimestamp="2026-04-17 07:52:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-17 07:52:29.728600361 +0000 UTC m=+20.782203573" watchObservedRunningTime="2026-04-17 07:52:29.729654772 +0000 UTC m=+20.783257983"
Apr 17 07:52:29.748059 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.748011    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-v5cdv" podStartSLOduration=2.584569937 podStartE2EDuration="20.747996171s" podCreationTimestamp="2026-04-17 07:52:09 +0000 UTC" firstStartedPulling="2026-04-17 07:52:10.895079609 +0000 UTC m=+1.948682798" lastFinishedPulling="2026-04-17 07:52:29.058505843 +0000 UTC m=+20.112109032" observedRunningTime="2026-04-17 07:52:29.74734839 +0000 UTC m=+20.800951603" watchObservedRunningTime="2026-04-17 07:52:29.747996171 +0000 UTC m=+20.801599382"
Apr 17 07:52:29.762520 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.762464    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v2g9s" podStartSLOduration=2.7262935 podStartE2EDuration="20.762450879s" podCreationTimestamp="2026-04-17 07:52:09 +0000 UTC" firstStartedPulling="2026-04-17 07:52:10.812346699 +0000 UTC m=+1.865949888" lastFinishedPulling="2026-04-17 07:52:28.848504071 +0000 UTC m=+19.902107267" observedRunningTime="2026-04-17 07:52:29.762240161 +0000 UTC m=+20.815843376" watchObservedRunningTime="2026-04-17 07:52:29.762450879 +0000 UTC m=+20.816054087"
Apr 17 07:52:29.780401 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:29.780336    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-s5skc" podStartSLOduration=2.792065652 podStartE2EDuration="20.780317302s" podCreationTimestamp="2026-04-17 07:52:09 +0000 UTC" firstStartedPulling="2026-04-17 07:52:10.839384862 +0000 UTC m=+1.892988053" lastFinishedPulling="2026-04-17 07:52:28.827636514 +0000 UTC m=+19.881239703" observedRunningTime="2026-04-17 07:52:29.779179058 +0000 UTC m=+20.832782264" watchObservedRunningTime="2026-04-17 07:52:29.780317302 +0000 UTC m=+20.833920513"
Apr 17 07:52:30.538599 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.538562    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:30.538762 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:30.538703    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:30.731948 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.731909    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8qgq7" event={"ID":"ff540440-d950-4564-be29-fd2a0bdc83d8","Type":"ContainerStarted","Data":"da36ff56a272a6e08e797dc2557ab1af345b908bcbeb1272aff52721253c50ff"}
Apr 17 07:52:30.733289 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.733255    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-27pc9" event={"ID":"07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32","Type":"ContainerStarted","Data":"d6b7d8493dbabc9705c3acfe63694ccff68fdbb524cf8cf319c3b1a414f82c2d"}
Apr 17 07:52:30.734443 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.734397    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vcnrb" event={"ID":"accd983c-5d20-4f58-9453-9dd0a312b13b","Type":"ContainerStarted","Data":"0820274617bdb3f523efc769b637c6923fb64cc2c21c936f55b687f6c3a9c21d"}
Apr 17 07:52:30.735572 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.735547    2571 generic.go:358] "Generic (PLEG): container finished" podID="4b6643937e58b8d1200745e1d581977f" containerID="96dd91634392a413834b599901000ec6f6dc916b1d2e4a630c62ae97763c206f" exitCode=0
Apr 17 07:52:30.735659 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.735610    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal" event={"ID":"4b6643937e58b8d1200745e1d581977f","Type":"ContainerDied","Data":"96dd91634392a413834b599901000ec6f6dc916b1d2e4a630c62ae97763c206f"}
Apr 17 07:52:30.737014 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.736993    2571 generic.go:358] "Generic (PLEG): container finished" podID="161f0c24-f3ae-495e-a9db-213c81687526" containerID="611b1bf8081955014013d7d97852f419d2f6f4bc725c51df2be955b7fd06868a" exitCode=0
Apr 17 07:52:30.737091 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.737055    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jsdp" event={"ID":"161f0c24-f3ae-495e-a9db-213c81687526","Type":"ContainerDied","Data":"611b1bf8081955014013d7d97852f419d2f6f4bc725c51df2be955b7fd06868a"}
Apr 17 07:52:30.738337 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.738285    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c" event={"ID":"41c98b12-f6cc-4300-ba04-7c5a891f74eb","Type":"ContainerStarted","Data":"378143d4a7035af70ae098de7893d35a2dd4134c33df928a53add75980c996c7"}
Apr 17 07:52:30.739487 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.739464    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v2g9s" event={"ID":"99ac4cf1-ceb9-44bd-9ab5-ad7588da1738","Type":"ContainerStarted","Data":"7f072043e8520129ab729fe1c96058d758b195b947c90add7b3cef19d2170d6e"}
Apr 17 07:52:30.748107 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.748065    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8qgq7" podStartSLOduration=3.61748153 podStartE2EDuration="21.748047065s" podCreationTimestamp="2026-04-17 07:52:09 +0000 UTC" firstStartedPulling="2026-04-17 07:52:10.773553694 +0000 UTC m=+1.827156883" lastFinishedPulling="2026-04-17 07:52:28.904119169 +0000 UTC m=+19.957722418" observedRunningTime="2026-04-17 07:52:30.747857638 +0000 UTC m=+21.801460847" watchObservedRunningTime="2026-04-17 07:52:30.748047065 +0000 UTC m=+21.801650270"
Apr 17 07:52:30.779901 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.779869    2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Apr 17 07:52:30.859891 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.859795    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-vcnrb" podStartSLOduration=3.702960919 podStartE2EDuration="21.85977836s" podCreationTimestamp="2026-04-17 07:52:09 +0000 UTC" firstStartedPulling="2026-04-17 07:52:10.742242034 +0000 UTC m=+1.795845223" lastFinishedPulling="2026-04-17 07:52:28.899059468 +0000 UTC m=+19.952662664" observedRunningTime="2026-04-17 07:52:30.84252387 +0000 UTC m=+21.896127078" watchObservedRunningTime="2026-04-17 07:52:30.85977836 +0000 UTC m=+21.913381572"
Apr 17 07:52:30.860013 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:30.859903    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-27pc9" podStartSLOduration=3.722431205 podStartE2EDuration="21.859894087s" podCreationTimestamp="2026-04-17 07:52:09 +0000 UTC" firstStartedPulling="2026-04-17 07:52:10.760539933 +0000 UTC m=+1.814143123" lastFinishedPulling="2026-04-17 07:52:28.898002808 +0000 UTC m=+19.951606005" observedRunningTime="2026-04-17 07:52:30.859541495 +0000 UTC m=+21.913144706" watchObservedRunningTime="2026-04-17 07:52:30.859894087 +0000 UTC m=+21.913497298"
Apr 17 07:52:31.494750 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:31.494647    2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-04-17T07:52:30.779892859Z","UUID":"e80cf530-70e2-4652-90e1-0e761b9a657b","Handler":null,"Name":"","Endpoint":""}
Apr 17 07:52:31.498198 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:31.498174    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Apr 17 07:52:31.498198 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:31.498207    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Apr 17 07:52:31.538674 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:31.538649    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:31.538795 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:31.538769    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:31.745046 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:31.744965    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 07:52:31.745559 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:31.745359    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p" event={"ID":"e65a3e99-bd0c-4035-b41f-db6915c67e9a","Type":"ContainerStarted","Data":"b00a32ebf8ddda8ceb156607457593e6c9e8e7fd04ad68653d5de88858afc11d"}
Apr 17 07:52:31.747808 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:31.747777    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal" event={"ID":"4b6643937e58b8d1200745e1d581977f","Type":"ContainerStarted","Data":"cb4acd7f4868c07b7500a31c188a6b141bf5accf749f6a5b922af0602ca7108a"}
Apr 17 07:52:31.750008 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:31.749979    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c" event={"ID":"41c98b12-f6cc-4300-ba04-7c5a891f74eb","Type":"ContainerStarted","Data":"f1e6d3ce4d621f773b70ae4596f25f0b6b9abcf81ca475901f8d3adb005be088"}
Apr 17 07:52:31.750135 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:31.750015    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c" event={"ID":"41c98b12-f6cc-4300-ba04-7c5a891f74eb","Type":"ContainerStarted","Data":"f79e658351dee28533081298b0252513880e9a8fb05864233db60f4251d6e2e8"}
Apr 17 07:52:31.762659 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:31.762616    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-45.ec2.internal" podStartSLOduration=22.762598538 podStartE2EDuration="22.762598538s" podCreationTimestamp="2026-04-17 07:52:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-17 07:52:31.762531715 +0000 UTC m=+22.816134936" watchObservedRunningTime="2026-04-17 07:52:31.762598538 +0000 UTC m=+22.816201765"
Apr 17 07:52:31.778022 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:31.777973    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w87c" podStartSLOduration=2.059966974 podStartE2EDuration="22.777938185s" podCreationTimestamp="2026-04-17 07:52:09 +0000 UTC" firstStartedPulling="2026-04-17 07:52:10.816893514 +0000 UTC m=+1.870496702" lastFinishedPulling="2026-04-17 07:52:31.534864711 +0000 UTC m=+22.588467913" observedRunningTime="2026-04-17 07:52:31.777828211 +0000 UTC m=+22.831431423" watchObservedRunningTime="2026-04-17 07:52:31.777938185 +0000 UTC m=+22.831541396"
Apr 17 07:52:32.538729 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:32.538698    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:32.538943 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:32.538836    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:33.538983 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:33.538947    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:33.539485 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:33.539081    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:34.025535 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:34.025246    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v2g9s"
Apr 17 07:52:34.026020 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:34.025998    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v2g9s"
Apr 17 07:52:34.538589 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:34.538550    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:34.538756 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:34.538702    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:34.755508 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:34.755483    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v2g9s"
Apr 17 07:52:34.756151 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:34.756136    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v2g9s"
Apr 17 07:52:35.538650 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:35.538459    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:35.538813 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:35.538723    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:35.758574 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:35.758535    2571 generic.go:358] "Generic (PLEG): container finished" podID="161f0c24-f3ae-495e-a9db-213c81687526" containerID="b50d009c08ceb0f5972c93f6fb71f0203071cef68767d54f038a260eceea3ea3" exitCode=0
Apr 17 07:52:35.759137 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:35.758627    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jsdp" event={"ID":"161f0c24-f3ae-495e-a9db-213c81687526","Type":"ContainerDied","Data":"b50d009c08ceb0f5972c93f6fb71f0203071cef68767d54f038a260eceea3ea3"}
Apr 17 07:52:35.761631 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:35.761614    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 07:52:35.761908 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:35.761886    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p" event={"ID":"e65a3e99-bd0c-4035-b41f-db6915c67e9a","Type":"ContainerStarted","Data":"77af231316957e8252a9fe4e980f651a2256ea028cf984cc0b6ecd368c532bd2"}
Apr 17 07:52:35.762329 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:35.762308    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:35.762411 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:35.762340    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:35.762485 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:35.762468    2571 scope.go:117] "RemoveContainer" containerID="39078473d6c7a14e0a40e23ebba387afbdb207d88568ae6de0bdc018abb13293"
Apr 17 07:52:35.779902 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:35.779881    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:36.538736 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:36.538270    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:36.538736 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:36.538398    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:36.739567 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:36.739485    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wxkt9"]
Apr 17 07:52:36.742338 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:36.742312    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r78gk"]
Apr 17 07:52:36.742471 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:36.742425    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:36.742555 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:36.742530    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:36.766088 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:36.766049    2571 generic.go:358] "Generic (PLEG): container finished" podID="161f0c24-f3ae-495e-a9db-213c81687526" containerID="c262fb20e306ae5ba7b009b7af2c0cb4d5564afdf1a4151518e4a0390df37168" exitCode=0
Apr 17 07:52:36.766497 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:36.766092    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jsdp" event={"ID":"161f0c24-f3ae-495e-a9db-213c81687526","Type":"ContainerDied","Data":"c262fb20e306ae5ba7b009b7af2c0cb4d5564afdf1a4151518e4a0390df37168"}
Apr 17 07:52:36.771985 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:36.771964    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 07:52:36.772380 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:36.772362    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:36.772380 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:36.772370    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p" event={"ID":"e65a3e99-bd0c-4035-b41f-db6915c67e9a","Type":"ContainerStarted","Data":"61fd1c0fb6b189c06661c8262871d8cd2eb4edb4fe18a29aa62d987f909c0fbe"}
Apr 17 07:52:36.772530 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:36.772488    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:36.772864 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:36.772840    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:36.789745 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:36.789719    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:52:36.810139 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:36.810089    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p" podStartSLOduration=9.690855638 podStartE2EDuration="27.81007814s" podCreationTimestamp="2026-04-17 07:52:09 +0000 UTC" firstStartedPulling="2026-04-17 07:52:10.796248781 +0000 UTC m=+1.849851973" lastFinishedPulling="2026-04-17 07:52:28.915471265 +0000 UTC m=+19.969074475" observedRunningTime="2026-04-17 07:52:36.808656015 +0000 UTC m=+27.862259225" watchObservedRunningTime="2026-04-17 07:52:36.81007814 +0000 UTC m=+27.863681350"
Apr 17 07:52:37.777735 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:37.777533    2571 generic.go:358] "Generic (PLEG): container finished" podID="161f0c24-f3ae-495e-a9db-213c81687526" containerID="15836200b2f9a2e6f07fc5abbb1d8a3f23f7cc56518bd10312a61441ab15ed26" exitCode=0
Apr 17 07:52:37.778111 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:37.777610    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jsdp" event={"ID":"161f0c24-f3ae-495e-a9db-213c81687526","Type":"ContainerDied","Data":"15836200b2f9a2e6f07fc5abbb1d8a3f23f7cc56518bd10312a61441ab15ed26"}
Apr 17 07:52:38.538350 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:38.538257    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:38.538549 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:38.538257    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:38.538549 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:38.538459    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:38.538549 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:38.538473    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:40.538580 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:40.538519    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:40.538580 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:40.538563    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:40.539354 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:40.538665    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxkt9" podUID="6f91ec65-f7dd-4657-838b-318538310ead"
Apr 17 07:52:40.539354 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:40.538780    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r78gk" podUID="bde75c6d-e1a4-4848-bef6-94161be7ed61"
Apr 17 07:52:41.768591 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.768563    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-45.ec2.internal" event="NodeReady"
Apr 17 07:52:41.769010 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.768711    2571 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
Apr 17 07:52:41.813378 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.813347    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jjcmh"]
Apr 17 07:52:41.839693 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.839662    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qqmvg"]
Apr 17 07:52:41.839864 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.839767    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:41.843503 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.843478    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Apr 17 07:52:41.843503 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.843480    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xs2k6\""
Apr 17 07:52:41.843673 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.843483    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Apr 17 07:52:41.852109 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.852091    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jjcmh"]
Apr 17 07:52:41.852109 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.852111    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qqmvg"]
Apr 17 07:52:41.852257 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.852204    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:52:41.854789 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.854768    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Apr 17 07:52:41.854886 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.854790    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Apr 17 07:52:41.854886 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.854819    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Apr 17 07:52:41.855010 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.854968    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wmmwx\""
Apr 17 07:52:41.959622 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.959538    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-config-volume\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:41.959622 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.959582    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:41.959845 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.959647    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:52:41.959845 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.959680    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccg2b\" (UniqueName: \"kubernetes.io/projected/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-kube-api-access-ccg2b\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:41.959845 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.959775    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqwv6\" (UniqueName: \"kubernetes.io/projected/68a69e91-436d-4788-b2de-32c453ccd36a-kube-api-access-xqwv6\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:52:41.959845 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:41.959811    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-tmp-dir\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:42.060871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.060822    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-config-volume\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:42.060871 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.060875    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:42.061123 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.060901    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:52:42.061123 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.060929    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ccg2b\" (UniqueName: \"kubernetes.io/projected/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-kube-api-access-ccg2b\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:42.061123 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.060961    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xqwv6\" (UniqueName: \"kubernetes.io/projected/68a69e91-436d-4788-b2de-32c453ccd36a-kube-api-access-xqwv6\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:52:42.061123 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.061059    2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 17 07:52:42.061323 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.061142    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert podName:68a69e91-436d-4788-b2de-32c453ccd36a nodeName:}" failed. No retries permitted until 2026-04-17 07:52:42.561116445 +0000 UTC m=+33.614719658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert") pod "ingress-canary-qqmvg" (UID: "68a69e91-436d-4788-b2de-32c453ccd36a") : secret "canary-serving-cert" not found
Apr 17 07:52:42.061323 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.061059    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 17 07:52:42.061323 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.061256    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls podName:b99d5d29-1f8b-46e2-b121-000bc5b77dfc nodeName:}" failed. No retries permitted until 2026-04-17 07:52:42.561236453 +0000 UTC m=+33.614839644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls") pod "dns-default-jjcmh" (UID: "b99d5d29-1f8b-46e2-b121-000bc5b77dfc") : secret "dns-default-metrics-tls" not found
Apr 17 07:52:42.061323 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.061275    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-tmp-dir\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:42.061568 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.061471    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-config-volume\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:42.061617 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.061595    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-tmp-dir\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:42.072305 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.072276    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccg2b\" (UniqueName: \"kubernetes.io/projected/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-kube-api-access-ccg2b\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:42.072471 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.072328    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqwv6\" (UniqueName: \"kubernetes.io/projected/68a69e91-436d-4788-b2de-32c453ccd36a-kube-api-access-xqwv6\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:52:42.162511 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.162469    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:42.162750 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.162623    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:42.162750 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.162699    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs podName:6f91ec65-f7dd-4657-838b-318538310ead nodeName:}" failed. No retries permitted until 2026-04-17 07:53:14.162683424 +0000 UTC m=+65.216286614 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs") pod "network-metrics-daemon-wxkt9" (UID: "6f91ec65-f7dd-4657-838b-318538310ead") : object "openshift-multus"/"metrics-daemon-secret" not registered
Apr 17 07:52:42.263539 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.263444    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k2nwx\" (UniqueName: \"kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx\") pod \"network-check-target-r78gk\" (UID: \"bde75c6d-e1a4-4848-bef6-94161be7ed61\") " pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:42.263701 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.263607    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Apr 17 07:52:42.263701 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.263638    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Apr 17 07:52:42.263701 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.263655    2571 projected.go:194] Error preparing data for projected volume kube-api-access-k2nwx for pod openshift-network-diagnostics/network-check-target-r78gk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:42.263828 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.263737    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx podName:bde75c6d-e1a4-4848-bef6-94161be7ed61 nodeName:}" failed. No retries permitted until 2026-04-17 07:53:14.26371827 +0000 UTC m=+65.317321459 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2nwx" (UniqueName: "kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx") pod "network-check-target-r78gk" (UID: "bde75c6d-e1a4-4848-bef6-94161be7ed61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Apr 17 07:52:42.538736 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.538696    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:52:42.538961 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.538696    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:52:42.541713 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.541683    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Apr 17 07:52:42.541878 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.541859    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z4m56\""
Apr 17 07:52:42.542041 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.542024    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Apr 17 07:52:42.542336 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.542318    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Apr 17 07:52:42.542898 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.542882    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ctqdx\""
Apr 17 07:52:42.565918 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.565873    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:42.566070 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:42.565922    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:52:42.566070 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.566046    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 17 07:52:42.566070 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.566061    2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 17 07:52:42.566202 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.566118    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert podName:68a69e91-436d-4788-b2de-32c453ccd36a nodeName:}" failed. No retries permitted until 2026-04-17 07:52:43.566102073 +0000 UTC m=+34.619705266 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert") pod "ingress-canary-qqmvg" (UID: "68a69e91-436d-4788-b2de-32c453ccd36a") : secret "canary-serving-cert" not found
Apr 17 07:52:42.566202 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:42.566132    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls podName:b99d5d29-1f8b-46e2-b121-000bc5b77dfc nodeName:}" failed. No retries permitted until 2026-04-17 07:52:43.566125904 +0000 UTC m=+34.619729093 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls") pod "dns-default-jjcmh" (UID: "b99d5d29-1f8b-46e2-b121-000bc5b77dfc") : secret "dns-default-metrics-tls" not found
Apr 17 07:52:43.573160 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:43.573120    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:43.573160 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:43.573160    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:52:43.573886 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:43.573260    2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 17 07:52:43.573886 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:43.573262    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 17 07:52:43.573886 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:43.573315    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls podName:b99d5d29-1f8b-46e2-b121-000bc5b77dfc nodeName:}" failed. No retries permitted until 2026-04-17 07:52:45.57330055 +0000 UTC m=+36.626903738 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls") pod "dns-default-jjcmh" (UID: "b99d5d29-1f8b-46e2-b121-000bc5b77dfc") : secret "dns-default-metrics-tls" not found
Apr 17 07:52:43.573886 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:43.573330    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert podName:68a69e91-436d-4788-b2de-32c453ccd36a nodeName:}" failed. No retries permitted until 2026-04-17 07:52:45.57332227 +0000 UTC m=+36.626925459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert") pod "ingress-canary-qqmvg" (UID: "68a69e91-436d-4788-b2de-32c453ccd36a") : secret "canary-serving-cert" not found
Apr 17 07:52:44.794948 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:44.794915    2571 generic.go:358] "Generic (PLEG): container finished" podID="161f0c24-f3ae-495e-a9db-213c81687526" containerID="0d38867b041612d96cf49595782ef1916f7da691337ad14b13eb3def16f5a639" exitCode=0
Apr 17 07:52:44.795375 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:44.794970    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jsdp" event={"ID":"161f0c24-f3ae-495e-a9db-213c81687526","Type":"ContainerDied","Data":"0d38867b041612d96cf49595782ef1916f7da691337ad14b13eb3def16f5a639"}
Apr 17 07:52:44.995601 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:44.995412    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq"]
Apr 17 07:52:45.010306 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.010281    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq"]
Apr 17 07:52:45.010446 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.010401    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq"
Apr 17 07:52:45.016284 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.013487    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Apr 17 07:52:45.016284 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.013552    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Apr 17 07:52:45.016284 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.013758    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Apr 17 07:52:45.016284 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.013837    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Apr 17 07:52:45.016284 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.013883    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-8ws99\""
Apr 17 07:52:45.024338 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.024314    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"]
Apr 17 07:52:45.031379 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.031363    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.036722 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.036705    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Apr 17 07:52:45.037183 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.037168    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Apr 17 07:52:45.050856 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.050769    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Apr 17 07:52:45.051909 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.051688    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Apr 17 07:52:45.052373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.052355    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"]
Apr 17 07:52:45.084325 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.084301    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kg6h\" (UniqueName: \"kubernetes.io/projected/521b6239-f9c5-4a22-937e-76f117665d99-kube-api-access-6kg6h\") pod \"managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq\" (UID: \"521b6239-f9c5-4a22-937e-76f117665d99\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq"
Apr 17 07:52:45.084464 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.084386    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/521b6239-f9c5-4a22-937e-76f117665d99-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq\" (UID: \"521b6239-f9c5-4a22-937e-76f117665d99\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq"
Apr 17 07:52:45.184906 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.184844    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/89477371-9b36-4af7-aff9-7c8ba8ced0ea-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.184906 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.184891    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkrj6\" (UniqueName: \"kubernetes.io/projected/89477371-9b36-4af7-aff9-7c8ba8ced0ea-kube-api-access-fkrj6\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.185150 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.184919    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6kg6h\" (UniqueName: \"kubernetes.io/projected/521b6239-f9c5-4a22-937e-76f117665d99-kube-api-access-6kg6h\") pod \"managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq\" (UID: \"521b6239-f9c5-4a22-937e-76f117665d99\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq"
Apr 17 07:52:45.185150 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.185028    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/89477371-9b36-4af7-aff9-7c8ba8ced0ea-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.185150 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.185065    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/521b6239-f9c5-4a22-937e-76f117665d99-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq\" (UID: \"521b6239-f9c5-4a22-937e-76f117665d99\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq"
Apr 17 07:52:45.185150 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.185095    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/89477371-9b36-4af7-aff9-7c8ba8ced0ea-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.185150 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.185115    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/89477371-9b36-4af7-aff9-7c8ba8ced0ea-ca\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.185339 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.185153    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/89477371-9b36-4af7-aff9-7c8ba8ced0ea-hub\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.188154 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.188131    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/521b6239-f9c5-4a22-937e-76f117665d99-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq\" (UID: \"521b6239-f9c5-4a22-937e-76f117665d99\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq"
Apr 17 07:52:45.193916 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.193894    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kg6h\" (UniqueName: \"kubernetes.io/projected/521b6239-f9c5-4a22-937e-76f117665d99-kube-api-access-6kg6h\") pod \"managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq\" (UID: \"521b6239-f9c5-4a22-937e-76f117665d99\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq"
Apr 17 07:52:45.286242 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.286204    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/89477371-9b36-4af7-aff9-7c8ba8ced0ea-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.286242 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.286244    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fkrj6\" (UniqueName: \"kubernetes.io/projected/89477371-9b36-4af7-aff9-7c8ba8ced0ea-kube-api-access-fkrj6\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.286494 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.286313    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/89477371-9b36-4af7-aff9-7c8ba8ced0ea-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.286494 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.286350    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/89477371-9b36-4af7-aff9-7c8ba8ced0ea-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.286494 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.286375    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/89477371-9b36-4af7-aff9-7c8ba8ced0ea-ca\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.286494 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.286412    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/89477371-9b36-4af7-aff9-7c8ba8ced0ea-hub\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.287213 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.287184    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/89477371-9b36-4af7-aff9-7c8ba8ced0ea-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.288795 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.288773    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/89477371-9b36-4af7-aff9-7c8ba8ced0ea-ca\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.288795 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.288790    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/89477371-9b36-4af7-aff9-7c8ba8ced0ea-hub\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.288925 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.288775    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/89477371-9b36-4af7-aff9-7c8ba8ced0ea-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.288977 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.288962    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/89477371-9b36-4af7-aff9-7c8ba8ced0ea-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.295514 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.295495    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkrj6\" (UniqueName: \"kubernetes.io/projected/89477371-9b36-4af7-aff9-7c8ba8ced0ea-kube-api-access-fkrj6\") pod \"cluster-proxy-proxy-agent-5b67b764f8-rvc8w\" (UID: \"89477371-9b36-4af7-aff9-7c8ba8ced0ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.360561 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.360400    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq"
Apr 17 07:52:45.366228 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.366207    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:52:45.529294 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.529261    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq"]
Apr 17 07:52:45.532052 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.532025    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"]
Apr 17 07:52:45.533382 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:45.533264    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod521b6239_f9c5_4a22_937e_76f117665d99.slice/crio-fda6b71d87f9db534a67459818af8e31cf1a96695159f4d6615c11f6785e8d44 WatchSource:0}: Error finding container fda6b71d87f9db534a67459818af8e31cf1a96695159f4d6615c11f6785e8d44: Status 404 returned error can't find the container with id fda6b71d87f9db534a67459818af8e31cf1a96695159f4d6615c11f6785e8d44
Apr 17 07:52:45.535170 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:52:45.535150    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89477371_9b36_4af7_aff9_7c8ba8ced0ea.slice/crio-ed150ac16e2c1e830ac1cfb9454f4817f21cb4ee3f254f2091aa3494f23f3a77 WatchSource:0}: Error finding container ed150ac16e2c1e830ac1cfb9454f4817f21cb4ee3f254f2091aa3494f23f3a77: Status 404 returned error can't find the container with id ed150ac16e2c1e830ac1cfb9454f4817f21cb4ee3f254f2091aa3494f23f3a77
Apr 17 07:52:45.589308 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.589271    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:45.589308 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.589318    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:52:45.589554 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:45.589465    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 17 07:52:45.589554 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:45.589498    2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 17 07:52:45.589554 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:45.589544    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert podName:68a69e91-436d-4788-b2de-32c453ccd36a nodeName:}" failed. No retries permitted until 2026-04-17 07:52:49.589525134 +0000 UTC m=+40.643128323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert") pod "ingress-canary-qqmvg" (UID: "68a69e91-436d-4788-b2de-32c453ccd36a") : secret "canary-serving-cert" not found
Apr 17 07:52:45.589656 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:45.589561    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls podName:b99d5d29-1f8b-46e2-b121-000bc5b77dfc nodeName:}" failed. No retries permitted until 2026-04-17 07:52:49.589553647 +0000 UTC m=+40.643156835 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls") pod "dns-default-jjcmh" (UID: "b99d5d29-1f8b-46e2-b121-000bc5b77dfc") : secret "dns-default-metrics-tls" not found
Apr 17 07:52:45.797869 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.797834    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq" event={"ID":"521b6239-f9c5-4a22-937e-76f117665d99","Type":"ContainerStarted","Data":"fda6b71d87f9db534a67459818af8e31cf1a96695159f4d6615c11f6785e8d44"}
Apr 17 07:52:45.798756 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.798732    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w" event={"ID":"89477371-9b36-4af7-aff9-7c8ba8ced0ea","Type":"ContainerStarted","Data":"ed150ac16e2c1e830ac1cfb9454f4817f21cb4ee3f254f2091aa3494f23f3a77"}
Apr 17 07:52:45.801593 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.801567    2571 generic.go:358] "Generic (PLEG): container finished" podID="161f0c24-f3ae-495e-a9db-213c81687526" containerID="27ac12cc309f16834a356ad02a329e421bf7a2e7d2cf2c802154b56332a96208" exitCode=0
Apr 17 07:52:45.801690 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:45.801608    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jsdp" event={"ID":"161f0c24-f3ae-495e-a9db-213c81687526","Type":"ContainerDied","Data":"27ac12cc309f16834a356ad02a329e421bf7a2e7d2cf2c802154b56332a96208"}
Apr 17 07:52:46.807820 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:46.807557    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jsdp" event={"ID":"161f0c24-f3ae-495e-a9db-213c81687526","Type":"ContainerStarted","Data":"9ebe24a51617215536920d0e5e26123cc57a738f9ede7254859b4c63abde0a18"}
Apr 17 07:52:46.835066 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:46.834994    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8jsdp" podStartSLOduration=5.037711152 podStartE2EDuration="37.834974307s" podCreationTimestamp="2026-04-17 07:52:09 +0000 UTC" firstStartedPulling="2026-04-17 07:52:10.880127711 +0000 UTC m=+1.933730900" lastFinishedPulling="2026-04-17 07:52:43.677390848 +0000 UTC m=+34.730994055" observedRunningTime="2026-04-17 07:52:46.834447956 +0000 UTC m=+37.888051175" watchObservedRunningTime="2026-04-17 07:52:46.834974307 +0000 UTC m=+37.888577521"
Apr 17 07:52:49.623191 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:49.623160    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:49.623191 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:49.623195    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:52:49.623658 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:49.623302    2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 17 07:52:49.623658 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:49.623304    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 17 07:52:49.623658 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:49.623356    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert podName:68a69e91-436d-4788-b2de-32c453ccd36a nodeName:}" failed. No retries permitted until 2026-04-17 07:52:57.623340949 +0000 UTC m=+48.676944138 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert") pod "ingress-canary-qqmvg" (UID: "68a69e91-436d-4788-b2de-32c453ccd36a") : secret "canary-serving-cert" not found
Apr 17 07:52:49.623658 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:49.623371    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls podName:b99d5d29-1f8b-46e2-b121-000bc5b77dfc nodeName:}" failed. No retries permitted until 2026-04-17 07:52:57.623362653 +0000 UTC m=+48.676965842 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls") pod "dns-default-jjcmh" (UID: "b99d5d29-1f8b-46e2-b121-000bc5b77dfc") : secret "dns-default-metrics-tls" not found
Apr 17 07:52:49.815270 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:49.815235    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w" event={"ID":"89477371-9b36-4af7-aff9-7c8ba8ced0ea","Type":"ContainerStarted","Data":"eb9881366d6e02e5b833cdd60cab17dcbc1011faab798fa483ac09e0b1a619b0"}
Apr 17 07:52:49.816408 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:49.816383    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq" event={"ID":"521b6239-f9c5-4a22-937e-76f117665d99","Type":"ContainerStarted","Data":"3a7dad675047f83f4fd0a0c3904c1a6ec54295caa479e5933e4635b103185efa"}
Apr 17 07:52:49.831773 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:49.831730    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f98ff9bc9-r8pfq" podStartSLOduration=1.9551344309999998 podStartE2EDuration="5.831717994s" podCreationTimestamp="2026-04-17 07:52:44 +0000 UTC" firstStartedPulling="2026-04-17 07:52:45.535367086 +0000 UTC m=+36.588970275" lastFinishedPulling="2026-04-17 07:52:49.411950645 +0000 UTC m=+40.465553838" observedRunningTime="2026-04-17 07:52:49.830939681 +0000 UTC m=+40.884542896" watchObservedRunningTime="2026-04-17 07:52:49.831717994 +0000 UTC m=+40.885321204"
Apr 17 07:52:52.824614 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:52.824572    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w" event={"ID":"89477371-9b36-4af7-aff9-7c8ba8ced0ea","Type":"ContainerStarted","Data":"cb10facb051f7ede94df013d35a53f3863a269580ed1e14e3d34582ddb5d0de0"}
Apr 17 07:52:52.824614 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:52.824610    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w" event={"ID":"89477371-9b36-4af7-aff9-7c8ba8ced0ea","Type":"ContainerStarted","Data":"6f2bbcecb8612d14ce67761f77a5c98bd97e52f45dff19ea387a406c364c8613"}
Apr 17 07:52:52.842348 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:52.842299    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w" podStartSLOduration=1.595250273 podStartE2EDuration="7.842284478s" podCreationTimestamp="2026-04-17 07:52:45 +0000 UTC" firstStartedPulling="2026-04-17 07:52:45.536842012 +0000 UTC m=+36.590445205" lastFinishedPulling="2026-04-17 07:52:51.783876201 +0000 UTC m=+42.837479410" observedRunningTime="2026-04-17 07:52:52.841091172 +0000 UTC m=+43.894694382" watchObservedRunningTime="2026-04-17 07:52:52.842284478 +0000 UTC m=+43.895887688"
Apr 17 07:52:57.682857 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:57.682808    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:52:57.682857 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:52:57.682853    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:52:57.683293 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:57.682961    2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 17 07:52:57.683293 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:57.682964    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 17 07:52:57.683293 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:57.683013    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert podName:68a69e91-436d-4788-b2de-32c453ccd36a nodeName:}" failed. No retries permitted until 2026-04-17 07:53:13.682999598 +0000 UTC m=+64.736602787 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert") pod "ingress-canary-qqmvg" (UID: "68a69e91-436d-4788-b2de-32c453ccd36a") : secret "canary-serving-cert" not found
Apr 17 07:52:57.683293 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:52:57.683026    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls podName:b99d5d29-1f8b-46e2-b121-000bc5b77dfc nodeName:}" failed. No retries permitted until 2026-04-17 07:53:13.68302007 +0000 UTC m=+64.736623259 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls") pod "dns-default-jjcmh" (UID: "b99d5d29-1f8b-46e2-b121-000bc5b77dfc") : secret "dns-default-metrics-tls" not found
Apr 17 07:53:08.790293 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:08.790264    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krj9p"
Apr 17 07:53:13.693885 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:13.693844    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:53:13.693885 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:13.693887    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:53:13.694299 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:53:13.693982    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 17 07:53:13.694299 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:53:13.694002    2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 17 07:53:13.694299 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:53:13.694051    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls podName:b99d5d29-1f8b-46e2-b121-000bc5b77dfc nodeName:}" failed. No retries permitted until 2026-04-17 07:53:45.694034874 +0000 UTC m=+96.747638062 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls") pod "dns-default-jjcmh" (UID: "b99d5d29-1f8b-46e2-b121-000bc5b77dfc") : secret "dns-default-metrics-tls" not found
Apr 17 07:53:13.694299 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:53:13.694065    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert podName:68a69e91-436d-4788-b2de-32c453ccd36a nodeName:}" failed. No retries permitted until 2026-04-17 07:53:45.694059209 +0000 UTC m=+96.747662398 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert") pod "ingress-canary-qqmvg" (UID: "68a69e91-436d-4788-b2de-32c453ccd36a") : secret "canary-serving-cert" not found
Apr 17 07:53:14.197054 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:14.197017    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:53:14.199721 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:14.199704    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Apr 17 07:53:14.207937 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:53:14.207915    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Apr 17 07:53:14.207995 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:53:14.207978    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs podName:6f91ec65-f7dd-4657-838b-318538310ead nodeName:}" failed. No retries permitted until 2026-04-17 07:54:18.207959939 +0000 UTC m=+129.261563128 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs") pod "network-metrics-daemon-wxkt9" (UID: "6f91ec65-f7dd-4657-838b-318538310ead") : secret "metrics-daemon-secret" not found
Apr 17 07:53:14.297551 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:14.297521    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k2nwx\" (UniqueName: \"kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx\") pod \"network-check-target-r78gk\" (UID: \"bde75c6d-e1a4-4848-bef6-94161be7ed61\") " pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:53:14.300301 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:14.300283    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Apr 17 07:53:14.310207 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:14.310187    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Apr 17 07:53:14.322636 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:14.322613    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2nwx\" (UniqueName: \"kubernetes.io/projected/bde75c6d-e1a4-4848-bef6-94161be7ed61-kube-api-access-k2nwx\") pod \"network-check-target-r78gk\" (UID: \"bde75c6d-e1a4-4848-bef6-94161be7ed61\") " pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:53:14.358002 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:14.357977    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z4m56\""
Apr 17 07:53:14.365980 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:14.365960    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:53:14.483028 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:14.482956    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r78gk"]
Apr 17 07:53:14.486038 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:53:14.486007    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbde75c6d_e1a4_4848_bef6_94161be7ed61.slice/crio-0f983434f5cef15a31f28aa4013ae184cec6dd64f95a5c31e8e6e33726fe6d82 WatchSource:0}: Error finding container 0f983434f5cef15a31f28aa4013ae184cec6dd64f95a5c31e8e6e33726fe6d82: Status 404 returned error can't find the container with id 0f983434f5cef15a31f28aa4013ae184cec6dd64f95a5c31e8e6e33726fe6d82
Apr 17 07:53:14.867047 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:14.867009    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r78gk" event={"ID":"bde75c6d-e1a4-4848-bef6-94161be7ed61","Type":"ContainerStarted","Data":"0f983434f5cef15a31f28aa4013ae184cec6dd64f95a5c31e8e6e33726fe6d82"}
Apr 17 07:53:18.877051 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:18.876958    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r78gk" event={"ID":"bde75c6d-e1a4-4848-bef6-94161be7ed61","Type":"ContainerStarted","Data":"ed95b6d7ff637539fb3189a3faf89cb92b9893bd2a88becc980de90aebe7bb6d"}
Apr 17 07:53:18.877538 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:18.877089    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:53:18.896763 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:18.896718    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r78gk" podStartSLOduration=65.765608619 podStartE2EDuration="1m9.8967032s" podCreationTimestamp="2026-04-17 07:52:09 +0000 UTC" firstStartedPulling="2026-04-17 07:53:14.487966292 +0000 UTC m=+65.541569482" lastFinishedPulling="2026-04-17 07:53:18.619060871 +0000 UTC m=+69.672664063" observedRunningTime="2026-04-17 07:53:18.896053805 +0000 UTC m=+69.949657007" watchObservedRunningTime="2026-04-17 07:53:18.8967032 +0000 UTC m=+69.950306406"
Apr 17 07:53:45.714459 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:45.714309    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:53:45.714459 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:45.714344    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:53:45.714962 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:53:45.714473    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Apr 17 07:53:45.714962 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:53:45.714483    2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Apr 17 07:53:45.714962 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:53:45.714537    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls podName:b99d5d29-1f8b-46e2-b121-000bc5b77dfc nodeName:}" failed. No retries permitted until 2026-04-17 07:54:49.71452235 +0000 UTC m=+160.768125539 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls") pod "dns-default-jjcmh" (UID: "b99d5d29-1f8b-46e2-b121-000bc5b77dfc") : secret "dns-default-metrics-tls" not found
Apr 17 07:53:45.714962 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:53:45.714552    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert podName:68a69e91-436d-4788-b2de-32c453ccd36a nodeName:}" failed. No retries permitted until 2026-04-17 07:54:49.714545934 +0000 UTC m=+160.768149122 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert") pod "ingress-canary-qqmvg" (UID: "68a69e91-436d-4788-b2de-32c453ccd36a") : secret "canary-serving-cert" not found
Apr 17 07:53:49.884774 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:53:49.884741    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r78gk"
Apr 17 07:54:05.259021 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:05.258993    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vcnrb_accd983c-5d20-4f58-9453-9dd0a312b13b/dns-node-resolver/0.log"
Apr 17 07:54:05.657948 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:05.657921    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-27pc9_07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32/node-ca/0.log"
Apr 17 07:54:18.235590 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:18.235552    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:54:18.237826 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:18.237801    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f91ec65-f7dd-4657-838b-318538310ead-metrics-certs\") pod \"network-metrics-daemon-wxkt9\" (UID: \"6f91ec65-f7dd-4657-838b-318538310ead\") " pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:54:18.252631 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:18.252609    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ctqdx\""
Apr 17 07:54:18.260631 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:18.260616    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxkt9"
Apr 17 07:54:18.370932 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:18.370905    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wxkt9"]
Apr 17 07:54:18.373876 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:54:18.373847    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f91ec65_f7dd_4657_838b_318538310ead.slice/crio-2d50c0dc929b66cd5d4ab0670e42bcb88e08cbe285932e242e258698162a4662 WatchSource:0}: Error finding container 2d50c0dc929b66cd5d4ab0670e42bcb88e08cbe285932e242e258698162a4662: Status 404 returned error can't find the container with id 2d50c0dc929b66cd5d4ab0670e42bcb88e08cbe285932e242e258698162a4662
Apr 17 07:54:19.014366 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:19.014326    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxkt9" event={"ID":"6f91ec65-f7dd-4657-838b-318538310ead","Type":"ContainerStarted","Data":"2d50c0dc929b66cd5d4ab0670e42bcb88e08cbe285932e242e258698162a4662"}
Apr 17 07:54:20.018725 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:20.018687    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxkt9" event={"ID":"6f91ec65-f7dd-4657-838b-318538310ead","Type":"ContainerStarted","Data":"64a2dc1fde958c308a4d395139d704de9ceebcd4a901cbb2f8209a71c6303af4"}
Apr 17 07:54:20.018725 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:20.018730    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxkt9" event={"ID":"6f91ec65-f7dd-4657-838b-318538310ead","Type":"ContainerStarted","Data":"87f9ad317a59dd927c34f2b13aaa6b733c89f07e83121a17889f5624d9cd8428"}
Apr 17 07:54:20.035137 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:20.035089    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wxkt9" podStartSLOduration=129.991035805 podStartE2EDuration="2m11.035074284s" podCreationTimestamp="2026-04-17 07:52:09 +0000 UTC" firstStartedPulling="2026-04-17 07:54:18.376163659 +0000 UTC m=+129.429766848" lastFinishedPulling="2026-04-17 07:54:19.420202135 +0000 UTC m=+130.473805327" observedRunningTime="2026-04-17 07:54:20.034306803 +0000 UTC m=+131.087910014" watchObservedRunningTime="2026-04-17 07:54:20.035074284 +0000 UTC m=+131.088677495"
Apr 17 07:54:24.014746 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.014717    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-74g7z"]
Apr 17 07:54:24.017732 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.017714    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.038363 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.038330    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Apr 17 07:54:24.039383 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.039364    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Apr 17 07:54:24.039383 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.039376    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Apr 17 07:54:24.039597 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.039421    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Apr 17 07:54:24.039597 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.039379    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s2zn8\""
Apr 17 07:54:24.045396 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.045369    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-74g7z"]
Apr 17 07:54:24.077499 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.077466    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.077689 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.077514    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-crio-socket\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.077689 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.077587    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-data-volume\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.077689 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.077613    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq998\" (UniqueName: \"kubernetes.io/projected/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-kube-api-access-sq998\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.077689 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.077666    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.178651 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.178604    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-data-volume\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.178651 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.178656    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sq998\" (UniqueName: \"kubernetes.io/projected/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-kube-api-access-sq998\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.178896 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.178688    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.178896 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.178721    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.178896 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.178755    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-crio-socket\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.178896 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.178841    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-crio-socket\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.179029 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.179010    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-data-volume\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.179232 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.179212    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.180967 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.180948    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.198093 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.198074    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq998\" (UniqueName: \"kubernetes.io/projected/4acfa4c5-aaf8-4d9b-bf5d-7b0640375711-kube-api-access-sq998\") pod \"insights-runtime-extractor-74g7z\" (UID: \"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711\") " pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.325783 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.325753    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-74g7z"
Apr 17 07:54:24.443393 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:24.443360    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-74g7z"]
Apr 17 07:54:24.446446 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:54:24.446404    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4acfa4c5_aaf8_4d9b_bf5d_7b0640375711.slice/crio-a03a1e0ae956ce432a2d47bee4c15c4db1f3740bcb3e944ace8804baf363250e WatchSource:0}: Error finding container a03a1e0ae956ce432a2d47bee4c15c4db1f3740bcb3e944ace8804baf363250e: Status 404 returned error can't find the container with id a03a1e0ae956ce432a2d47bee4c15c4db1f3740bcb3e944ace8804baf363250e
Apr 17 07:54:25.032386 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:25.032353    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-74g7z" event={"ID":"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711","Type":"ContainerStarted","Data":"f43607e0ac2db194f6e1e7b58e2b98b6be68fc6d8296e9935f8d1f9faf45da59"}
Apr 17 07:54:25.032386 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:25.032389    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-74g7z" event={"ID":"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711","Type":"ContainerStarted","Data":"a03a1e0ae956ce432a2d47bee4c15c4db1f3740bcb3e944ace8804baf363250e"}
Apr 17 07:54:25.367890 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:25.367768    2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w" podUID="89477371-9b36-4af7-aff9-7c8ba8ced0ea" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Apr 17 07:54:26.036809 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:26.036765    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-74g7z" event={"ID":"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711","Type":"ContainerStarted","Data":"0278d27bbee26c512b6afb8f0014c0439557a11c55f557445bc0afff0c5c77e8"}
Apr 17 07:54:27.041257 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:27.041221    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-74g7z" event={"ID":"4acfa4c5-aaf8-4d9b-bf5d-7b0640375711","Type":"ContainerStarted","Data":"f6b62d2831d9969d2758ccc7fb1ac9b95b4b179b000db074bb7879fae2db8b00"}
Apr 17 07:54:27.057886 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:27.057844    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-74g7z" podStartSLOduration=2.155257212 podStartE2EDuration="4.057830164s" podCreationTimestamp="2026-04-17 07:54:23 +0000 UTC" firstStartedPulling="2026-04-17 07:54:24.494702886 +0000 UTC m=+135.548306074" lastFinishedPulling="2026-04-17 07:54:26.397275837 +0000 UTC m=+137.450879026" observedRunningTime="2026-04-17 07:54:27.056175706 +0000 UTC m=+138.109778918" watchObservedRunningTime="2026-04-17 07:54:27.057830164 +0000 UTC m=+138.111433374"
Apr 17 07:54:30.815178 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.815141    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-647ffb494d-9p7x5"]
Apr 17 07:54:30.818034 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.818017    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:30.822084 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.822064    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Apr 17 07:54:30.822208 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.822186    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Apr 17 07:54:30.822263 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.822251    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Apr 17 07:54:30.822552 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.822532    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Apr 17 07:54:30.822674 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.822591    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Apr 17 07:54:30.823220 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.823202    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Apr 17 07:54:30.823356 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.823261    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Apr 17 07:54:30.823356 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.823307    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jmjjr\""
Apr 17 07:54:30.835352 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.835329    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-647ffb494d-9p7x5"]
Apr 17 07:54:30.928630 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.928597    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-serving-cert\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:30.928816 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.928709    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-service-ca\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:30.928816 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.928753    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-config\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:30.928816 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.928786    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-oauth-config\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:30.928816 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.928811    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-oauth-serving-cert\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:30.928990 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:30.928835    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wftr9\" (UniqueName: \"kubernetes.io/projected/e13695f3-4327-46c1-b4d1-4225aa82f4bb-kube-api-access-wftr9\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.029813 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.029786    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-serving-cert\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.030016 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.029852    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-service-ca\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.030305 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.030261    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-config\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.030466 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.030314    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-oauth-config\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.030466 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.030361    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-oauth-serving-cert\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.030466 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.030394    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wftr9\" (UniqueName: \"kubernetes.io/projected/e13695f3-4327-46c1-b4d1-4225aa82f4bb-kube-api-access-wftr9\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.031273 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.031072    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-config\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.032024 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.031999    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-service-ca\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.032156 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.032088    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-oauth-serving-cert\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.033195 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.033171    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-serving-cert\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.033546 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.033521    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-oauth-config\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.039231 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.039206    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wftr9\" (UniqueName: \"kubernetes.io/projected/e13695f3-4327-46c1-b4d1-4225aa82f4bb-kube-api-access-wftr9\") pod \"console-647ffb494d-9p7x5\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") " pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.127036 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.126935    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:31.239861 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:31.239832    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-647ffb494d-9p7x5"]
Apr 17 07:54:31.243080 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:54:31.243049    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode13695f3_4327_46c1_b4d1_4225aa82f4bb.slice/crio-733e7364e224dbb6a62f3e06b268ab16de72e335d17f0a12d092aad34533c695 WatchSource:0}: Error finding container 733e7364e224dbb6a62f3e06b268ab16de72e335d17f0a12d092aad34533c695: Status 404 returned error can't find the container with id 733e7364e224dbb6a62f3e06b268ab16de72e335d17f0a12d092aad34533c695
Apr 17 07:54:32.055198 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:32.055163    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-647ffb494d-9p7x5" event={"ID":"e13695f3-4327-46c1-b4d1-4225aa82f4bb","Type":"ContainerStarted","Data":"733e7364e224dbb6a62f3e06b268ab16de72e335d17f0a12d092aad34533c695"}
Apr 17 07:54:32.667046 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:32.667010    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-2zvfs"]
Apr 17 07:54:32.670116 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:32.670095    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-2zvfs"
Apr 17 07:54:32.672367 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:32.672340    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Apr 17 07:54:32.672483 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:32.672462    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-zxn88\""
Apr 17 07:54:32.678883 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:32.678536    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-2zvfs"]
Apr 17 07:54:32.744738 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:32.744687    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49619515-087b-4a5a-aeb4-b95a5802cde4-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-2zvfs\" (UID: \"49619515-087b-4a5a-aeb4-b95a5802cde4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-2zvfs"
Apr 17 07:54:32.846087 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:32.846045    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49619515-087b-4a5a-aeb4-b95a5802cde4-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-2zvfs\" (UID: \"49619515-087b-4a5a-aeb4-b95a5802cde4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-2zvfs"
Apr 17 07:54:32.848764 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:32.848736    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49619515-087b-4a5a-aeb4-b95a5802cde4-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-2zvfs\" (UID: \"49619515-087b-4a5a-aeb4-b95a5802cde4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-2zvfs"
Apr 17 07:54:32.982111 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:32.982026    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-2zvfs"
Apr 17 07:54:33.875399 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:33.875374    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-2zvfs"]
Apr 17 07:54:33.878444 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:54:33.878412    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49619515_087b_4a5a_aeb4_b95a5802cde4.slice/crio-2e611690d7ae059ba9d89f09ca5b6d3b2f3691a3db231d2d97bd1eb61617aa84 WatchSource:0}: Error finding container 2e611690d7ae059ba9d89f09ca5b6d3b2f3691a3db231d2d97bd1eb61617aa84: Status 404 returned error can't find the container with id 2e611690d7ae059ba9d89f09ca5b6d3b2f3691a3db231d2d97bd1eb61617aa84
Apr 17 07:54:34.060554 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:34.060515    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-2zvfs" event={"ID":"49619515-087b-4a5a-aeb4-b95a5802cde4","Type":"ContainerStarted","Data":"2e611690d7ae059ba9d89f09ca5b6d3b2f3691a3db231d2d97bd1eb61617aa84"}
Apr 17 07:54:34.061732 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:34.061709    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-647ffb494d-9p7x5" event={"ID":"e13695f3-4327-46c1-b4d1-4225aa82f4bb","Type":"ContainerStarted","Data":"ab92f7841c370d68dd04858c498d998829eaec58812d954a1121af7c78dee68e"}
Apr 17 07:54:34.078621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:34.078560    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-647ffb494d-9p7x5" podStartSLOduration=1.504686416 podStartE2EDuration="4.078548596s" podCreationTimestamp="2026-04-17 07:54:30 +0000 UTC" firstStartedPulling="2026-04-17 07:54:31.244810424 +0000 UTC m=+142.298413614" lastFinishedPulling="2026-04-17 07:54:33.81867259 +0000 UTC m=+144.872275794" observedRunningTime="2026-04-17 07:54:34.077336328 +0000 UTC m=+145.130939541" watchObservedRunningTime="2026-04-17 07:54:34.078548596 +0000 UTC m=+145.132151806"
Apr 17 07:54:35.065593 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:35.065558    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-2zvfs" event={"ID":"49619515-087b-4a5a-aeb4-b95a5802cde4","Type":"ContainerStarted","Data":"70bc082b5aa1c7c558d2f375ce1ce74439422897eb58c23a8ce8ffe31aa0983e"}
Apr 17 07:54:35.066023 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:35.065842    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-2zvfs"
Apr 17 07:54:35.071061 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:35.071039    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-2zvfs"
Apr 17 07:54:35.082809 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:35.082775    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-2zvfs" podStartSLOduration=2.086118989 podStartE2EDuration="3.082765738s" podCreationTimestamp="2026-04-17 07:54:32 +0000 UTC" firstStartedPulling="2026-04-17 07:54:33.880331927 +0000 UTC m=+144.933935117" lastFinishedPulling="2026-04-17 07:54:34.876978665 +0000 UTC m=+145.930581866" observedRunningTime="2026-04-17 07:54:35.081752988 +0000 UTC m=+146.135356199" watchObservedRunningTime="2026-04-17 07:54:35.082765738 +0000 UTC m=+146.136368948"
Apr 17 07:54:35.367419 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:35.367385    2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w" podUID="89477371-9b36-4af7-aff9-7c8ba8ced0ea" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Apr 17 07:54:41.127427 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:41.127391    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:41.127825 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:41.127523    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:41.133181 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:41.133160    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:42.084945 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:42.084918    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:54:43.162938 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.162903    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"]
Apr 17 07:54:43.166129 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.166111    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.168365 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.168343    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Apr 17 07:54:43.168822 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.168800    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Apr 17 07:54:43.169178 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.169162    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Apr 17 07:54:43.170886 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.170867    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-cnvks\""
Apr 17 07:54:43.170964 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.170895    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Apr 17 07:54:43.171026 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.170899    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Apr 17 07:54:43.180718 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.180698    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"]
Apr 17 07:54:43.317777 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.317741    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b7k7k"]
Apr 17 07:54:43.320802 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.320784    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.323258 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.323228    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Apr 17 07:54:43.323373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.323303    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Apr 17 07:54:43.323373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.323231    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d171f5a-aa91-4614-9e29-c3e5b59fdba3-metrics-client-ca\") pod \"openshift-state-metrics-9d44df66c-258jh\" (UID: \"9d171f5a-aa91-4614-9e29-c3e5b59fdba3\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.323373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.323318    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9c8dc\""
Apr 17 07:54:43.323373 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.323318    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Apr 17 07:54:43.323575 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.323377    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d171f5a-aa91-4614-9e29-c3e5b59fdba3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9d44df66c-258jh\" (UID: \"9d171f5a-aa91-4614-9e29-c3e5b59fdba3\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.323575 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.323488    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d171f5a-aa91-4614-9e29-c3e5b59fdba3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9d44df66c-258jh\" (UID: \"9d171f5a-aa91-4614-9e29-c3e5b59fdba3\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.323575 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.323520    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr7xw\" (UniqueName: \"kubernetes.io/projected/9d171f5a-aa91-4614-9e29-c3e5b59fdba3-kube-api-access-gr7xw\") pod \"openshift-state-metrics-9d44df66c-258jh\" (UID: \"9d171f5a-aa91-4614-9e29-c3e5b59fdba3\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.424250 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.424162    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgwwr\" (UniqueName: \"kubernetes.io/projected/763faa91-078d-456f-9daf-77ee5acc9aca-kube-api-access-sgwwr\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.424250 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.424197    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/763faa91-078d-456f-9daf-77ee5acc9aca-sys\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.424250 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.424222    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d171f5a-aa91-4614-9e29-c3e5b59fdba3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9d44df66c-258jh\" (UID: \"9d171f5a-aa91-4614-9e29-c3e5b59fdba3\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.424250 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.424244    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-textfile\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.424570 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.424273    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.424570 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.424320    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gr7xw\" (UniqueName: \"kubernetes.io/projected/9d171f5a-aa91-4614-9e29-c3e5b59fdba3-kube-api-access-gr7xw\") pod \"openshift-state-metrics-9d44df66c-258jh\" (UID: \"9d171f5a-aa91-4614-9e29-c3e5b59fdba3\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.424570 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.424359    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-tls\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.424570 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.424396    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d171f5a-aa91-4614-9e29-c3e5b59fdba3-metrics-client-ca\") pod \"openshift-state-metrics-9d44df66c-258jh\" (UID: \"9d171f5a-aa91-4614-9e29-c3e5b59fdba3\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.424570 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.424420    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/763faa91-078d-456f-9daf-77ee5acc9aca-metrics-client-ca\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.424570 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.424461    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/763faa91-078d-456f-9daf-77ee5acc9aca-root\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.424570 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.424560    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-wtmp\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.424893 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.424609    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d171f5a-aa91-4614-9e29-c3e5b59fdba3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9d44df66c-258jh\" (UID: \"9d171f5a-aa91-4614-9e29-c3e5b59fdba3\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.424893 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.424699    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-accelerators-collector-config\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.425025 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.425008    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d171f5a-aa91-4614-9e29-c3e5b59fdba3-metrics-client-ca\") pod \"openshift-state-metrics-9d44df66c-258jh\" (UID: \"9d171f5a-aa91-4614-9e29-c3e5b59fdba3\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.426777 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.426756    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d171f5a-aa91-4614-9e29-c3e5b59fdba3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9d44df66c-258jh\" (UID: \"9d171f5a-aa91-4614-9e29-c3e5b59fdba3\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.435237 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.435204    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr7xw\" (UniqueName: \"kubernetes.io/projected/9d171f5a-aa91-4614-9e29-c3e5b59fdba3-kube-api-access-gr7xw\") pod \"openshift-state-metrics-9d44df66c-258jh\" (UID: \"9d171f5a-aa91-4614-9e29-c3e5b59fdba3\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.435702 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.435681    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d171f5a-aa91-4614-9e29-c3e5b59fdba3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9d44df66c-258jh\" (UID: \"9d171f5a-aa91-4614-9e29-c3e5b59fdba3\") " pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.475420 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.475388    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"
Apr 17 07:54:43.528262 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.527758    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.528262 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.527807    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-tls\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.528262 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.527846    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/763faa91-078d-456f-9daf-77ee5acc9aca-metrics-client-ca\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.528262 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.527871    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/763faa91-078d-456f-9daf-77ee5acc9aca-root\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.528262 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.527894    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-wtmp\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.528262 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.527970    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-accelerators-collector-config\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.528262 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.528016    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sgwwr\" (UniqueName: \"kubernetes.io/projected/763faa91-078d-456f-9daf-77ee5acc9aca-kube-api-access-sgwwr\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.528262 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.528041    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/763faa91-078d-456f-9daf-77ee5acc9aca-sys\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.528262 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.528073    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-textfile\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.529228 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.528834    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/763faa91-078d-456f-9daf-77ee5acc9aca-root\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.529228 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.529037    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-textfile\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.529228 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.529186    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-wtmp\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.530573 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.529717    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-accelerators-collector-config\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.530573 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.530084    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/763faa91-078d-456f-9daf-77ee5acc9aca-sys\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.530573 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.530531    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/763faa91-078d-456f-9daf-77ee5acc9aca-metrics-client-ca\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.534004 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.533963    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.536148 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.536123    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/763faa91-078d-456f-9daf-77ee5acc9aca-node-exporter-tls\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.559027 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.558959    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgwwr\" (UniqueName: \"kubernetes.io/projected/763faa91-078d-456f-9daf-77ee5acc9aca-kube-api-access-sgwwr\") pod \"node-exporter-b7k7k\" (UID: \"763faa91-078d-456f-9daf-77ee5acc9aca\") " pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.629844 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.629814    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b7k7k"
Apr 17 07:54:43.637526 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:54:43.637457    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod763faa91_078d_456f_9daf_77ee5acc9aca.slice/crio-c1af139d5a3bf57fb79fbaf9489155ebdabfa7a55ac55a55f136e4dd4d6297f9 WatchSource:0}: Error finding container c1af139d5a3bf57fb79fbaf9489155ebdabfa7a55ac55a55f136e4dd4d6297f9: Status 404 returned error can't find the container with id c1af139d5a3bf57fb79fbaf9489155ebdabfa7a55ac55a55f136e4dd4d6297f9
Apr 17 07:54:43.639298 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:43.639173    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9d44df66c-258jh"]
Apr 17 07:54:43.642163 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:54:43.642138    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d171f5a_aa91_4614_9e29_c3e5b59fdba3.slice/crio-4e3d59aeb1388823b16af40f70782fa9d8d289a4fe77c22318337c0d6fe76907 WatchSource:0}: Error finding container 4e3d59aeb1388823b16af40f70782fa9d8d289a4fe77c22318337c0d6fe76907: Status 404 returned error can't find the container with id 4e3d59aeb1388823b16af40f70782fa9d8d289a4fe77c22318337c0d6fe76907
Apr 17 07:54:44.086605 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:44.086570    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b7k7k" event={"ID":"763faa91-078d-456f-9daf-77ee5acc9aca","Type":"ContainerStarted","Data":"c1af139d5a3bf57fb79fbaf9489155ebdabfa7a55ac55a55f136e4dd4d6297f9"}
Apr 17 07:54:44.088272 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:44.088238    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh" event={"ID":"9d171f5a-aa91-4614-9e29-c3e5b59fdba3","Type":"ContainerStarted","Data":"937d4a6525914ef910723cd95065a1fed93f20a596ab58366183da39350ca707"}
Apr 17 07:54:44.088417 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:44.088278    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh" event={"ID":"9d171f5a-aa91-4614-9e29-c3e5b59fdba3","Type":"ContainerStarted","Data":"c3cdc9d435aea6e3f75567de7019875d896c173a1121a4b2280e9a316ee43df5"}
Apr 17 07:54:44.088417 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:44.088293    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh" event={"ID":"9d171f5a-aa91-4614-9e29-c3e5b59fdba3","Type":"ContainerStarted","Data":"4e3d59aeb1388823b16af40f70782fa9d8d289a4fe77c22318337c0d6fe76907"}
Apr 17 07:54:44.851672 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:54:44.851630    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jjcmh" podUID="b99d5d29-1f8b-46e2-b121-000bc5b77dfc"
Apr 17 07:54:44.861888 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:54:44.861852    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-qqmvg" podUID="68a69e91-436d-4788-b2de-32c453ccd36a"
Apr 17 07:54:45.092557 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.092518    2571 generic.go:358] "Generic (PLEG): container finished" podID="763faa91-078d-456f-9daf-77ee5acc9aca" containerID="b2c3ea48435ac20c11392cd9622ccea9278c667c42cc483e5dc2074e287fd05f" exitCode=0
Apr 17 07:54:45.092734 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.092601    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b7k7k" event={"ID":"763faa91-078d-456f-9daf-77ee5acc9aca","Type":"ContainerDied","Data":"b2c3ea48435ac20c11392cd9622ccea9278c667c42cc483e5dc2074e287fd05f"}
Apr 17 07:54:45.094490 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.094457    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh" event={"ID":"9d171f5a-aa91-4614-9e29-c3e5b59fdba3","Type":"ContainerStarted","Data":"4bd1ca6b6dde7f921ae53bb1295a34e9e3cafcec93961a58c698937d474a185c"}
Apr 17 07:54:45.094490 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.094479    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:54:45.135706 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.135665    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-9d44df66c-258jh" podStartSLOduration=1.215781876 podStartE2EDuration="2.135650475s" podCreationTimestamp="2026-04-17 07:54:43 +0000 UTC" firstStartedPulling="2026-04-17 07:54:43.759988453 +0000 UTC m=+154.813591642" lastFinishedPulling="2026-04-17 07:54:44.679857052 +0000 UTC m=+155.733460241" observedRunningTime="2026-04-17 07:54:45.134353883 +0000 UTC m=+156.187957091" watchObservedRunningTime="2026-04-17 07:54:45.135650475 +0000 UTC m=+156.189253710"
Apr 17 07:54:45.273634 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.273600    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-56684c54b9-49k8p"]
Apr 17 07:54:45.276795 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.276781    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.282038 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.282017    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Apr 17 07:54:45.282166 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.282041    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rmk2w\""
Apr 17 07:54:45.282166 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.282136    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Apr 17 07:54:45.282277 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.282175    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Apr 17 07:54:45.282277 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.282232    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Apr 17 07:54:45.282374 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.282326    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Apr 17 07:54:45.282507 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.282492    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-52ftkdp78tjvd\""
Apr 17 07:54:45.300820 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.300799    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56684c54b9-49k8p"]
Apr 17 07:54:45.368031 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.367958    2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w" podUID="89477371-9b36-4af7-aff9-7c8ba8ced0ea" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Apr 17 07:54:45.368031 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.368016    2571 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w"
Apr 17 07:54:45.368479 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.368448    2571 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"cb10facb051f7ede94df013d35a53f3863a269580ed1e14e3d34582ddb5d0de0"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Apr 17 07:54:45.368528 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.368498    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w" podUID="89477371-9b36-4af7-aff9-7c8ba8ced0ea" containerName="service-proxy" containerID="cri-o://cb10facb051f7ede94df013d35a53f3863a269580ed1e14e3d34582ddb5d0de0" gracePeriod=30
Apr 17 07:54:45.446102 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.446068    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.446204 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.446108    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-grpc-tls\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.446204 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.446135    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-tls\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.446204 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.446155    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp5dz\" (UniqueName: \"kubernetes.io/projected/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-kube-api-access-zp5dz\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.446344 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.446234    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.446344 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.446277    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.446344 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.446314    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.446344 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.446333    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-metrics-client-ca\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.547339 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.547312    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-metrics-client-ca\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.547498 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.547459    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.547498 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.547491    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-grpc-tls\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.547613 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.547534    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-tls\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.547613 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.547582    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zp5dz\" (UniqueName: \"kubernetes.io/projected/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-kube-api-access-zp5dz\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.547718 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.547617    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.547780 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.547692    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.547829 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.547802    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.548098 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.548071    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-metrics-client-ca\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.551103 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.550602    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-tls\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.551103 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.550862    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.551103 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.551033    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.551103 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.551067    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-grpc-tls\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.551365 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.551205    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.551549 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.551522    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.555260 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.555240    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp5dz\" (UniqueName: \"kubernetes.io/projected/8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23-kube-api-access-zp5dz\") pod \"thanos-querier-56684c54b9-49k8p\" (UID: \"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23\") " pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.585309 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.585285    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:45.702132 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:45.702104    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56684c54b9-49k8p"]
Apr 17 07:54:45.704239 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:54:45.704210    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d6e4e5e_1d41_4f7c_b780_9a2bd9a98a23.slice/crio-7a3d161c9e6a9bcc50c86e8ee88807433c93e02a3e9aa065e99352992d1e9091 WatchSource:0}: Error finding container 7a3d161c9e6a9bcc50c86e8ee88807433c93e02a3e9aa065e99352992d1e9091: Status 404 returned error can't find the container with id 7a3d161c9e6a9bcc50c86e8ee88807433c93e02a3e9aa065e99352992d1e9091
Apr 17 07:54:46.099928 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:46.099895    2571 generic.go:358] "Generic (PLEG): container finished" podID="89477371-9b36-4af7-aff9-7c8ba8ced0ea" containerID="cb10facb051f7ede94df013d35a53f3863a269580ed1e14e3d34582ddb5d0de0" exitCode=2
Apr 17 07:54:46.100406 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:46.099934    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w" event={"ID":"89477371-9b36-4af7-aff9-7c8ba8ced0ea","Type":"ContainerDied","Data":"cb10facb051f7ede94df013d35a53f3863a269580ed1e14e3d34582ddb5d0de0"}
Apr 17 07:54:46.100406 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:46.099983    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b67b764f8-rvc8w" event={"ID":"89477371-9b36-4af7-aff9-7c8ba8ced0ea","Type":"ContainerStarted","Data":"049ab69b6911d04561a4d25c16fa0bd860f5f2f272f589dd5ca9e0664bba2501"}
Apr 17 07:54:46.101664 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:46.101633    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p" event={"ID":"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23","Type":"ContainerStarted","Data":"7a3d161c9e6a9bcc50c86e8ee88807433c93e02a3e9aa065e99352992d1e9091"}
Apr 17 07:54:46.103589 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:46.103557    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b7k7k" event={"ID":"763faa91-078d-456f-9daf-77ee5acc9aca","Type":"ContainerStarted","Data":"ffb99bd20b21ecff15cf9fabbd39a601758291aa0eb78fd4ebe6d027a2c7a1c5"}
Apr 17 07:54:46.103589 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:46.103587    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b7k7k" event={"ID":"763faa91-078d-456f-9daf-77ee5acc9aca","Type":"ContainerStarted","Data":"198538b67644839b439987dd96a512902233832b79f55a55d701962e8218dcbf"}
Apr 17 07:54:46.143986 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:46.143930    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b7k7k" podStartSLOduration=2.105083358 podStartE2EDuration="3.143910394s" podCreationTimestamp="2026-04-17 07:54:43 +0000 UTC" firstStartedPulling="2026-04-17 07:54:43.639703771 +0000 UTC m=+154.693306964" lastFinishedPulling="2026-04-17 07:54:44.678530796 +0000 UTC m=+155.732134000" observedRunningTime="2026-04-17 07:54:46.143011029 +0000 UTC m=+157.196614309" watchObservedRunningTime="2026-04-17 07:54:46.143910394 +0000 UTC m=+157.197513606"
Apr 17 07:54:48.113107 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:48.113066    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p" event={"ID":"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23","Type":"ContainerStarted","Data":"78aba9c011dfb5aa90cacf1c64e0cca367a3a6d4ccfe1918b6be0d22c01a008e"}
Apr 17 07:54:48.113107 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:48.113118    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p" event={"ID":"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23","Type":"ContainerStarted","Data":"0f8baf390ed560e52dbd5d87984ed860830e68b51395d4973fbfbc4a7be1d9e6"}
Apr 17 07:54:48.113627 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:48.113133    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p" event={"ID":"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23","Type":"ContainerStarted","Data":"f859c2b12a6cd53f31dfaa891cfb524bafbf08d8239ddd66e467d428da4d5963"}
Apr 17 07:54:49.118262 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.118223    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p" event={"ID":"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23","Type":"ContainerStarted","Data":"3926ded760db3ad885cb212d8cca8c5a841a4c29892ab4c76a13e781d280d2aa"}
Apr 17 07:54:49.118677 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.118266    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p" event={"ID":"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23","Type":"ContainerStarted","Data":"a192f027f50ac723b86b6cd591edf45728c9ae15aa16cb0af42621e7c1b1120e"}
Apr 17 07:54:49.118677 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.118290    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p" event={"ID":"8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23","Type":"ContainerStarted","Data":"43de0cbbe201ef9c43b68e1fc54fc1d42ecd039ffa3ba5ee4f521936b00bb530"}
Apr 17 07:54:49.118677 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.118456    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:49.148518 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.148477    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p" podStartSLOduration=1.680016291 podStartE2EDuration="4.148464416s" podCreationTimestamp="2026-04-17 07:54:45 +0000 UTC" firstStartedPulling="2026-04-17 07:54:45.706117116 +0000 UTC m=+156.759720306" lastFinishedPulling="2026-04-17 07:54:48.174565237 +0000 UTC m=+159.228168431" observedRunningTime="2026-04-17 07:54:49.14653192 +0000 UTC m=+160.200135132" watchObservedRunningTime="2026-04-17 07:54:49.148464416 +0000 UTC m=+160.202067627"
Apr 17 07:54:49.785485 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.785419    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:54:49.785485 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.785488    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:54:49.787896 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.787870    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b99d5d29-1f8b-46e2-b121-000bc5b77dfc-metrics-tls\") pod \"dns-default-jjcmh\" (UID: \"b99d5d29-1f8b-46e2-b121-000bc5b77dfc\") " pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:54:49.788019 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.788000    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a69e91-436d-4788-b2de-32c453ccd36a-cert\") pod \"ingress-canary-qqmvg\" (UID: \"68a69e91-436d-4788-b2de-32c453ccd36a\") " pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:54:49.854593 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.854559    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f657c46f6-kllxp"]
Apr 17 07:54:49.858830 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.858812    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:49.866886 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.866865    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Apr 17 07:54:49.872178 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.872155    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f657c46f6-kllxp"]
Apr 17 07:54:49.897809 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.897788    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xs2k6\""
Apr 17 07:54:49.906310 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.906288    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:54:49.986927 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.986886    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-trusted-ca-bundle\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:49.987050 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.986978    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-oauth-serving-cert\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:49.987050 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.987017    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-oauth-config\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:49.987050 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.987033    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-config\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:49.987203 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.987053    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2swt\" (UniqueName: \"kubernetes.io/projected/f23f7f76-f35f-4f48-b474-6d21c66886d3-kube-api-access-k2swt\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:49.987203 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.987136    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-service-ca\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:49.987203 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:49.987180    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-serving-cert\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.024287 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.024257    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jjcmh"]
Apr 17 07:54:50.088398 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.088324    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-oauth-serving-cert\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.088398 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.088364    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-oauth-config\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.088398 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.088381    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-config\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.088611 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.088504    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k2swt\" (UniqueName: \"kubernetes.io/projected/f23f7f76-f35f-4f48-b474-6d21c66886d3-kube-api-access-k2swt\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.088611 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.088548    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-service-ca\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.088611 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.088584    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-serving-cert\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.088719 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.088639    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-trusted-ca-bundle\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.089099 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.089077    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-config\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.089236 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.089128    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-oauth-serving-cert\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.089236 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.089148    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-service-ca\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.089556 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.089536    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-trusted-ca-bundle\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.090835 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.090817    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-oauth-config\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.090956 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.090937    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-serving-cert\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.096400 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.096380    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2swt\" (UniqueName: \"kubernetes.io/projected/f23f7f76-f35f-4f48-b474-6d21c66886d3-kube-api-access-k2swt\") pod \"console-5f657c46f6-kllxp\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") " pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.122467 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.122418    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jjcmh" event={"ID":"b99d5d29-1f8b-46e2-b121-000bc5b77dfc","Type":"ContainerStarted","Data":"83b01e52b5ef3f12435fd88f4d7c3819870fe7bb7c216a5da82156d6f8befecc"}
Apr 17 07:54:50.168068 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.168033    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:54:50.280452 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:50.280410    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f657c46f6-kllxp"]
Apr 17 07:54:50.283205 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:54:50.283178    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf23f7f76_f35f_4f48_b474_6d21c66886d3.slice/crio-43f3d233db05f95a40b7b0d15287a55e862fcb6b312dfb86bf9f53c61cff63ba WatchSource:0}: Error finding container 43f3d233db05f95a40b7b0d15287a55e862fcb6b312dfb86bf9f53c61cff63ba: Status 404 returned error can't find the container with id 43f3d233db05f95a40b7b0d15287a55e862fcb6b312dfb86bf9f53c61cff63ba
Apr 17 07:54:51.126407 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:51.126362    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f657c46f6-kllxp" event={"ID":"f23f7f76-f35f-4f48-b474-6d21c66886d3","Type":"ContainerStarted","Data":"7a43fa33295c129d5cab79dc140f9da66be06aed1def82f686f05791eba5617e"}
Apr 17 07:54:51.126407 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:51.126404    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f657c46f6-kllxp" event={"ID":"f23f7f76-f35f-4f48-b474-6d21c66886d3","Type":"ContainerStarted","Data":"43f3d233db05f95a40b7b0d15287a55e862fcb6b312dfb86bf9f53c61cff63ba"}
Apr 17 07:54:51.144632 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:51.144584    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5f657c46f6-kllxp" podStartSLOduration=2.144566911 podStartE2EDuration="2.144566911s" podCreationTimestamp="2026-04-17 07:54:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-17 07:54:51.142244898 +0000 UTC m=+162.195848110" watchObservedRunningTime="2026-04-17 07:54:51.144566911 +0000 UTC m=+162.198170124"
Apr 17 07:54:52.130923 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:52.130883    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jjcmh" event={"ID":"b99d5d29-1f8b-46e2-b121-000bc5b77dfc","Type":"ContainerStarted","Data":"65a019bbcb9394340f2820560d1212e8c57d0de71693955f1f587139da73931d"}
Apr 17 07:54:52.130923 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:52.130923    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jjcmh" event={"ID":"b99d5d29-1f8b-46e2-b121-000bc5b77dfc","Type":"ContainerStarted","Data":"9916d53622f7d35986284b1ceccefc401ea905369c942434af0f6a0db30633a3"}
Apr 17 07:54:52.160952 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:52.160900    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-jjcmh" podStartSLOduration=129.909314808 podStartE2EDuration="2m11.160886763s" podCreationTimestamp="2026-04-17 07:52:41 +0000 UTC" firstStartedPulling="2026-04-17 07:54:50.030615599 +0000 UTC m=+161.084218788" lastFinishedPulling="2026-04-17 07:54:51.282187539 +0000 UTC m=+162.335790743" observedRunningTime="2026-04-17 07:54:52.160126863 +0000 UTC m=+163.213730074" watchObservedRunningTime="2026-04-17 07:54:52.160886763 +0000 UTC m=+163.214490007"
Apr 17 07:54:53.133946 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.133917    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:54:53.195888 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.195858    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f657c46f6-kllxp"]
Apr 17 07:54:53.248996 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.248958    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bcc59b8f7-bp4d9"]
Apr 17 07:54:53.252115 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.252100    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.291785 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.291758    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bcc59b8f7-bp4d9"]
Apr 17 07:54:53.313782 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.313754    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-trusted-ca-bundle\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.313782 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.313783    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-config\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.313928 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.313803    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-serving-cert\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.313928 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.313829    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6pmb\" (UniqueName: \"kubernetes.io/projected/fa5c103a-608e-4786-a52a-f85ff80e4bdc-kube-api-access-k6pmb\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.313928 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.313856    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-oauth-serving-cert\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.313928 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.313883    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-oauth-config\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.314037 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.313933    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-service-ca\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.415084 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.415007    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-trusted-ca-bundle\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.415084 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.415038    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-config\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.415084 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.415059    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-serving-cert\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.415084 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.415075    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k6pmb\" (UniqueName: \"kubernetes.io/projected/fa5c103a-608e-4786-a52a-f85ff80e4bdc-kube-api-access-k6pmb\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.415370 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.415093    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-oauth-serving-cert\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.415370 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.415121    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-oauth-config\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.415370 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.415154    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-service-ca\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.415958 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.415935    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-service-ca\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.416082 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.415962    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-config\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.416082 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.416005    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-trusted-ca-bundle\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.416082 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.416035    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-oauth-serving-cert\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.417563 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.417541    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-oauth-config\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.417761 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.417741    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-serving-cert\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.423733 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.423713    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6pmb\" (UniqueName: \"kubernetes.io/projected/fa5c103a-608e-4786-a52a-f85ff80e4bdc-kube-api-access-k6pmb\") pod \"console-6bcc59b8f7-bp4d9\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") " pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.560297 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.560262    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:54:53.677223 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:53.677148    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bcc59b8f7-bp4d9"]
Apr 17 07:54:53.680337 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:54:53.680314    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa5c103a_608e_4786_a52a_f85ff80e4bdc.slice/crio-38464664f9b86ec23ead6ba8ca3fa4c8b6f264afd00c8cbf17aa21069b647772 WatchSource:0}: Error finding container 38464664f9b86ec23ead6ba8ca3fa4c8b6f264afd00c8cbf17aa21069b647772: Status 404 returned error can't find the container with id 38464664f9b86ec23ead6ba8ca3fa4c8b6f264afd00c8cbf17aa21069b647772
Apr 17 07:54:54.138572 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:54.138525    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bcc59b8f7-bp4d9" event={"ID":"fa5c103a-608e-4786-a52a-f85ff80e4bdc","Type":"ContainerStarted","Data":"f4d8e5dfc318c3369f3ac3430ea9d399a4f5a351c8a3825d31294f07fba7ce98"}
Apr 17 07:54:54.138572 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:54.138577    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bcc59b8f7-bp4d9" event={"ID":"fa5c103a-608e-4786-a52a-f85ff80e4bdc","Type":"ContainerStarted","Data":"38464664f9b86ec23ead6ba8ca3fa4c8b6f264afd00c8cbf17aa21069b647772"}
Apr 17 07:54:54.163862 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:54.163817    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6bcc59b8f7-bp4d9" podStartSLOduration=1.163804998 podStartE2EDuration="1.163804998s" podCreationTimestamp="2026-04-17 07:54:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-17 07:54:54.163199012 +0000 UTC m=+165.216802225" watchObservedRunningTime="2026-04-17 07:54:54.163804998 +0000 UTC m=+165.217408210"
Apr 17 07:54:55.129266 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:55.129238    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-56684c54b9-49k8p"
Apr 17 07:54:59.540475 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:59.540389    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:54:59.543092 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:59.543066    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wmmwx\""
Apr 17 07:54:59.551131 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:59.551114    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qqmvg"
Apr 17 07:54:59.668554 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:54:59.668359    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qqmvg"]
Apr 17 07:54:59.671017 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:54:59.670989    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a69e91_436d_4788_b2de_32c453ccd36a.slice/crio-fb82493dd87490b03892738dd1679fcb90044e36250cb08ed506085e7b8ffa1a WatchSource:0}: Error finding container fb82493dd87490b03892738dd1679fcb90044e36250cb08ed506085e7b8ffa1a: Status 404 returned error can't find the container with id fb82493dd87490b03892738dd1679fcb90044e36250cb08ed506085e7b8ffa1a
Apr 17 07:55:00.158502 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:00.158465    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qqmvg" event={"ID":"68a69e91-436d-4788-b2de-32c453ccd36a","Type":"ContainerStarted","Data":"fb82493dd87490b03892738dd1679fcb90044e36250cb08ed506085e7b8ffa1a"}
Apr 17 07:55:00.168808 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:00.168781    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:55:02.165372 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:02.165336    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qqmvg" event={"ID":"68a69e91-436d-4788-b2de-32c453ccd36a","Type":"ContainerStarted","Data":"bc8bf42e2af7518c81d516532245c9dff3719cd086b988ae9ec4f9411d4c500b"}
Apr 17 07:55:02.186208 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:02.186155    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qqmvg" podStartSLOduration=139.597904488 podStartE2EDuration="2m21.18614135s" podCreationTimestamp="2026-04-17 07:52:41 +0000 UTC" firstStartedPulling="2026-04-17 07:54:59.672910554 +0000 UTC m=+170.726513744" lastFinishedPulling="2026-04-17 07:55:01.261147402 +0000 UTC m=+172.314750606" observedRunningTime="2026-04-17 07:55:02.185063318 +0000 UTC m=+173.238666529" watchObservedRunningTime="2026-04-17 07:55:02.18614135 +0000 UTC m=+173.239744560"
Apr 17 07:55:03.140946 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:03.140917    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jjcmh"
Apr 17 07:55:03.560380 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:03.560341    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:55:03.560846 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:03.560465    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:55:03.565244 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:03.565224    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:55:04.174455 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:04.174408    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:55:04.216806 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:04.216771    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-647ffb494d-9p7x5"]
Apr 17 07:55:18.214689 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.214635    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-5f657c46f6-kllxp" podUID="f23f7f76-f35f-4f48-b474-6d21c66886d3" containerName="console" containerID="cri-o://7a43fa33295c129d5cab79dc140f9da66be06aed1def82f686f05791eba5617e" gracePeriod=15
Apr 17 07:55:18.456567 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.456545    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5f657c46f6-kllxp_f23f7f76-f35f-4f48-b474-6d21c66886d3/console/0.log"
Apr 17 07:55:18.456673 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.456616    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:55:18.502890 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.502806    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2swt\" (UniqueName: \"kubernetes.io/projected/f23f7f76-f35f-4f48-b474-6d21c66886d3-kube-api-access-k2swt\") pod \"f23f7f76-f35f-4f48-b474-6d21c66886d3\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") "
Apr 17 07:55:18.502890 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.502852    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-trusted-ca-bundle\") pod \"f23f7f76-f35f-4f48-b474-6d21c66886d3\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") "
Apr 17 07:55:18.502890 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.502879    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-oauth-serving-cert\") pod \"f23f7f76-f35f-4f48-b474-6d21c66886d3\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") "
Apr 17 07:55:18.503150 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.502908    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-config\") pod \"f23f7f76-f35f-4f48-b474-6d21c66886d3\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") "
Apr 17 07:55:18.503150 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.502961    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-oauth-config\") pod \"f23f7f76-f35f-4f48-b474-6d21c66886d3\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") "
Apr 17 07:55:18.503150 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.502993    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-service-ca\") pod \"f23f7f76-f35f-4f48-b474-6d21c66886d3\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") "
Apr 17 07:55:18.503150 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.503019    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-serving-cert\") pod \"f23f7f76-f35f-4f48-b474-6d21c66886d3\" (UID: \"f23f7f76-f35f-4f48-b474-6d21c66886d3\") "
Apr 17 07:55:18.503366 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.503337    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f23f7f76-f35f-4f48-b474-6d21c66886d3" (UID: "f23f7f76-f35f-4f48-b474-6d21c66886d3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 17 07:55:18.503422 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.503386    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-service-ca" (OuterVolumeSpecName: "service-ca") pod "f23f7f76-f35f-4f48-b474-6d21c66886d3" (UID: "f23f7f76-f35f-4f48-b474-6d21c66886d3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 17 07:55:18.503422 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.503398    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f23f7f76-f35f-4f48-b474-6d21c66886d3" (UID: "f23f7f76-f35f-4f48-b474-6d21c66886d3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 17 07:55:18.503422 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.503394    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-config" (OuterVolumeSpecName: "console-config") pod "f23f7f76-f35f-4f48-b474-6d21c66886d3" (UID: "f23f7f76-f35f-4f48-b474-6d21c66886d3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 17 07:55:18.505253 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.505225    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f23f7f76-f35f-4f48-b474-6d21c66886d3" (UID: "f23f7f76-f35f-4f48-b474-6d21c66886d3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 07:55:18.505410 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.505394    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f23f7f76-f35f-4f48-b474-6d21c66886d3-kube-api-access-k2swt" (OuterVolumeSpecName: "kube-api-access-k2swt") pod "f23f7f76-f35f-4f48-b474-6d21c66886d3" (UID: "f23f7f76-f35f-4f48-b474-6d21c66886d3"). InnerVolumeSpecName "kube-api-access-k2swt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 07:55:18.505410 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.505395    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f23f7f76-f35f-4f48-b474-6d21c66886d3" (UID: "f23f7f76-f35f-4f48-b474-6d21c66886d3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 07:55:18.604309 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.604253    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2swt\" (UniqueName: \"kubernetes.io/projected/f23f7f76-f35f-4f48-b474-6d21c66886d3-kube-api-access-k2swt\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:55:18.604309 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.604300    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-trusted-ca-bundle\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:55:18.604309 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.604311    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-oauth-serving-cert\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:55:18.604309 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.604320    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-config\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:55:18.604309 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.604330    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-oauth-config\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:55:18.604630 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.604340    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f23f7f76-f35f-4f48-b474-6d21c66886d3-service-ca\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:55:18.604630 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:18.604350    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f23f7f76-f35f-4f48-b474-6d21c66886d3-console-serving-cert\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:55:19.211472 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:19.211447    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5f657c46f6-kllxp_f23f7f76-f35f-4f48-b474-6d21c66886d3/console/0.log"
Apr 17 07:55:19.211630 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:19.211486    2571 generic.go:358] "Generic (PLEG): container finished" podID="f23f7f76-f35f-4f48-b474-6d21c66886d3" containerID="7a43fa33295c129d5cab79dc140f9da66be06aed1def82f686f05791eba5617e" exitCode=2
Apr 17 07:55:19.211630 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:19.211524    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f657c46f6-kllxp" event={"ID":"f23f7f76-f35f-4f48-b474-6d21c66886d3","Type":"ContainerDied","Data":"7a43fa33295c129d5cab79dc140f9da66be06aed1def82f686f05791eba5617e"}
Apr 17 07:55:19.211630 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:19.211546    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f657c46f6-kllxp" event={"ID":"f23f7f76-f35f-4f48-b474-6d21c66886d3","Type":"ContainerDied","Data":"43f3d233db05f95a40b7b0d15287a55e862fcb6b312dfb86bf9f53c61cff63ba"}
Apr 17 07:55:19.211630 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:19.211545    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f657c46f6-kllxp"
Apr 17 07:55:19.211630 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:19.211560    2571 scope.go:117] "RemoveContainer" containerID="7a43fa33295c129d5cab79dc140f9da66be06aed1def82f686f05791eba5617e"
Apr 17 07:55:19.220369 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:19.220290    2571 scope.go:117] "RemoveContainer" containerID="7a43fa33295c129d5cab79dc140f9da66be06aed1def82f686f05791eba5617e"
Apr 17 07:55:19.220646 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:55:19.220548    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a43fa33295c129d5cab79dc140f9da66be06aed1def82f686f05791eba5617e\": container with ID starting with 7a43fa33295c129d5cab79dc140f9da66be06aed1def82f686f05791eba5617e not found: ID does not exist" containerID="7a43fa33295c129d5cab79dc140f9da66be06aed1def82f686f05791eba5617e"
Apr 17 07:55:19.220646 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:19.220572    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a43fa33295c129d5cab79dc140f9da66be06aed1def82f686f05791eba5617e"} err="failed to get container status \"7a43fa33295c129d5cab79dc140f9da66be06aed1def82f686f05791eba5617e\": rpc error: code = NotFound desc = could not find container \"7a43fa33295c129d5cab79dc140f9da66be06aed1def82f686f05791eba5617e\": container with ID starting with 7a43fa33295c129d5cab79dc140f9da66be06aed1def82f686f05791eba5617e not found: ID does not exist"
Apr 17 07:55:19.233326 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:19.233304    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f657c46f6-kllxp"]
Apr 17 07:55:19.237295 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:19.237274    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f657c46f6-kllxp"]
Apr 17 07:55:19.542446 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:19.542408    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f23f7f76-f35f-4f48-b474-6d21c66886d3" path="/var/lib/kubelet/pods/f23f7f76-f35f-4f48-b474-6d21c66886d3/volumes"
Apr 17 07:55:29.237926 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.237865    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-647ffb494d-9p7x5" podUID="e13695f3-4327-46c1-b4d1-4225aa82f4bb" containerName="console" containerID="cri-o://ab92f7841c370d68dd04858c498d998829eaec58812d954a1121af7c78dee68e" gracePeriod=15
Apr 17 07:55:29.466681 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.466661    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-647ffb494d-9p7x5_e13695f3-4327-46c1-b4d1-4225aa82f4bb/console/0.log"
Apr 17 07:55:29.466787 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.466720    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:55:29.581489 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.581458    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-service-ca\") pod \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") "
Apr 17 07:55:29.581660 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.581494    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-oauth-serving-cert\") pod \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") "
Apr 17 07:55:29.581660 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.581523    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-config\") pod \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") "
Apr 17 07:55:29.581660 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.581595    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-serving-cert\") pod \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") "
Apr 17 07:55:29.581660 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.581629    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-oauth-config\") pod \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") "
Apr 17 07:55:29.581660 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.581656    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wftr9\" (UniqueName: \"kubernetes.io/projected/e13695f3-4327-46c1-b4d1-4225aa82f4bb-kube-api-access-wftr9\") pod \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\" (UID: \"e13695f3-4327-46c1-b4d1-4225aa82f4bb\") "
Apr 17 07:55:29.581911 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.581866    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-service-ca" (OuterVolumeSpecName: "service-ca") pod "e13695f3-4327-46c1-b4d1-4225aa82f4bb" (UID: "e13695f3-4327-46c1-b4d1-4225aa82f4bb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 17 07:55:29.581911 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.581885    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e13695f3-4327-46c1-b4d1-4225aa82f4bb" (UID: "e13695f3-4327-46c1-b4d1-4225aa82f4bb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 17 07:55:29.582025 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.581928    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-config" (OuterVolumeSpecName: "console-config") pod "e13695f3-4327-46c1-b4d1-4225aa82f4bb" (UID: "e13695f3-4327-46c1-b4d1-4225aa82f4bb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 17 07:55:29.583774 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.583749    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e13695f3-4327-46c1-b4d1-4225aa82f4bb-kube-api-access-wftr9" (OuterVolumeSpecName: "kube-api-access-wftr9") pod "e13695f3-4327-46c1-b4d1-4225aa82f4bb" (UID: "e13695f3-4327-46c1-b4d1-4225aa82f4bb"). InnerVolumeSpecName "kube-api-access-wftr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 07:55:29.583774 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.583768    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e13695f3-4327-46c1-b4d1-4225aa82f4bb" (UID: "e13695f3-4327-46c1-b4d1-4225aa82f4bb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 07:55:29.583974 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.583791    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e13695f3-4327-46c1-b4d1-4225aa82f4bb" (UID: "e13695f3-4327-46c1-b4d1-4225aa82f4bb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 07:55:29.683133 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.683098    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-serving-cert\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:55:29.683133 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.683127    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-oauth-config\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:55:29.683133 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.683138    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wftr9\" (UniqueName: \"kubernetes.io/projected/e13695f3-4327-46c1-b4d1-4225aa82f4bb-kube-api-access-wftr9\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:55:29.683357 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.683148    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-service-ca\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:55:29.683357 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.683157    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-oauth-serving-cert\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:55:29.683357 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:29.683166    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e13695f3-4327-46c1-b4d1-4225aa82f4bb-console-config\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:55:30.244576 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:30.244550    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-647ffb494d-9p7x5_e13695f3-4327-46c1-b4d1-4225aa82f4bb/console/0.log"
Apr 17 07:55:30.244971 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:30.244587    2571 generic.go:358] "Generic (PLEG): container finished" podID="e13695f3-4327-46c1-b4d1-4225aa82f4bb" containerID="ab92f7841c370d68dd04858c498d998829eaec58812d954a1121af7c78dee68e" exitCode=2
Apr 17 07:55:30.244971 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:30.244640    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-647ffb494d-9p7x5" event={"ID":"e13695f3-4327-46c1-b4d1-4225aa82f4bb","Type":"ContainerDied","Data":"ab92f7841c370d68dd04858c498d998829eaec58812d954a1121af7c78dee68e"}
Apr 17 07:55:30.244971 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:30.244660    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-647ffb494d-9p7x5" event={"ID":"e13695f3-4327-46c1-b4d1-4225aa82f4bb","Type":"ContainerDied","Data":"733e7364e224dbb6a62f3e06b268ab16de72e335d17f0a12d092aad34533c695"}
Apr 17 07:55:30.244971 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:30.244674    2571 scope.go:117] "RemoveContainer" containerID="ab92f7841c370d68dd04858c498d998829eaec58812d954a1121af7c78dee68e"
Apr 17 07:55:30.244971 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:30.244676    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-647ffb494d-9p7x5"
Apr 17 07:55:30.253283 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:30.253265    2571 scope.go:117] "RemoveContainer" containerID="ab92f7841c370d68dd04858c498d998829eaec58812d954a1121af7c78dee68e"
Apr 17 07:55:30.253533 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:55:30.253515    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab92f7841c370d68dd04858c498d998829eaec58812d954a1121af7c78dee68e\": container with ID starting with ab92f7841c370d68dd04858c498d998829eaec58812d954a1121af7c78dee68e not found: ID does not exist" containerID="ab92f7841c370d68dd04858c498d998829eaec58812d954a1121af7c78dee68e"
Apr 17 07:55:30.253586 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:30.253542    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab92f7841c370d68dd04858c498d998829eaec58812d954a1121af7c78dee68e"} err="failed to get container status \"ab92f7841c370d68dd04858c498d998829eaec58812d954a1121af7c78dee68e\": rpc error: code = NotFound desc = could not find container \"ab92f7841c370d68dd04858c498d998829eaec58812d954a1121af7c78dee68e\": container with ID starting with ab92f7841c370d68dd04858c498d998829eaec58812d954a1121af7c78dee68e not found: ID does not exist"
Apr 17 07:55:30.266195 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:30.266174    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-647ffb494d-9p7x5"]
Apr 17 07:55:30.269688 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:30.269671    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-647ffb494d-9p7x5"]
Apr 17 07:55:31.542141 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:55:31.542104    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e13695f3-4327-46c1-b4d1-4225aa82f4bb" path="/var/lib/kubelet/pods/e13695f3-4327-46c1-b4d1-4225aa82f4bb/volumes"
Apr 17 07:56:21.732079 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:21.732046    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bcc59b8f7-bp4d9"]
Apr 17 07:56:46.750744 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:46.750680    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-6bcc59b8f7-bp4d9" podUID="fa5c103a-608e-4786-a52a-f85ff80e4bdc" containerName="console" containerID="cri-o://f4d8e5dfc318c3369f3ac3430ea9d399a4f5a351c8a3825d31294f07fba7ce98" gracePeriod=15
Apr 17 07:56:46.980550 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:46.980526    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6bcc59b8f7-bp4d9_fa5c103a-608e-4786-a52a-f85ff80e4bdc/console/0.log"
Apr 17 07:56:46.980661 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:46.980587    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:56:47.031950 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.031926    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-service-ca\") pod \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") "
Apr 17 07:56:47.032075 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.031966    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-serving-cert\") pod \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") "
Apr 17 07:56:47.032075 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.031985    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-trusted-ca-bundle\") pod \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") "
Apr 17 07:56:47.032075 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.032010    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-config\") pod \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") "
Apr 17 07:56:47.032075 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.032040    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-oauth-serving-cert\") pod \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") "
Apr 17 07:56:47.032075 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.032073    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6pmb\" (UniqueName: \"kubernetes.io/projected/fa5c103a-608e-4786-a52a-f85ff80e4bdc-kube-api-access-k6pmb\") pod \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") "
Apr 17 07:56:47.032297 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.032088    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-oauth-config\") pod \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\" (UID: \"fa5c103a-608e-4786-a52a-f85ff80e4bdc\") "
Apr 17 07:56:47.032413 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.032379    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-service-ca" (OuterVolumeSpecName: "service-ca") pod "fa5c103a-608e-4786-a52a-f85ff80e4bdc" (UID: "fa5c103a-608e-4786-a52a-f85ff80e4bdc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 17 07:56:47.032520 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.032487    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-config" (OuterVolumeSpecName: "console-config") pod "fa5c103a-608e-4786-a52a-f85ff80e4bdc" (UID: "fa5c103a-608e-4786-a52a-f85ff80e4bdc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 17 07:56:47.032593 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.032523    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fa5c103a-608e-4786-a52a-f85ff80e4bdc" (UID: "fa5c103a-608e-4786-a52a-f85ff80e4bdc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 17 07:56:47.032668 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.032638    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fa5c103a-608e-4786-a52a-f85ff80e4bdc" (UID: "fa5c103a-608e-4786-a52a-f85ff80e4bdc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Apr 17 07:56:47.034210 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.034183    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fa5c103a-608e-4786-a52a-f85ff80e4bdc" (UID: "fa5c103a-608e-4786-a52a-f85ff80e4bdc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 07:56:47.034301 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.034241    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa5c103a-608e-4786-a52a-f85ff80e4bdc-kube-api-access-k6pmb" (OuterVolumeSpecName: "kube-api-access-k6pmb") pod "fa5c103a-608e-4786-a52a-f85ff80e4bdc" (UID: "fa5c103a-608e-4786-a52a-f85ff80e4bdc"). InnerVolumeSpecName "kube-api-access-k6pmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 07:56:47.034343 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.034306    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fa5c103a-608e-4786-a52a-f85ff80e4bdc" (UID: "fa5c103a-608e-4786-a52a-f85ff80e4bdc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 07:56:47.133509 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.133475    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-oauth-serving-cert\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:56:47.133509 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.133501    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6pmb\" (UniqueName: \"kubernetes.io/projected/fa5c103a-608e-4786-a52a-f85ff80e4bdc-kube-api-access-k6pmb\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:56:47.133509 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.133511    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-oauth-config\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:56:47.133509 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.133520    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-service-ca\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:56:47.133748 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.133529    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-serving-cert\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:56:47.133748 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.133538    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-trusted-ca-bundle\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:56:47.133748 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.133546    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa5c103a-608e-4786-a52a-f85ff80e4bdc-console-config\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:56:47.450392 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.450319    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6bcc59b8f7-bp4d9_fa5c103a-608e-4786-a52a-f85ff80e4bdc/console/0.log"
Apr 17 07:56:47.450392 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.450359    2571 generic.go:358] "Generic (PLEG): container finished" podID="fa5c103a-608e-4786-a52a-f85ff80e4bdc" containerID="f4d8e5dfc318c3369f3ac3430ea9d399a4f5a351c8a3825d31294f07fba7ce98" exitCode=2
Apr 17 07:56:47.450583 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.450421    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bcc59b8f7-bp4d9"
Apr 17 07:56:47.450583 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.450428    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bcc59b8f7-bp4d9" event={"ID":"fa5c103a-608e-4786-a52a-f85ff80e4bdc","Type":"ContainerDied","Data":"f4d8e5dfc318c3369f3ac3430ea9d399a4f5a351c8a3825d31294f07fba7ce98"}
Apr 17 07:56:47.450583 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.450476    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bcc59b8f7-bp4d9" event={"ID":"fa5c103a-608e-4786-a52a-f85ff80e4bdc","Type":"ContainerDied","Data":"38464664f9b86ec23ead6ba8ca3fa4c8b6f264afd00c8cbf17aa21069b647772"}
Apr 17 07:56:47.450583 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.450493    2571 scope.go:117] "RemoveContainer" containerID="f4d8e5dfc318c3369f3ac3430ea9d399a4f5a351c8a3825d31294f07fba7ce98"
Apr 17 07:56:47.460250 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.460234    2571 scope.go:117] "RemoveContainer" containerID="f4d8e5dfc318c3369f3ac3430ea9d399a4f5a351c8a3825d31294f07fba7ce98"
Apr 17 07:56:47.460493 ip-10-0-143-45 kubenswrapper[2571]: E0417 07:56:47.460475    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4d8e5dfc318c3369f3ac3430ea9d399a4f5a351c8a3825d31294f07fba7ce98\": container with ID starting with f4d8e5dfc318c3369f3ac3430ea9d399a4f5a351c8a3825d31294f07fba7ce98 not found: ID does not exist" containerID="f4d8e5dfc318c3369f3ac3430ea9d399a4f5a351c8a3825d31294f07fba7ce98"
Apr 17 07:56:47.460534 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.460502    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4d8e5dfc318c3369f3ac3430ea9d399a4f5a351c8a3825d31294f07fba7ce98"} err="failed to get container status \"f4d8e5dfc318c3369f3ac3430ea9d399a4f5a351c8a3825d31294f07fba7ce98\": rpc error: code = NotFound desc = could not find container \"f4d8e5dfc318c3369f3ac3430ea9d399a4f5a351c8a3825d31294f07fba7ce98\": container with ID starting with f4d8e5dfc318c3369f3ac3430ea9d399a4f5a351c8a3825d31294f07fba7ce98 not found: ID does not exist"
Apr 17 07:56:47.471252 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.471229    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bcc59b8f7-bp4d9"]
Apr 17 07:56:47.474705 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.474683    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bcc59b8f7-bp4d9"]
Apr 17 07:56:47.542762 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:47.542729    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa5c103a-608e-4786-a52a-f85ff80e4bdc" path="/var/lib/kubelet/pods/fa5c103a-608e-4786-a52a-f85ff80e4bdc/volumes"
Apr 17 07:56:52.234317 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.234281    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-bgjfg"]
Apr 17 07:56:52.234713 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.234562    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f23f7f76-f35f-4f48-b474-6d21c66886d3" containerName="console"
Apr 17 07:56:52.234713 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.234575    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f23f7f76-f35f-4f48-b474-6d21c66886d3" containerName="console"
Apr 17 07:56:52.234713 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.234583    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e13695f3-4327-46c1-b4d1-4225aa82f4bb" containerName="console"
Apr 17 07:56:52.234713 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.234588    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e13695f3-4327-46c1-b4d1-4225aa82f4bb" containerName="console"
Apr 17 07:56:52.234713 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.234595    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa5c103a-608e-4786-a52a-f85ff80e4bdc" containerName="console"
Apr 17 07:56:52.234713 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.234603    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa5c103a-608e-4786-a52a-f85ff80e4bdc" containerName="console"
Apr 17 07:56:52.234713 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.234644    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e13695f3-4327-46c1-b4d1-4225aa82f4bb" containerName="console"
Apr 17 07:56:52.234713 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.234652    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="f23f7f76-f35f-4f48-b474-6d21c66886d3" containerName="console"
Apr 17 07:56:52.234713 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.234659    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="fa5c103a-608e-4786-a52a-f85ff80e4bdc" containerName="console"
Apr 17 07:56:52.238489 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.238468    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bgjfg"
Apr 17 07:56:52.240929 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.240897    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Apr 17 07:56:52.245689 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.245662    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bgjfg"]
Apr 17 07:56:52.268741 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.268714    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f1e8aadb-7290-45b3-8467-a2a13b104ce7-kubelet-config\") pod \"global-pull-secret-syncer-bgjfg\" (UID: \"f1e8aadb-7290-45b3-8467-a2a13b104ce7\") " pod="kube-system/global-pull-secret-syncer-bgjfg"
Apr 17 07:56:52.268839 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.268760    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f1e8aadb-7290-45b3-8467-a2a13b104ce7-dbus\") pod \"global-pull-secret-syncer-bgjfg\" (UID: \"f1e8aadb-7290-45b3-8467-a2a13b104ce7\") " pod="kube-system/global-pull-secret-syncer-bgjfg"
Apr 17 07:56:52.268839 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.268804    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1e8aadb-7290-45b3-8467-a2a13b104ce7-original-pull-secret\") pod \"global-pull-secret-syncer-bgjfg\" (UID: \"f1e8aadb-7290-45b3-8467-a2a13b104ce7\") " pod="kube-system/global-pull-secret-syncer-bgjfg"
Apr 17 07:56:52.369204 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.369168    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f1e8aadb-7290-45b3-8467-a2a13b104ce7-kubelet-config\") pod \"global-pull-secret-syncer-bgjfg\" (UID: \"f1e8aadb-7290-45b3-8467-a2a13b104ce7\") " pod="kube-system/global-pull-secret-syncer-bgjfg"
Apr 17 07:56:52.369328 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.369221    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f1e8aadb-7290-45b3-8467-a2a13b104ce7-dbus\") pod \"global-pull-secret-syncer-bgjfg\" (UID: \"f1e8aadb-7290-45b3-8467-a2a13b104ce7\") " pod="kube-system/global-pull-secret-syncer-bgjfg"
Apr 17 07:56:52.369328 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.369256    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1e8aadb-7290-45b3-8467-a2a13b104ce7-original-pull-secret\") pod \"global-pull-secret-syncer-bgjfg\" (UID: \"f1e8aadb-7290-45b3-8467-a2a13b104ce7\") " pod="kube-system/global-pull-secret-syncer-bgjfg"
Apr 17 07:56:52.369328 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.369294    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f1e8aadb-7290-45b3-8467-a2a13b104ce7-kubelet-config\") pod \"global-pull-secret-syncer-bgjfg\" (UID: \"f1e8aadb-7290-45b3-8467-a2a13b104ce7\") " pod="kube-system/global-pull-secret-syncer-bgjfg"
Apr 17 07:56:52.369456 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.369416    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f1e8aadb-7290-45b3-8467-a2a13b104ce7-dbus\") pod \"global-pull-secret-syncer-bgjfg\" (UID: \"f1e8aadb-7290-45b3-8467-a2a13b104ce7\") " pod="kube-system/global-pull-secret-syncer-bgjfg"
Apr 17 07:56:52.371537 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.371514    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1e8aadb-7290-45b3-8467-a2a13b104ce7-original-pull-secret\") pod \"global-pull-secret-syncer-bgjfg\" (UID: \"f1e8aadb-7290-45b3-8467-a2a13b104ce7\") " pod="kube-system/global-pull-secret-syncer-bgjfg"
Apr 17 07:56:52.547631 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.547592    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bgjfg"
Apr 17 07:56:52.658544 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:52.658514    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bgjfg"]
Apr 17 07:56:52.661769 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:56:52.661741    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e8aadb_7290_45b3_8467_a2a13b104ce7.slice/crio-02f51b0b2f50876f402bef0c7188255a1518c455d3fc258105ada809c66dc8fa WatchSource:0}: Error finding container 02f51b0b2f50876f402bef0c7188255a1518c455d3fc258105ada809c66dc8fa: Status 404 returned error can't find the container with id 02f51b0b2f50876f402bef0c7188255a1518c455d3fc258105ada809c66dc8fa
Apr 17 07:56:53.468847 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:53.468805    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bgjfg" event={"ID":"f1e8aadb-7290-45b3-8467-a2a13b104ce7","Type":"ContainerStarted","Data":"02f51b0b2f50876f402bef0c7188255a1518c455d3fc258105ada809c66dc8fa"}
Apr 17 07:56:57.482077 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:57.482044    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bgjfg" event={"ID":"f1e8aadb-7290-45b3-8467-a2a13b104ce7","Type":"ContainerStarted","Data":"d1158a09cec4f4dc883f9ff33200d7f4ded15ce1b471dd44ddfd4eb53db302b6"}
Apr 17 07:56:57.496764 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:56:57.496719    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bgjfg" podStartSLOduration=1.482799767 podStartE2EDuration="5.496705856s" podCreationTimestamp="2026-04-17 07:56:52 +0000 UTC" firstStartedPulling="2026-04-17 07:56:52.663252673 +0000 UTC m=+283.716855863" lastFinishedPulling="2026-04-17 07:56:56.677158747 +0000 UTC m=+287.730761952" observedRunningTime="2026-04-17 07:56:57.495792424 +0000 UTC m=+288.549395635" watchObservedRunningTime="2026-04-17 07:56:57.496705856 +0000 UTC m=+288.550309064"
Apr 17 07:57:09.426835 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:09.426806    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 07:57:09.429782 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:09.429760    2571 kubelet.go:1628] "Image garbage collection succeeded"
Apr 17 07:57:09.432340 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:09.432321    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 07:57:10.381838 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.381786    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"]
Apr 17 07:57:10.384964 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.384943    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"
Apr 17 07:57:10.387408 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.387388    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 17 07:57:10.388288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.388260    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xv8sk\""
Apr 17 07:57:10.388389 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.388266    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 17 07:57:10.392743 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.392723    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"]
Apr 17 07:57:10.494914 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.494877    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37f5eaea-3c5c-4294-a605-f044831d6d46-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6\" (UID: \"37f5eaea-3c5c-4294-a605-f044831d6d46\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"
Apr 17 07:57:10.495298 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.494930    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5wg6\" (UniqueName: \"kubernetes.io/projected/37f5eaea-3c5c-4294-a605-f044831d6d46-kube-api-access-g5wg6\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6\" (UID: \"37f5eaea-3c5c-4294-a605-f044831d6d46\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"
Apr 17 07:57:10.495298 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.494998    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37f5eaea-3c5c-4294-a605-f044831d6d46-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6\" (UID: \"37f5eaea-3c5c-4294-a605-f044831d6d46\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"
Apr 17 07:57:10.596013 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.595963    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37f5eaea-3c5c-4294-a605-f044831d6d46-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6\" (UID: \"37f5eaea-3c5c-4294-a605-f044831d6d46\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"
Apr 17 07:57:10.596214 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.596030    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g5wg6\" (UniqueName: \"kubernetes.io/projected/37f5eaea-3c5c-4294-a605-f044831d6d46-kube-api-access-g5wg6\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6\" (UID: \"37f5eaea-3c5c-4294-a605-f044831d6d46\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"
Apr 17 07:57:10.596214 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.596067    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37f5eaea-3c5c-4294-a605-f044831d6d46-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6\" (UID: \"37f5eaea-3c5c-4294-a605-f044831d6d46\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"
Apr 17 07:57:10.596457 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.596410    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37f5eaea-3c5c-4294-a605-f044831d6d46-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6\" (UID: \"37f5eaea-3c5c-4294-a605-f044831d6d46\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"
Apr 17 07:57:10.596533 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.596425    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37f5eaea-3c5c-4294-a605-f044831d6d46-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6\" (UID: \"37f5eaea-3c5c-4294-a605-f044831d6d46\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"
Apr 17 07:57:10.604299 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.604276    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5wg6\" (UniqueName: \"kubernetes.io/projected/37f5eaea-3c5c-4294-a605-f044831d6d46-kube-api-access-g5wg6\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6\" (UID: \"37f5eaea-3c5c-4294-a605-f044831d6d46\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"
Apr 17 07:57:10.694999 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.694904    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"
Apr 17 07:57:10.817317 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.817292    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"]
Apr 17 07:57:10.819659 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:57:10.819631    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f5eaea_3c5c_4294_a605_f044831d6d46.slice/crio-a813b1d47a26ef2da7adc015330cb795878f0fe40c7d185bc1053b43da52d2f4 WatchSource:0}: Error finding container a813b1d47a26ef2da7adc015330cb795878f0fe40c7d185bc1053b43da52d2f4: Status 404 returned error can't find the container with id a813b1d47a26ef2da7adc015330cb795878f0fe40c7d185bc1053b43da52d2f4
Apr 17 07:57:10.821556 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:10.821534    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 17 07:57:11.522914 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:11.522880    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6" event={"ID":"37f5eaea-3c5c-4294-a605-f044831d6d46","Type":"ContainerStarted","Data":"a813b1d47a26ef2da7adc015330cb795878f0fe40c7d185bc1053b43da52d2f4"}
Apr 17 07:57:18.543864 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:18.543827    2571 generic.go:358] "Generic (PLEG): container finished" podID="37f5eaea-3c5c-4294-a605-f044831d6d46" containerID="baa59b00c2bc1f1bbaa9b92636932236cd0edaa53f3353ac2856b888743af303" exitCode=0
Apr 17 07:57:18.544224 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:18.543909    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6" event={"ID":"37f5eaea-3c5c-4294-a605-f044831d6d46","Type":"ContainerDied","Data":"baa59b00c2bc1f1bbaa9b92636932236cd0edaa53f3353ac2856b888743af303"}
Apr 17 07:57:21.553541 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:21.553504    2571 generic.go:358] "Generic (PLEG): container finished" podID="37f5eaea-3c5c-4294-a605-f044831d6d46" containerID="7c7072186cc671833549daa331096aef9b8307dfe08b21b8063fbd0677a4642e" exitCode=0
Apr 17 07:57:21.553986 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:21.553589    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6" event={"ID":"37f5eaea-3c5c-4294-a605-f044831d6d46","Type":"ContainerDied","Data":"7c7072186cc671833549daa331096aef9b8307dfe08b21b8063fbd0677a4642e"}
Apr 17 07:57:29.576159 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:29.576116    2571 generic.go:358] "Generic (PLEG): container finished" podID="37f5eaea-3c5c-4294-a605-f044831d6d46" containerID="6ff1a454c3ed903e7d99d72a122441aff43fd6eab78bcd4e55a62e2877eabc28" exitCode=0
Apr 17 07:57:29.576539 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:29.576195    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6" event={"ID":"37f5eaea-3c5c-4294-a605-f044831d6d46","Type":"ContainerDied","Data":"6ff1a454c3ed903e7d99d72a122441aff43fd6eab78bcd4e55a62e2877eabc28"}
Apr 17 07:57:30.693711 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:30.693687    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"
Apr 17 07:57:30.755520 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:30.755494    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5wg6\" (UniqueName: \"kubernetes.io/projected/37f5eaea-3c5c-4294-a605-f044831d6d46-kube-api-access-g5wg6\") pod \"37f5eaea-3c5c-4294-a605-f044831d6d46\" (UID: \"37f5eaea-3c5c-4294-a605-f044831d6d46\") "
Apr 17 07:57:30.755682 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:30.755526    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37f5eaea-3c5c-4294-a605-f044831d6d46-bundle\") pod \"37f5eaea-3c5c-4294-a605-f044831d6d46\" (UID: \"37f5eaea-3c5c-4294-a605-f044831d6d46\") "
Apr 17 07:57:30.755682 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:30.755604    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37f5eaea-3c5c-4294-a605-f044831d6d46-util\") pod \"37f5eaea-3c5c-4294-a605-f044831d6d46\" (UID: \"37f5eaea-3c5c-4294-a605-f044831d6d46\") "
Apr 17 07:57:30.756065 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:30.756042    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f5eaea-3c5c-4294-a605-f044831d6d46-bundle" (OuterVolumeSpecName: "bundle") pod "37f5eaea-3c5c-4294-a605-f044831d6d46" (UID: "37f5eaea-3c5c-4294-a605-f044831d6d46"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 07:57:30.757737 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:30.757703    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37f5eaea-3c5c-4294-a605-f044831d6d46-kube-api-access-g5wg6" (OuterVolumeSpecName: "kube-api-access-g5wg6") pod "37f5eaea-3c5c-4294-a605-f044831d6d46" (UID: "37f5eaea-3c5c-4294-a605-f044831d6d46"). InnerVolumeSpecName "kube-api-access-g5wg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 07:57:30.759658 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:30.759633    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f5eaea-3c5c-4294-a605-f044831d6d46-util" (OuterVolumeSpecName: "util") pod "37f5eaea-3c5c-4294-a605-f044831d6d46" (UID: "37f5eaea-3c5c-4294-a605-f044831d6d46"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 07:57:30.856225 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:30.856143    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5wg6\" (UniqueName: \"kubernetes.io/projected/37f5eaea-3c5c-4294-a605-f044831d6d46-kube-api-access-g5wg6\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:57:30.856225 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:30.856174    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37f5eaea-3c5c-4294-a605-f044831d6d46-bundle\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:57:30.856225 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:30.856185    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37f5eaea-3c5c-4294-a605-f044831d6d46-util\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:57:31.587176 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:31.587137    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6" event={"ID":"37f5eaea-3c5c-4294-a605-f044831d6d46","Type":"ContainerDied","Data":"a813b1d47a26ef2da7adc015330cb795878f0fe40c7d185bc1053b43da52d2f4"}
Apr 17 07:57:31.587176 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:31.587170    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cdpsk6"
Apr 17 07:57:31.587176 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:57:31.587180    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a813b1d47a26ef2da7adc015330cb795878f0fe40c7d185bc1053b43da52d2f4"
Apr 17 07:58:35.632910 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.632866    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"]
Apr 17 07:58:35.633380 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.633138    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f5eaea-3c5c-4294-a605-f044831d6d46" containerName="extract"
Apr 17 07:58:35.633380 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.633151    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f5eaea-3c5c-4294-a605-f044831d6d46" containerName="extract"
Apr 17 07:58:35.633380 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.633161    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f5eaea-3c5c-4294-a605-f044831d6d46" containerName="util"
Apr 17 07:58:35.633380 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.633166    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f5eaea-3c5c-4294-a605-f044831d6d46" containerName="util"
Apr 17 07:58:35.633380 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.633179    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f5eaea-3c5c-4294-a605-f044831d6d46" containerName="pull"
Apr 17 07:58:35.633380 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.633184    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f5eaea-3c5c-4294-a605-f044831d6d46" containerName="pull"
Apr 17 07:58:35.633380 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.633222    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="37f5eaea-3c5c-4294-a605-f044831d6d46" containerName="extract"
Apr 17 07:58:35.636122 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.636106    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"
Apr 17 07:58:35.638825 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.638805    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 17 07:58:35.639760 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.639742    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 17 07:58:35.639867 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.639744    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xv8sk\""
Apr 17 07:58:35.644130 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.644106    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"]
Apr 17 07:58:35.728495 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.728455    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp6cg\" (UniqueName: \"kubernetes.io/projected/f04eb305-7a2a-4296-b2b2-b49c4892a58f-kube-api-access-pp6cg\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd\" (UID: \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"
Apr 17 07:58:35.728664 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.728559    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f04eb305-7a2a-4296-b2b2-b49c4892a58f-bundle\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd\" (UID: \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"
Apr 17 07:58:35.728664 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.728585    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f04eb305-7a2a-4296-b2b2-b49c4892a58f-util\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd\" (UID: \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"
Apr 17 07:58:35.829200 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.829166    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pp6cg\" (UniqueName: \"kubernetes.io/projected/f04eb305-7a2a-4296-b2b2-b49c4892a58f-kube-api-access-pp6cg\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd\" (UID: \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"
Apr 17 07:58:35.829357 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.829221    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f04eb305-7a2a-4296-b2b2-b49c4892a58f-bundle\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd\" (UID: \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"
Apr 17 07:58:35.829357 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.829238    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f04eb305-7a2a-4296-b2b2-b49c4892a58f-util\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd\" (UID: \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"
Apr 17 07:58:35.829567 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.829552    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f04eb305-7a2a-4296-b2b2-b49c4892a58f-util\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd\" (UID: \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"
Apr 17 07:58:35.829646 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.829629    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f04eb305-7a2a-4296-b2b2-b49c4892a58f-bundle\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd\" (UID: \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"
Apr 17 07:58:35.837416 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.837395    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp6cg\" (UniqueName: \"kubernetes.io/projected/f04eb305-7a2a-4296-b2b2-b49c4892a58f-kube-api-access-pp6cg\") pod \"925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd\" (UID: \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\") " pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"
Apr 17 07:58:35.945490 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:35.945386    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"
Apr 17 07:58:36.062911 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:36.062875    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"]
Apr 17 07:58:36.065648 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:58:36.065609    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf04eb305_7a2a_4296_b2b2_b49c4892a58f.slice/crio-04e3fa62a52848ce083c27cdbca2926971487346726a0f7027b1d33f94e08167 WatchSource:0}: Error finding container 04e3fa62a52848ce083c27cdbca2926971487346726a0f7027b1d33f94e08167: Status 404 returned error can't find the container with id 04e3fa62a52848ce083c27cdbca2926971487346726a0f7027b1d33f94e08167
Apr 17 07:58:36.760777 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:36.760744    2571 generic.go:358] "Generic (PLEG): container finished" podID="f04eb305-7a2a-4296-b2b2-b49c4892a58f" containerID="7715975fc49b9ec7866f14fac57875d281cdcce27476482bab29676aecfb33ee" exitCode=0
Apr 17 07:58:36.761151 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:36.760799    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd" event={"ID":"f04eb305-7a2a-4296-b2b2-b49c4892a58f","Type":"ContainerDied","Data":"7715975fc49b9ec7866f14fac57875d281cdcce27476482bab29676aecfb33ee"}
Apr 17 07:58:36.761151 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:36.760821    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd" event={"ID":"f04eb305-7a2a-4296-b2b2-b49c4892a58f","Type":"ContainerStarted","Data":"04e3fa62a52848ce083c27cdbca2926971487346726a0f7027b1d33f94e08167"}
Apr 17 07:58:37.765323 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:37.765226    2571 generic.go:358] "Generic (PLEG): container finished" podID="f04eb305-7a2a-4296-b2b2-b49c4892a58f" containerID="cc754ae549b54abac3a7be67bb09a5587e52daa2f8fc93c193ec04b735a01ea8" exitCode=0
Apr 17 07:58:37.765710 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:37.765310    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd" event={"ID":"f04eb305-7a2a-4296-b2b2-b49c4892a58f","Type":"ContainerDied","Data":"cc754ae549b54abac3a7be67bb09a5587e52daa2f8fc93c193ec04b735a01ea8"}
Apr 17 07:58:38.769745 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:38.769702    2571 generic.go:358] "Generic (PLEG): container finished" podID="f04eb305-7a2a-4296-b2b2-b49c4892a58f" containerID="051499f70366bf34161c87763b4a355c7b7a5b4c1c5c769eecfe40c6e5839094" exitCode=0
Apr 17 07:58:38.770115 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:38.769760    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd" event={"ID":"f04eb305-7a2a-4296-b2b2-b49c4892a58f","Type":"ContainerDied","Data":"051499f70366bf34161c87763b4a355c7b7a5b4c1c5c769eecfe40c6e5839094"}
Apr 17 07:58:39.888821 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:39.888799    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"
Apr 17 07:58:39.960759 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:39.960731    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f04eb305-7a2a-4296-b2b2-b49c4892a58f-bundle\") pod \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\" (UID: \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\") "
Apr 17 07:58:39.960904 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:39.960805    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pp6cg\" (UniqueName: \"kubernetes.io/projected/f04eb305-7a2a-4296-b2b2-b49c4892a58f-kube-api-access-pp6cg\") pod \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\" (UID: \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\") "
Apr 17 07:58:39.960904 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:39.960841    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f04eb305-7a2a-4296-b2b2-b49c4892a58f-util\") pod \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\" (UID: \"f04eb305-7a2a-4296-b2b2-b49c4892a58f\") "
Apr 17 07:58:39.961395 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:39.961370    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f04eb305-7a2a-4296-b2b2-b49c4892a58f-bundle" (OuterVolumeSpecName: "bundle") pod "f04eb305-7a2a-4296-b2b2-b49c4892a58f" (UID: "f04eb305-7a2a-4296-b2b2-b49c4892a58f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 07:58:39.962844 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:39.962825    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f04eb305-7a2a-4296-b2b2-b49c4892a58f-kube-api-access-pp6cg" (OuterVolumeSpecName: "kube-api-access-pp6cg") pod "f04eb305-7a2a-4296-b2b2-b49c4892a58f" (UID: "f04eb305-7a2a-4296-b2b2-b49c4892a58f"). InnerVolumeSpecName "kube-api-access-pp6cg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 07:58:39.966258 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:39.966158    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f04eb305-7a2a-4296-b2b2-b49c4892a58f-util" (OuterVolumeSpecName: "util") pod "f04eb305-7a2a-4296-b2b2-b49c4892a58f" (UID: "f04eb305-7a2a-4296-b2b2-b49c4892a58f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 07:58:40.062071 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:40.062044    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f04eb305-7a2a-4296-b2b2-b49c4892a58f-util\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:58:40.062071 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:40.062073    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f04eb305-7a2a-4296-b2b2-b49c4892a58f-bundle\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:58:40.062202 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:40.062087    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp6cg\" (UniqueName: \"kubernetes.io/projected/f04eb305-7a2a-4296-b2b2-b49c4892a58f-kube-api-access-pp6cg\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:58:40.777205 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:40.777178    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd"
Apr 17 07:58:40.777389 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:40.777173    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/925ad1f05bf386dc21bdfe2f8249c1fbfd04a404dec7a7fb6362d758e5x7wxd" event={"ID":"f04eb305-7a2a-4296-b2b2-b49c4892a58f","Type":"ContainerDied","Data":"04e3fa62a52848ce083c27cdbca2926971487346726a0f7027b1d33f94e08167"}
Apr 17 07:58:40.777389 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:40.777291    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04e3fa62a52848ce083c27cdbca2926971487346726a0f7027b1d33f94e08167"
Apr 17 07:58:48.132131 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.132097    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx"]
Apr 17 07:58:48.132491 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.132342    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04eb305-7a2a-4296-b2b2-b49c4892a58f" containerName="pull"
Apr 17 07:58:48.132491 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.132352    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04eb305-7a2a-4296-b2b2-b49c4892a58f" containerName="pull"
Apr 17 07:58:48.132491 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.132368    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04eb305-7a2a-4296-b2b2-b49c4892a58f" containerName="extract"
Apr 17 07:58:48.132491 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.132373    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04eb305-7a2a-4296-b2b2-b49c4892a58f" containerName="extract"
Apr 17 07:58:48.132491 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.132386    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04eb305-7a2a-4296-b2b2-b49c4892a58f" containerName="util"
Apr 17 07:58:48.132491 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.132392    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04eb305-7a2a-4296-b2b2-b49c4892a58f" containerName="util"
Apr 17 07:58:48.132491 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.132450    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="f04eb305-7a2a-4296-b2b2-b49c4892a58f" containerName="extract"
Apr 17 07:58:48.136706 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.136690    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx"
Apr 17 07:58:48.140785 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.140761    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-b7zgx\""
Apr 17 07:58:48.140963 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.140943    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Apr 17 07:58:48.141044 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.140996    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Apr 17 07:58:48.151135 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.151112    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx"]
Apr 17 07:58:48.217136 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.217099    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95065a6d-2f55-42e3-8a96-c4876a620588-tmp\") pod \"cert-manager-operator-controller-manager-7ccfb878b5-pgsvx\" (UID: \"95065a6d-2f55-42e3-8a96-c4876a620588\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx"
Apr 17 07:58:48.217285 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.217152    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6sp6\" (UniqueName: \"kubernetes.io/projected/95065a6d-2f55-42e3-8a96-c4876a620588-kube-api-access-w6sp6\") pod \"cert-manager-operator-controller-manager-7ccfb878b5-pgsvx\" (UID: \"95065a6d-2f55-42e3-8a96-c4876a620588\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx"
Apr 17 07:58:48.317604 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.317570    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w6sp6\" (UniqueName: \"kubernetes.io/projected/95065a6d-2f55-42e3-8a96-c4876a620588-kube-api-access-w6sp6\") pod \"cert-manager-operator-controller-manager-7ccfb878b5-pgsvx\" (UID: \"95065a6d-2f55-42e3-8a96-c4876a620588\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx"
Apr 17 07:58:48.317744 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.317619    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95065a6d-2f55-42e3-8a96-c4876a620588-tmp\") pod \"cert-manager-operator-controller-manager-7ccfb878b5-pgsvx\" (UID: \"95065a6d-2f55-42e3-8a96-c4876a620588\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx"
Apr 17 07:58:48.317992 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.317975    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95065a6d-2f55-42e3-8a96-c4876a620588-tmp\") pod \"cert-manager-operator-controller-manager-7ccfb878b5-pgsvx\" (UID: \"95065a6d-2f55-42e3-8a96-c4876a620588\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx"
Apr 17 07:58:48.326567 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.326537    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6sp6\" (UniqueName: \"kubernetes.io/projected/95065a6d-2f55-42e3-8a96-c4876a620588-kube-api-access-w6sp6\") pod \"cert-manager-operator-controller-manager-7ccfb878b5-pgsvx\" (UID: \"95065a6d-2f55-42e3-8a96-c4876a620588\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx"
Apr 17 07:58:48.445915 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.445842    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx"
Apr 17 07:58:48.565673 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.565652    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx"]
Apr 17 07:58:48.568691 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:58:48.568654    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95065a6d_2f55_42e3_8a96_c4876a620588.slice/crio-1f3b7fd55d6caad7bd344694be47915a9f126b742b3cd63baa04d0f67c64d5ec WatchSource:0}: Error finding container 1f3b7fd55d6caad7bd344694be47915a9f126b742b3cd63baa04d0f67c64d5ec: Status 404 returned error can't find the container with id 1f3b7fd55d6caad7bd344694be47915a9f126b742b3cd63baa04d0f67c64d5ec
Apr 17 07:58:48.806552 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:48.806521    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx" event={"ID":"95065a6d-2f55-42e3-8a96-c4876a620588","Type":"ContainerStarted","Data":"1f3b7fd55d6caad7bd344694be47915a9f126b742b3cd63baa04d0f67c64d5ec"}
Apr 17 07:58:50.813395 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:50.813355    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx" event={"ID":"95065a6d-2f55-42e3-8a96-c4876a620588","Type":"ContainerStarted","Data":"4d9eeee5b892b3d0f7813d01bd16d2525ee0d041ee95d94fd8c21594bff1c19f"}
Apr 17 07:58:50.834914 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:50.834870    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7ccfb878b5-pgsvx" podStartSLOduration=0.951274732 podStartE2EDuration="2.834855589s" podCreationTimestamp="2026-04-17 07:58:48 +0000 UTC" firstStartedPulling="2026-04-17 07:58:48.571077489 +0000 UTC m=+399.624680677" lastFinishedPulling="2026-04-17 07:58:50.454658345 +0000 UTC m=+401.508261534" observedRunningTime="2026-04-17 07:58:50.831874717 +0000 UTC m=+401.885477931" watchObservedRunningTime="2026-04-17 07:58:50.834855589 +0000 UTC m=+401.888458799"
Apr 17 07:58:52.945290 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:52.945255    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"]
Apr 17 07:58:52.948595 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:52.948577    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"
Apr 17 07:58:52.951150 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:52.951126    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 17 07:58:52.951374 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:52.951358    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xv8sk\""
Apr 17 07:58:52.952207 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:52.952191    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 17 07:58:52.956509 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:52.956478    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"]
Apr 17 07:58:53.055908 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.055874    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ad83d8-ace5-4114-9f18-ea209490b976-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5\" (UID: \"d7ad83d8-ace5-4114-9f18-ea209490b976\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"
Apr 17 07:58:53.056066 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.055912    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ad83d8-ace5-4114-9f18-ea209490b976-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5\" (UID: \"d7ad83d8-ace5-4114-9f18-ea209490b976\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"
Apr 17 07:58:53.056066 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.055941    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68ss2\" (UniqueName: \"kubernetes.io/projected/d7ad83d8-ace5-4114-9f18-ea209490b976-kube-api-access-68ss2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5\" (UID: \"d7ad83d8-ace5-4114-9f18-ea209490b976\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"
Apr 17 07:58:53.156695 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.156657    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ad83d8-ace5-4114-9f18-ea209490b976-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5\" (UID: \"d7ad83d8-ace5-4114-9f18-ea209490b976\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"
Apr 17 07:58:53.156695 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.156699    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ad83d8-ace5-4114-9f18-ea209490b976-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5\" (UID: \"d7ad83d8-ace5-4114-9f18-ea209490b976\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"
Apr 17 07:58:53.156913 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.156724    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-68ss2\" (UniqueName: \"kubernetes.io/projected/d7ad83d8-ace5-4114-9f18-ea209490b976-kube-api-access-68ss2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5\" (UID: \"d7ad83d8-ace5-4114-9f18-ea209490b976\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"
Apr 17 07:58:53.157064 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.157037    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ad83d8-ace5-4114-9f18-ea209490b976-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5\" (UID: \"d7ad83d8-ace5-4114-9f18-ea209490b976\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"
Apr 17 07:58:53.157120 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.157097    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ad83d8-ace5-4114-9f18-ea209490b976-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5\" (UID: \"d7ad83d8-ace5-4114-9f18-ea209490b976\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"
Apr 17 07:58:53.165058 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.165032    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68ss2\" (UniqueName: \"kubernetes.io/projected/d7ad83d8-ace5-4114-9f18-ea209490b976-kube-api-access-68ss2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5\" (UID: \"d7ad83d8-ace5-4114-9f18-ea209490b976\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"
Apr 17 07:58:53.257283 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.257202    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"
Apr 17 07:58:53.383951 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.383910    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"]
Apr 17 07:58:53.386347 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:58:53.386320    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ad83d8_ace5_4114_9f18_ea209490b976.slice/crio-dfcfce5b248af4b653cfcb58c4f3efccbfa0fefa38c29fc1304e8a7b6750b131 WatchSource:0}: Error finding container dfcfce5b248af4b653cfcb58c4f3efccbfa0fefa38c29fc1304e8a7b6750b131: Status 404 returned error can't find the container with id dfcfce5b248af4b653cfcb58c4f3efccbfa0fefa38c29fc1304e8a7b6750b131
Apr 17 07:58:53.823920 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.823886    2571 generic.go:358] "Generic (PLEG): container finished" podID="d7ad83d8-ace5-4114-9f18-ea209490b976" containerID="a6aae9641a5ae0d6bdf9c29382d6d1a7f7d0d0a16bf8eff0aace69b8bc019c8b" exitCode=0
Apr 17 07:58:53.824202 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.823952    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5" event={"ID":"d7ad83d8-ace5-4114-9f18-ea209490b976","Type":"ContainerDied","Data":"a6aae9641a5ae0d6bdf9c29382d6d1a7f7d0d0a16bf8eff0aace69b8bc019c8b"}
Apr 17 07:58:53.824202 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:53.823976    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5" event={"ID":"d7ad83d8-ace5-4114-9f18-ea209490b976","Type":"ContainerStarted","Data":"dfcfce5b248af4b653cfcb58c4f3efccbfa0fefa38c29fc1304e8a7b6750b131"}
Apr 17 07:58:54.322307 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.322268    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-597b96b99b-zt5vq"]
Apr 17 07:58:54.325513 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.325496    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-597b96b99b-zt5vq"
Apr 17 07:58:54.327970 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.327948    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Apr 17 07:58:54.328990 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.328972    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-fhjfp\""
Apr 17 07:58:54.329766 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.329737    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Apr 17 07:58:54.333522 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.333497    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-597b96b99b-zt5vq"]
Apr 17 07:58:54.366209 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.366180    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0be2f736-7895-4fd1-9538-6f1a9fd2714c-bound-sa-token\") pod \"cert-manager-webhook-597b96b99b-zt5vq\" (UID: \"0be2f736-7895-4fd1-9538-6f1a9fd2714c\") " pod="cert-manager/cert-manager-webhook-597b96b99b-zt5vq"
Apr 17 07:58:54.366356 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.366235    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl2g7\" (UniqueName: \"kubernetes.io/projected/0be2f736-7895-4fd1-9538-6f1a9fd2714c-kube-api-access-nl2g7\") pod \"cert-manager-webhook-597b96b99b-zt5vq\" (UID: \"0be2f736-7895-4fd1-9538-6f1a9fd2714c\") " pod="cert-manager/cert-manager-webhook-597b96b99b-zt5vq"
Apr 17 07:58:54.467247 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.467162    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0be2f736-7895-4fd1-9538-6f1a9fd2714c-bound-sa-token\") pod \"cert-manager-webhook-597b96b99b-zt5vq\" (UID: \"0be2f736-7895-4fd1-9538-6f1a9fd2714c\") " pod="cert-manager/cert-manager-webhook-597b96b99b-zt5vq"
Apr 17 07:58:54.467247 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.467219    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nl2g7\" (UniqueName: \"kubernetes.io/projected/0be2f736-7895-4fd1-9538-6f1a9fd2714c-kube-api-access-nl2g7\") pod \"cert-manager-webhook-597b96b99b-zt5vq\" (UID: \"0be2f736-7895-4fd1-9538-6f1a9fd2714c\") " pod="cert-manager/cert-manager-webhook-597b96b99b-zt5vq"
Apr 17 07:58:54.477657 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.477629    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0be2f736-7895-4fd1-9538-6f1a9fd2714c-bound-sa-token\") pod \"cert-manager-webhook-597b96b99b-zt5vq\" (UID: \"0be2f736-7895-4fd1-9538-6f1a9fd2714c\") " pod="cert-manager/cert-manager-webhook-597b96b99b-zt5vq"
Apr 17 07:58:54.477764 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.477748    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl2g7\" (UniqueName: \"kubernetes.io/projected/0be2f736-7895-4fd1-9538-6f1a9fd2714c-kube-api-access-nl2g7\") pod \"cert-manager-webhook-597b96b99b-zt5vq\" (UID: \"0be2f736-7895-4fd1-9538-6f1a9fd2714c\") " pod="cert-manager/cert-manager-webhook-597b96b99b-zt5vq"
Apr 17 07:58:54.635000 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.634963    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-597b96b99b-zt5vq"
Apr 17 07:58:54.752954 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.752870    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-597b96b99b-zt5vq"]
Apr 17 07:58:54.757326 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:58:54.757297    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0be2f736_7895_4fd1_9538_6f1a9fd2714c.slice/crio-8c0c29fb81acf529d7c3a7340a991678081030685efd49b3f352787882a52944 WatchSource:0}: Error finding container 8c0c29fb81acf529d7c3a7340a991678081030685efd49b3f352787882a52944: Status 404 returned error can't find the container with id 8c0c29fb81acf529d7c3a7340a991678081030685efd49b3f352787882a52944
Apr 17 07:58:54.831831 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:54.831794    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-597b96b99b-zt5vq" event={"ID":"0be2f736-7895-4fd1-9538-6f1a9fd2714c","Type":"ContainerStarted","Data":"8c0c29fb81acf529d7c3a7340a991678081030685efd49b3f352787882a52944"}
Apr 17 07:58:55.295284 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:55.295243    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-8966b78d4-ttdv4"]
Apr 17 07:58:55.298766 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:55.298740    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-8966b78d4-ttdv4"
Apr 17 07:58:55.301169 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:55.301146    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-khzfm\""
Apr 17 07:58:55.305357 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:55.305317    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-8966b78d4-ttdv4"]
Apr 17 07:58:55.375487 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:55.375451    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0707020e-e4e0-4a97-8348-b820ea74cb71-bound-sa-token\") pod \"cert-manager-cainjector-8966b78d4-ttdv4\" (UID: \"0707020e-e4e0-4a97-8348-b820ea74cb71\") " pod="cert-manager/cert-manager-cainjector-8966b78d4-ttdv4"
Apr 17 07:58:55.375842 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:55.375515    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsf4q\" (UniqueName: \"kubernetes.io/projected/0707020e-e4e0-4a97-8348-b820ea74cb71-kube-api-access-qsf4q\") pod \"cert-manager-cainjector-8966b78d4-ttdv4\" (UID: \"0707020e-e4e0-4a97-8348-b820ea74cb71\") " pod="cert-manager/cert-manager-cainjector-8966b78d4-ttdv4"
Apr 17 07:58:55.476941 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:55.476903    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0707020e-e4e0-4a97-8348-b820ea74cb71-bound-sa-token\") pod \"cert-manager-cainjector-8966b78d4-ttdv4\" (UID: \"0707020e-e4e0-4a97-8348-b820ea74cb71\") " pod="cert-manager/cert-manager-cainjector-8966b78d4-ttdv4"
Apr 17 07:58:55.476941 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:55.476944    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qsf4q\" (UniqueName: \"kubernetes.io/projected/0707020e-e4e0-4a97-8348-b820ea74cb71-kube-api-access-qsf4q\") pod \"cert-manager-cainjector-8966b78d4-ttdv4\" (UID: \"0707020e-e4e0-4a97-8348-b820ea74cb71\") " pod="cert-manager/cert-manager-cainjector-8966b78d4-ttdv4"
Apr 17 07:58:55.493236 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:55.493172    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0707020e-e4e0-4a97-8348-b820ea74cb71-bound-sa-token\") pod \"cert-manager-cainjector-8966b78d4-ttdv4\" (UID: \"0707020e-e4e0-4a97-8348-b820ea74cb71\") " pod="cert-manager/cert-manager-cainjector-8966b78d4-ttdv4"
Apr 17 07:58:55.493411 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:55.493388    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsf4q\" (UniqueName: \"kubernetes.io/projected/0707020e-e4e0-4a97-8348-b820ea74cb71-kube-api-access-qsf4q\") pod \"cert-manager-cainjector-8966b78d4-ttdv4\" (UID: \"0707020e-e4e0-4a97-8348-b820ea74cb71\") " pod="cert-manager/cert-manager-cainjector-8966b78d4-ttdv4"
Apr 17 07:58:55.611156 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:55.611081    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-8966b78d4-ttdv4"
Apr 17 07:58:56.147487 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:56.147461    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-8966b78d4-ttdv4"]
Apr 17 07:58:56.149728 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:58:56.149700    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0707020e_e4e0_4a97_8348_b820ea74cb71.slice/crio-142acbe7381204606e31935de21e6b54ab52799df90422d67eba796c6510486e WatchSource:0}: Error finding container 142acbe7381204606e31935de21e6b54ab52799df90422d67eba796c6510486e: Status 404 returned error can't find the container with id 142acbe7381204606e31935de21e6b54ab52799df90422d67eba796c6510486e
Apr 17 07:58:56.842157 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:56.842117    2571 generic.go:358] "Generic (PLEG): container finished" podID="d7ad83d8-ace5-4114-9f18-ea209490b976" containerID="86fbbd7dae11f82c070f20a9c8180c319bcfc3710aefc32faac80ae0248d6e26" exitCode=0
Apr 17 07:58:56.842836 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:56.842200    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5" event={"ID":"d7ad83d8-ace5-4114-9f18-ea209490b976","Type":"ContainerDied","Data":"86fbbd7dae11f82c070f20a9c8180c319bcfc3710aefc32faac80ae0248d6e26"}
Apr 17 07:58:56.843872 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:56.843843    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-8966b78d4-ttdv4" event={"ID":"0707020e-e4e0-4a97-8348-b820ea74cb71","Type":"ContainerStarted","Data":"142acbe7381204606e31935de21e6b54ab52799df90422d67eba796c6510486e"}
Apr 17 07:58:57.850156 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:57.850091    2571 generic.go:358] "Generic (PLEG): container finished" podID="d7ad83d8-ace5-4114-9f18-ea209490b976" containerID="f0455199ac40fcaf138c75a095c8855e96c6b4271432578a487d1a21e7fa898b" exitCode=0
Apr 17 07:58:57.850156 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:57.850145    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5" event={"ID":"d7ad83d8-ace5-4114-9f18-ea209490b976","Type":"ContainerDied","Data":"f0455199ac40fcaf138c75a095c8855e96c6b4271432578a487d1a21e7fa898b"}
Apr 17 07:58:58.854946 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:58.854909    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-8966b78d4-ttdv4" event={"ID":"0707020e-e4e0-4a97-8348-b820ea74cb71","Type":"ContainerStarted","Data":"0fa87427cdcb379d59ef5586bd3c9e059a87d1e3f88a56d1c863ed57a4a13b53"}
Apr 17 07:58:58.856449 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:58.856400    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-597b96b99b-zt5vq" event={"ID":"0be2f736-7895-4fd1-9538-6f1a9fd2714c","Type":"ContainerStarted","Data":"bcf42b28db30d23e4a633604ac53edfd3582753579ffaeab3182a452eee3c708"}
Apr 17 07:58:58.856653 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:58.856634    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-597b96b99b-zt5vq"
Apr 17 07:58:58.870680 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:58.870501    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-8966b78d4-ttdv4" podStartSLOduration=1.59368722 podStartE2EDuration="3.870486255s" podCreationTimestamp="2026-04-17 07:58:55 +0000 UTC" firstStartedPulling="2026-04-17 07:58:56.151523242 +0000 UTC m=+407.205126430" lastFinishedPulling="2026-04-17 07:58:58.428322272 +0000 UTC m=+409.481925465" observedRunningTime="2026-04-17 07:58:58.869824985 +0000 UTC m=+409.923428196" watchObservedRunningTime="2026-04-17 07:58:58.870486255 +0000 UTC m=+409.924089467"
Apr 17 07:58:58.884955 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:58.884914    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-597b96b99b-zt5vq" podStartSLOduration=1.222568289 podStartE2EDuration="4.884902133s" podCreationTimestamp="2026-04-17 07:58:54 +0000 UTC" firstStartedPulling="2026-04-17 07:58:54.759950111 +0000 UTC m=+405.813553299" lastFinishedPulling="2026-04-17 07:58:58.422283951 +0000 UTC m=+409.475887143" observedRunningTime="2026-04-17 07:58:58.884057045 +0000 UTC m=+409.937660259" watchObservedRunningTime="2026-04-17 07:58:58.884902133 +0000 UTC m=+409.938505346"
Apr 17 07:58:58.989607 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:58.989586    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"
Apr 17 07:58:59.110616 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:59.110540    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ad83d8-ace5-4114-9f18-ea209490b976-util\") pod \"d7ad83d8-ace5-4114-9f18-ea209490b976\" (UID: \"d7ad83d8-ace5-4114-9f18-ea209490b976\") "
Apr 17 07:58:59.110616 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:59.110581    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68ss2\" (UniqueName: \"kubernetes.io/projected/d7ad83d8-ace5-4114-9f18-ea209490b976-kube-api-access-68ss2\") pod \"d7ad83d8-ace5-4114-9f18-ea209490b976\" (UID: \"d7ad83d8-ace5-4114-9f18-ea209490b976\") "
Apr 17 07:58:59.110803 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:59.110660    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ad83d8-ace5-4114-9f18-ea209490b976-bundle\") pod \"d7ad83d8-ace5-4114-9f18-ea209490b976\" (UID: \"d7ad83d8-ace5-4114-9f18-ea209490b976\") "
Apr 17 07:58:59.111069 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:59.111041    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ad83d8-ace5-4114-9f18-ea209490b976-bundle" (OuterVolumeSpecName: "bundle") pod "d7ad83d8-ace5-4114-9f18-ea209490b976" (UID: "d7ad83d8-ace5-4114-9f18-ea209490b976"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 07:58:59.112758 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:59.112737    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ad83d8-ace5-4114-9f18-ea209490b976-kube-api-access-68ss2" (OuterVolumeSpecName: "kube-api-access-68ss2") pod "d7ad83d8-ace5-4114-9f18-ea209490b976" (UID: "d7ad83d8-ace5-4114-9f18-ea209490b976"). InnerVolumeSpecName "kube-api-access-68ss2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 07:58:59.115914 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:59.115887    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ad83d8-ace5-4114-9f18-ea209490b976-util" (OuterVolumeSpecName: "util") pod "d7ad83d8-ace5-4114-9f18-ea209490b976" (UID: "d7ad83d8-ace5-4114-9f18-ea209490b976"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 07:58:59.212121 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:59.212094    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ad83d8-ace5-4114-9f18-ea209490b976-util\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:58:59.212121 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:59.212118    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68ss2\" (UniqueName: \"kubernetes.io/projected/d7ad83d8-ace5-4114-9f18-ea209490b976-kube-api-access-68ss2\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:58:59.212243 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:59.212130    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ad83d8-ace5-4114-9f18-ea209490b976-bundle\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:58:59.861338 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:59.861234    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5" event={"ID":"d7ad83d8-ace5-4114-9f18-ea209490b976","Type":"ContainerDied","Data":"dfcfce5b248af4b653cfcb58c4f3efccbfa0fefa38c29fc1304e8a7b6750b131"}
Apr 17 07:58:59.861338 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:59.861280    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dfcfce5b248af4b653cfcb58c4f3efccbfa0fefa38c29fc1304e8a7b6750b131"
Apr 17 07:58:59.861758 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:58:59.861359    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fct8c5"
Apr 17 07:59:04.864289 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:04.864253    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-597b96b99b-zt5vq"
Apr 17 07:59:20.355017 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.354982    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"]
Apr 17 07:59:20.355635 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.355357    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ad83d8-ace5-4114-9f18-ea209490b976" containerName="util"
Apr 17 07:59:20.355635 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.355376    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ad83d8-ace5-4114-9f18-ea209490b976" containerName="util"
Apr 17 07:59:20.355635 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.355390    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ad83d8-ace5-4114-9f18-ea209490b976" containerName="pull"
Apr 17 07:59:20.355635 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.355398    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ad83d8-ace5-4114-9f18-ea209490b976" containerName="pull"
Apr 17 07:59:20.355635 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.355411    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ad83d8-ace5-4114-9f18-ea209490b976" containerName="extract"
Apr 17 07:59:20.355635 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.355419    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ad83d8-ace5-4114-9f18-ea209490b976" containerName="extract"
Apr 17 07:59:20.355635 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.355528    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="d7ad83d8-ace5-4114-9f18-ea209490b976" containerName="extract"
Apr 17 07:59:20.360266 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.360244    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"
Apr 17 07:59:20.362820 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.362792    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 17 07:59:20.362820 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.362795    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xv8sk\""
Apr 17 07:59:20.362997 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.362860    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 17 07:59:20.365836 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.365815    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"]
Apr 17 07:59:20.467478 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.467447    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n\" (UID: \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"
Apr 17 07:59:20.467627 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.467534    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n\" (UID: \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"
Apr 17 07:59:20.467627 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.467564    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx5fn\" (UniqueName: \"kubernetes.io/projected/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-kube-api-access-zx5fn\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n\" (UID: \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"
Apr 17 07:59:20.568470 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.568419    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n\" (UID: \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"
Apr 17 07:59:20.568647 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.568480    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zx5fn\" (UniqueName: \"kubernetes.io/projected/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-kube-api-access-zx5fn\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n\" (UID: \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"
Apr 17 07:59:20.568647 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.568550    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n\" (UID: \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"
Apr 17 07:59:20.568823 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.568789    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n\" (UID: \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"
Apr 17 07:59:20.568898 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.568865    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n\" (UID: \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"
Apr 17 07:59:20.577635 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.577606    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx5fn\" (UniqueName: \"kubernetes.io/projected/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-kube-api-access-zx5fn\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n\" (UID: \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"
Apr 17 07:59:20.670253 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.670179    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"
Apr 17 07:59:20.788745 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.788710    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"]
Apr 17 07:59:20.792093 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:59:20.792065    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c0d4f42_8df0_48fc_8363_85a2f11cb59b.slice/crio-73eed3d1ad47706326ac85233e1b480b398c6b62fcfc4e2240b639e84a6ef0c6 WatchSource:0}: Error finding container 73eed3d1ad47706326ac85233e1b480b398c6b62fcfc4e2240b639e84a6ef0c6: Status 404 returned error can't find the container with id 73eed3d1ad47706326ac85233e1b480b398c6b62fcfc4e2240b639e84a6ef0c6
Apr 17 07:59:20.925046 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.924963    2571 generic.go:358] "Generic (PLEG): container finished" podID="2c0d4f42-8df0-48fc-8363-85a2f11cb59b" containerID="1c07817cde11ec96be8907baa86ffff3f782ad19155b351a63cc5b9f019c9858" exitCode=0
Apr 17 07:59:20.925046 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.925029    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n" event={"ID":"2c0d4f42-8df0-48fc-8363-85a2f11cb59b","Type":"ContainerDied","Data":"1c07817cde11ec96be8907baa86ffff3f782ad19155b351a63cc5b9f019c9858"}
Apr 17 07:59:20.925221 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:20.925060    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n" event={"ID":"2c0d4f42-8df0-48fc-8363-85a2f11cb59b","Type":"ContainerStarted","Data":"73eed3d1ad47706326ac85233e1b480b398c6b62fcfc4e2240b639e84a6ef0c6"}
Apr 17 07:59:22.932420 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:22.932384    2571 generic.go:358] "Generic (PLEG): container finished" podID="2c0d4f42-8df0-48fc-8363-85a2f11cb59b" containerID="4c453fed2e1fb84ba17e39fbe066cb19f10f947e7eec4438ee5ef773d14a373e" exitCode=0
Apr 17 07:59:22.932808 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:22.932469    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n" event={"ID":"2c0d4f42-8df0-48fc-8363-85a2f11cb59b","Type":"ContainerDied","Data":"4c453fed2e1fb84ba17e39fbe066cb19f10f947e7eec4438ee5ef773d14a373e"}
Apr 17 07:59:23.936948 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:23.936914    2571 generic.go:358] "Generic (PLEG): container finished" podID="2c0d4f42-8df0-48fc-8363-85a2f11cb59b" containerID="f6d1954db536294379c0d698c2c693acc2cb7ff8532dfdce6fef6d51dba8ea0f" exitCode=0
Apr 17 07:59:23.937385 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:23.936987    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n" event={"ID":"2c0d4f42-8df0-48fc-8363-85a2f11cb59b","Type":"ContainerDied","Data":"f6d1954db536294379c0d698c2c693acc2cb7ff8532dfdce6fef6d51dba8ea0f"}
Apr 17 07:59:25.052447 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:25.052410    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"
Apr 17 07:59:25.202209 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:25.202114    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zx5fn\" (UniqueName: \"kubernetes.io/projected/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-kube-api-access-zx5fn\") pod \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\" (UID: \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\") "
Apr 17 07:59:25.202209 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:25.202198    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-util\") pod \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\" (UID: \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\") "
Apr 17 07:59:25.202456 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:25.202222    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-bundle\") pod \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\" (UID: \"2c0d4f42-8df0-48fc-8363-85a2f11cb59b\") "
Apr 17 07:59:25.203068 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:25.203035    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-bundle" (OuterVolumeSpecName: "bundle") pod "2c0d4f42-8df0-48fc-8363-85a2f11cb59b" (UID: "2c0d4f42-8df0-48fc-8363-85a2f11cb59b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 07:59:25.204287 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:25.204253    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-kube-api-access-zx5fn" (OuterVolumeSpecName: "kube-api-access-zx5fn") pod "2c0d4f42-8df0-48fc-8363-85a2f11cb59b" (UID: "2c0d4f42-8df0-48fc-8363-85a2f11cb59b"). InnerVolumeSpecName "kube-api-access-zx5fn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 07:59:25.210276 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:25.210252    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-util" (OuterVolumeSpecName: "util") pod "2c0d4f42-8df0-48fc-8363-85a2f11cb59b" (UID: "2c0d4f42-8df0-48fc-8363-85a2f11cb59b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 07:59:25.303212 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:25.303162    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-util\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:59:25.303212 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:25.303206    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-bundle\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:59:25.303212 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:25.303218    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zx5fn\" (UniqueName: \"kubernetes.io/projected/2c0d4f42-8df0-48fc-8363-85a2f11cb59b-kube-api-access-zx5fn\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:59:25.944582 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:25.944506    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n"
Apr 17 07:59:25.944715 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:25.944507    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dmb4n" event={"ID":"2c0d4f42-8df0-48fc-8363-85a2f11cb59b","Type":"ContainerDied","Data":"73eed3d1ad47706326ac85233e1b480b398c6b62fcfc4e2240b639e84a6ef0c6"}
Apr 17 07:59:25.944715 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:25.944610    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73eed3d1ad47706326ac85233e1b480b398c6b62fcfc4e2240b639e84a6ef0c6"
Apr 17 07:59:34.183946 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.183866    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"]
Apr 17 07:59:34.184288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.184125    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c0d4f42-8df0-48fc-8363-85a2f11cb59b" containerName="extract"
Apr 17 07:59:34.184288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.184135    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c0d4f42-8df0-48fc-8363-85a2f11cb59b" containerName="extract"
Apr 17 07:59:34.184288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.184150    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c0d4f42-8df0-48fc-8363-85a2f11cb59b" containerName="util"
Apr 17 07:59:34.184288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.184155    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c0d4f42-8df0-48fc-8363-85a2f11cb59b" containerName="util"
Apr 17 07:59:34.184288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.184164    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c0d4f42-8df0-48fc-8363-85a2f11cb59b" containerName="pull"
Apr 17 07:59:34.184288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.184169    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c0d4f42-8df0-48fc-8363-85a2f11cb59b" containerName="pull"
Apr 17 07:59:34.184288 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.184209    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="2c0d4f42-8df0-48fc-8363-85a2f11cb59b" containerName="extract"
Apr 17 07:59:34.187220 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.187203    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"
Apr 17 07:59:34.190217 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.190199    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xv8sk\""
Apr 17 07:59:34.190596 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.190578    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 17 07:59:34.191071 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.191051    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 17 07:59:34.205532 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.205507    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"]
Apr 17 07:59:34.267937 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.267912    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/281c3452-3212-4d21-a3a9-c7b592439a14-util\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx\" (UID: \"281c3452-3212-4d21-a3a9-c7b592439a14\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"
Apr 17 07:59:34.268033 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.267957    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsvss\" (UniqueName: \"kubernetes.io/projected/281c3452-3212-4d21-a3a9-c7b592439a14-kube-api-access-qsvss\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx\" (UID: \"281c3452-3212-4d21-a3a9-c7b592439a14\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"
Apr 17 07:59:34.268033 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.267978    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/281c3452-3212-4d21-a3a9-c7b592439a14-bundle\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx\" (UID: \"281c3452-3212-4d21-a3a9-c7b592439a14\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"
Apr 17 07:59:34.368398 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.368365    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qsvss\" (UniqueName: \"kubernetes.io/projected/281c3452-3212-4d21-a3a9-c7b592439a14-kube-api-access-qsvss\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx\" (UID: \"281c3452-3212-4d21-a3a9-c7b592439a14\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"
Apr 17 07:59:34.368398 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.368403    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/281c3452-3212-4d21-a3a9-c7b592439a14-bundle\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx\" (UID: \"281c3452-3212-4d21-a3a9-c7b592439a14\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"
Apr 17 07:59:34.368603 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.368450    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/281c3452-3212-4d21-a3a9-c7b592439a14-util\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx\" (UID: \"281c3452-3212-4d21-a3a9-c7b592439a14\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"
Apr 17 07:59:34.368804 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.368789    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/281c3452-3212-4d21-a3a9-c7b592439a14-util\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx\" (UID: \"281c3452-3212-4d21-a3a9-c7b592439a14\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"
Apr 17 07:59:34.368849 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.368829    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/281c3452-3212-4d21-a3a9-c7b592439a14-bundle\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx\" (UID: \"281c3452-3212-4d21-a3a9-c7b592439a14\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"
Apr 17 07:59:34.389420 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.389397    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsvss\" (UniqueName: \"kubernetes.io/projected/281c3452-3212-4d21-a3a9-c7b592439a14-kube-api-access-qsvss\") pod \"d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx\" (UID: \"281c3452-3212-4d21-a3a9-c7b592439a14\") " pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"
Apr 17 07:59:34.495743 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.495661    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"
Apr 17 07:59:34.622452 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.622409    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"]
Apr 17 07:59:34.625889 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:59:34.625853    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod281c3452_3212_4d21_a3a9_c7b592439a14.slice/crio-4e7c43641f13234a16306d8357c5e01c23258db8c4c181c246092347f186d9ef WatchSource:0}: Error finding container 4e7c43641f13234a16306d8357c5e01c23258db8c4c181c246092347f186d9ef: Status 404 returned error can't find the container with id 4e7c43641f13234a16306d8357c5e01c23258db8c4c181c246092347f186d9ef
Apr 17 07:59:34.869794 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.869762    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"]
Apr 17 07:59:34.872804 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.872783    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:34.879329 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.879301    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Apr 17 07:59:34.879443 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.879396    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Apr 17 07:59:34.879514 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.879399    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-w2djl\""
Apr 17 07:59:34.879593 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.879577    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Apr 17 07:59:34.879725 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.879706    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Apr 17 07:59:34.879787 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.879758    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Apr 17 07:59:34.899163 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.899140    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"]
Apr 17 07:59:34.972950 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.972920    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9feb5804-7f75-4b2e-90f2-fb1c62bd5396-manager-config\") pod \"lws-controller-manager-5c76446df-qdq6q\" (UID: \"9feb5804-7f75-4b2e-90f2-fb1c62bd5396\") " pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:34.973094 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.972974    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9feb5804-7f75-4b2e-90f2-fb1c62bd5396-metrics-cert\") pod \"lws-controller-manager-5c76446df-qdq6q\" (UID: \"9feb5804-7f75-4b2e-90f2-fb1c62bd5396\") " pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:34.973094 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.973001    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mhk5\" (UniqueName: \"kubernetes.io/projected/9feb5804-7f75-4b2e-90f2-fb1c62bd5396-kube-api-access-9mhk5\") pod \"lws-controller-manager-5c76446df-qdq6q\" (UID: \"9feb5804-7f75-4b2e-90f2-fb1c62bd5396\") " pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:34.973094 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.973051    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9feb5804-7f75-4b2e-90f2-fb1c62bd5396-cert\") pod \"lws-controller-manager-5c76446df-qdq6q\" (UID: \"9feb5804-7f75-4b2e-90f2-fb1c62bd5396\") " pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:34.975450 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.975405    2571 generic.go:358] "Generic (PLEG): container finished" podID="281c3452-3212-4d21-a3a9-c7b592439a14" containerID="3b78007b840aee5ca391aa5d9d7bd39735e58bbc11447833864731622c9f9241" exitCode=0
Apr 17 07:59:34.975549 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.975470    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx" event={"ID":"281c3452-3212-4d21-a3a9-c7b592439a14","Type":"ContainerDied","Data":"3b78007b840aee5ca391aa5d9d7bd39735e58bbc11447833864731622c9f9241"}
Apr 17 07:59:34.975549 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:34.975506    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx" event={"ID":"281c3452-3212-4d21-a3a9-c7b592439a14","Type":"ContainerStarted","Data":"4e7c43641f13234a16306d8357c5e01c23258db8c4c181c246092347f186d9ef"}
Apr 17 07:59:35.073732 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:35.073698    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9feb5804-7f75-4b2e-90f2-fb1c62bd5396-manager-config\") pod \"lws-controller-manager-5c76446df-qdq6q\" (UID: \"9feb5804-7f75-4b2e-90f2-fb1c62bd5396\") " pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:35.073906 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:35.073758    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9feb5804-7f75-4b2e-90f2-fb1c62bd5396-metrics-cert\") pod \"lws-controller-manager-5c76446df-qdq6q\" (UID: \"9feb5804-7f75-4b2e-90f2-fb1c62bd5396\") " pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:35.073906 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:35.073789    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9mhk5\" (UniqueName: \"kubernetes.io/projected/9feb5804-7f75-4b2e-90f2-fb1c62bd5396-kube-api-access-9mhk5\") pod \"lws-controller-manager-5c76446df-qdq6q\" (UID: \"9feb5804-7f75-4b2e-90f2-fb1c62bd5396\") " pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:35.073906 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:35.073814    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9feb5804-7f75-4b2e-90f2-fb1c62bd5396-cert\") pod \"lws-controller-manager-5c76446df-qdq6q\" (UID: \"9feb5804-7f75-4b2e-90f2-fb1c62bd5396\") " pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:35.074447 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:35.074404    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9feb5804-7f75-4b2e-90f2-fb1c62bd5396-manager-config\") pod \"lws-controller-manager-5c76446df-qdq6q\" (UID: \"9feb5804-7f75-4b2e-90f2-fb1c62bd5396\") " pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:35.076207 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:35.076178    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9feb5804-7f75-4b2e-90f2-fb1c62bd5396-cert\") pod \"lws-controller-manager-5c76446df-qdq6q\" (UID: \"9feb5804-7f75-4b2e-90f2-fb1c62bd5396\") " pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:35.076481 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:35.076458    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9feb5804-7f75-4b2e-90f2-fb1c62bd5396-metrics-cert\") pod \"lws-controller-manager-5c76446df-qdq6q\" (UID: \"9feb5804-7f75-4b2e-90f2-fb1c62bd5396\") " pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:35.093567 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:35.093541    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mhk5\" (UniqueName: \"kubernetes.io/projected/9feb5804-7f75-4b2e-90f2-fb1c62bd5396-kube-api-access-9mhk5\") pod \"lws-controller-manager-5c76446df-qdq6q\" (UID: \"9feb5804-7f75-4b2e-90f2-fb1c62bd5396\") " pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:35.181531 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:35.181428    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:35.324564 ip-10-0-143-45 kubenswrapper[2571]: W0417 07:59:35.324535    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9feb5804_7f75_4b2e_90f2_fb1c62bd5396.slice/crio-114eeefdd9f8c8a5cb1d87d0a74107332318a4887ea168357927bd6c8966289a WatchSource:0}: Error finding container 114eeefdd9f8c8a5cb1d87d0a74107332318a4887ea168357927bd6c8966289a: Status 404 returned error can't find the container with id 114eeefdd9f8c8a5cb1d87d0a74107332318a4887ea168357927bd6c8966289a
Apr 17 07:59:35.331297 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:35.329614    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"]
Apr 17 07:59:35.981006 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:35.980968    2571 generic.go:358] "Generic (PLEG): container finished" podID="281c3452-3212-4d21-a3a9-c7b592439a14" containerID="0e9cf4d4c87ada5dfd5bdae7961606e3493266c56fa951426e8886dbfb8ffc2e" exitCode=0
Apr 17 07:59:35.981189 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:35.981058    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx" event={"ID":"281c3452-3212-4d21-a3a9-c7b592439a14","Type":"ContainerDied","Data":"0e9cf4d4c87ada5dfd5bdae7961606e3493266c56fa951426e8886dbfb8ffc2e"}
Apr 17 07:59:35.982392 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:35.982339    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q" event={"ID":"9feb5804-7f75-4b2e-90f2-fb1c62bd5396","Type":"ContainerStarted","Data":"114eeefdd9f8c8a5cb1d87d0a74107332318a4887ea168357927bd6c8966289a"}
Apr 17 07:59:36.987396 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:36.987357    2571 generic.go:358] "Generic (PLEG): container finished" podID="281c3452-3212-4d21-a3a9-c7b592439a14" containerID="16eb26dac0bcbc43600b7a4f949e302337d72b3f2d6869224e24856f4f37695e" exitCode=0
Apr 17 07:59:36.987768 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:36.987408    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx" event={"ID":"281c3452-3212-4d21-a3a9-c7b592439a14","Type":"ContainerDied","Data":"16eb26dac0bcbc43600b7a4f949e302337d72b3f2d6869224e24856f4f37695e"}
Apr 17 07:59:37.992078 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:37.992042    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q" event={"ID":"9feb5804-7f75-4b2e-90f2-fb1c62bd5396","Type":"ContainerStarted","Data":"c5f81b288c20e4ed638fdcd8ed5740d8814c780d9136d93f48fbf186ca110ec9"}
Apr 17 07:59:37.992519 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:37.992456    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 07:59:38.036548 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.036492    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q" podStartSLOduration=1.5332935810000001 podStartE2EDuration="4.036473672s" podCreationTimestamp="2026-04-17 07:59:34 +0000 UTC" firstStartedPulling="2026-04-17 07:59:35.326855512 +0000 UTC m=+446.380458716" lastFinishedPulling="2026-04-17 07:59:37.830035614 +0000 UTC m=+448.883638807" observedRunningTime="2026-04-17 07:59:38.033318084 +0000 UTC m=+449.086921307" watchObservedRunningTime="2026-04-17 07:59:38.036473672 +0000 UTC m=+449.090076884"
Apr 17 07:59:38.106558 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.106535    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"
Apr 17 07:59:38.198406 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.198376    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsvss\" (UniqueName: \"kubernetes.io/projected/281c3452-3212-4d21-a3a9-c7b592439a14-kube-api-access-qsvss\") pod \"281c3452-3212-4d21-a3a9-c7b592439a14\" (UID: \"281c3452-3212-4d21-a3a9-c7b592439a14\") "
Apr 17 07:59:38.198584 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.198470    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/281c3452-3212-4d21-a3a9-c7b592439a14-bundle\") pod \"281c3452-3212-4d21-a3a9-c7b592439a14\" (UID: \"281c3452-3212-4d21-a3a9-c7b592439a14\") "
Apr 17 07:59:38.198584 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.198499    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/281c3452-3212-4d21-a3a9-c7b592439a14-util\") pod \"281c3452-3212-4d21-a3a9-c7b592439a14\" (UID: \"281c3452-3212-4d21-a3a9-c7b592439a14\") "
Apr 17 07:59:38.199409 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.199382    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/281c3452-3212-4d21-a3a9-c7b592439a14-bundle" (OuterVolumeSpecName: "bundle") pod "281c3452-3212-4d21-a3a9-c7b592439a14" (UID: "281c3452-3212-4d21-a3a9-c7b592439a14"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 07:59:38.200526 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.200500    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/281c3452-3212-4d21-a3a9-c7b592439a14-kube-api-access-qsvss" (OuterVolumeSpecName: "kube-api-access-qsvss") pod "281c3452-3212-4d21-a3a9-c7b592439a14" (UID: "281c3452-3212-4d21-a3a9-c7b592439a14"). InnerVolumeSpecName "kube-api-access-qsvss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 07:59:38.204667 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.204649    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/281c3452-3212-4d21-a3a9-c7b592439a14-util" (OuterVolumeSpecName: "util") pod "281c3452-3212-4d21-a3a9-c7b592439a14" (UID: "281c3452-3212-4d21-a3a9-c7b592439a14"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 07:59:38.299383 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.299339    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsvss\" (UniqueName: \"kubernetes.io/projected/281c3452-3212-4d21-a3a9-c7b592439a14-kube-api-access-qsvss\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:59:38.299383 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.299382    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/281c3452-3212-4d21-a3a9-c7b592439a14-bundle\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:59:38.299621 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.299398    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/281c3452-3212-4d21-a3a9-c7b592439a14-util\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 07:59:38.996932 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.996901    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx"
Apr 17 07:59:38.996932 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.996920    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d813cbca2f2d4ae5bb9c3e9ca6bc9dc97fa22f4f10cc797dd3b2c1f0c2t4dgx" event={"ID":"281c3452-3212-4d21-a3a9-c7b592439a14","Type":"ContainerDied","Data":"4e7c43641f13234a16306d8357c5e01c23258db8c4c181c246092347f186d9ef"}
Apr 17 07:59:38.997342 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:38.996954    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e7c43641f13234a16306d8357c5e01c23258db8c4c181c246092347f186d9ef"
Apr 17 07:59:50.002860 ip-10-0-143-45 kubenswrapper[2571]: I0417 07:59:50.002833    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c76446df-qdq6q"
Apr 17 08:00:05.511405 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.511367    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"]
Apr 17 08:00:05.511895 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.511648    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="281c3452-3212-4d21-a3a9-c7b592439a14" containerName="extract"
Apr 17 08:00:05.511895 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.511661    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="281c3452-3212-4d21-a3a9-c7b592439a14" containerName="extract"
Apr 17 08:00:05.511895 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.511678    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="281c3452-3212-4d21-a3a9-c7b592439a14" containerName="pull"
Apr 17 08:00:05.511895 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.511684    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="281c3452-3212-4d21-a3a9-c7b592439a14" containerName="pull"
Apr 17 08:00:05.511895 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.511692    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="281c3452-3212-4d21-a3a9-c7b592439a14" containerName="util"
Apr 17 08:00:05.511895 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.511697    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="281c3452-3212-4d21-a3a9-c7b592439a14" containerName="util"
Apr 17 08:00:05.511895 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.511747    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="281c3452-3212-4d21-a3a9-c7b592439a14" containerName="extract"
Apr 17 08:00:05.514644 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.514628    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"
Apr 17 08:00:05.516945 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.516918    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Apr 17 08:00:05.517970 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.517950    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Apr 17 08:00:05.518355 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.518336    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xv8sk\""
Apr 17 08:00:05.521131 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.521110    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"]
Apr 17 08:00:05.598609 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.598576    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41972c34-2df9-40b2-a166-8b34dbb41806-bundle\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx\" (UID: \"41972c34-2df9-40b2-a166-8b34dbb41806\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"
Apr 17 08:00:05.598785 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.598630    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md7pj\" (UniqueName: \"kubernetes.io/projected/41972c34-2df9-40b2-a166-8b34dbb41806-kube-api-access-md7pj\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx\" (UID: \"41972c34-2df9-40b2-a166-8b34dbb41806\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"
Apr 17 08:00:05.598785 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.598689    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41972c34-2df9-40b2-a166-8b34dbb41806-util\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx\" (UID: \"41972c34-2df9-40b2-a166-8b34dbb41806\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"
Apr 17 08:00:05.612582 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.612544    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"]
Apr 17 08:00:05.617279 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.617255    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"
Apr 17 08:00:05.620476 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.620451    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"]
Apr 17 08:00:05.699975 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.699940    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41972c34-2df9-40b2-a166-8b34dbb41806-bundle\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx\" (UID: \"41972c34-2df9-40b2-a166-8b34dbb41806\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"
Apr 17 08:00:05.699975 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.699988    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5449cdc8-0ead-4013-bbe2-4a73019e2734-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm\" (UID: \"5449cdc8-0ead-4013-bbe2-4a73019e2734\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"
Apr 17 08:00:05.700203 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.700012    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qg5p\" (UniqueName: \"kubernetes.io/projected/5449cdc8-0ead-4013-bbe2-4a73019e2734-kube-api-access-8qg5p\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm\" (UID: \"5449cdc8-0ead-4013-bbe2-4a73019e2734\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"
Apr 17 08:00:05.700203 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.700085    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-md7pj\" (UniqueName: \"kubernetes.io/projected/41972c34-2df9-40b2-a166-8b34dbb41806-kube-api-access-md7pj\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx\" (UID: \"41972c34-2df9-40b2-a166-8b34dbb41806\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"
Apr 17 08:00:05.700203 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.700126    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5449cdc8-0ead-4013-bbe2-4a73019e2734-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm\" (UID: \"5449cdc8-0ead-4013-bbe2-4a73019e2734\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"
Apr 17 08:00:05.700203 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.700159    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41972c34-2df9-40b2-a166-8b34dbb41806-util\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx\" (UID: \"41972c34-2df9-40b2-a166-8b34dbb41806\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"
Apr 17 08:00:05.700341 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.700311    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41972c34-2df9-40b2-a166-8b34dbb41806-bundle\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx\" (UID: \"41972c34-2df9-40b2-a166-8b34dbb41806\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"
Apr 17 08:00:05.700446 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.700418    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41972c34-2df9-40b2-a166-8b34dbb41806-util\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx\" (UID: \"41972c34-2df9-40b2-a166-8b34dbb41806\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"
Apr 17 08:00:05.710492 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.710466    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"]
Apr 17 08:00:05.713485 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.713460    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-md7pj\" (UniqueName: \"kubernetes.io/projected/41972c34-2df9-40b2-a166-8b34dbb41806-kube-api-access-md7pj\") pod \"5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx\" (UID: \"41972c34-2df9-40b2-a166-8b34dbb41806\") " pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"
Apr 17 08:00:05.713735 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.713721    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"
Apr 17 08:00:05.721153 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.721132    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"]
Apr 17 08:00:05.800694 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.800657    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8cs9\" (UniqueName: \"kubernetes.io/projected/ef18a471-e3c5-4baf-ac41-299748334698-kube-api-access-t8cs9\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz\" (UID: \"ef18a471-e3c5-4baf-ac41-299748334698\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"
Apr 17 08:00:05.800694 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.800695    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef18a471-e3c5-4baf-ac41-299748334698-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz\" (UID: \"ef18a471-e3c5-4baf-ac41-299748334698\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"
Apr 17 08:00:05.800924 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.800757    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5449cdc8-0ead-4013-bbe2-4a73019e2734-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm\" (UID: \"5449cdc8-0ead-4013-bbe2-4a73019e2734\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"
Apr 17 08:00:05.800924 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.800775    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef18a471-e3c5-4baf-ac41-299748334698-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz\" (UID: \"ef18a471-e3c5-4baf-ac41-299748334698\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"
Apr 17 08:00:05.800924 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.800795    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8qg5p\" (UniqueName: \"kubernetes.io/projected/5449cdc8-0ead-4013-bbe2-4a73019e2734-kube-api-access-8qg5p\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm\" (UID: \"5449cdc8-0ead-4013-bbe2-4a73019e2734\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"
Apr 17 08:00:05.800924 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.800870    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5449cdc8-0ead-4013-bbe2-4a73019e2734-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm\" (UID: \"5449cdc8-0ead-4013-bbe2-4a73019e2734\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"
Apr 17 08:00:05.801148 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.801133    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5449cdc8-0ead-4013-bbe2-4a73019e2734-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm\" (UID: \"5449cdc8-0ead-4013-bbe2-4a73019e2734\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"
Apr 17 08:00:05.801196 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.801159    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5449cdc8-0ead-4013-bbe2-4a73019e2734-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm\" (UID: \"5449cdc8-0ead-4013-bbe2-4a73019e2734\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"
Apr 17 08:00:05.813616 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.813592    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"]
Apr 17 08:00:05.816133 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.816103    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qg5p\" (UniqueName: \"kubernetes.io/projected/5449cdc8-0ead-4013-bbe2-4a73019e2734-kube-api-access-8qg5p\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm\" (UID: \"5449cdc8-0ead-4013-bbe2-4a73019e2734\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"
Apr 17 08:00:05.816954 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.816938    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"
Apr 17 08:00:05.824578 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.824559    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"
Apr 17 08:00:05.829504 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.829484    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"]
Apr 17 08:00:05.901991 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.901954    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef18a471-e3c5-4baf-ac41-299748334698-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz\" (UID: \"ef18a471-e3c5-4baf-ac41-299748334698\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"
Apr 17 08:00:05.902114 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.902008    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tth2\" (UniqueName: \"kubernetes.io/projected/1c818b50-6a17-459e-95f7-998f0e0aab28-kube-api-access-9tth2\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg\" (UID: \"1c818b50-6a17-459e-95f7-998f0e0aab28\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"
Apr 17 08:00:05.902114 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.902097    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef18a471-e3c5-4baf-ac41-299748334698-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz\" (UID: \"ef18a471-e3c5-4baf-ac41-299748334698\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"
Apr 17 08:00:05.902233 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.902162    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c818b50-6a17-459e-95f7-998f0e0aab28-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg\" (UID: \"1c818b50-6a17-459e-95f7-998f0e0aab28\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"
Apr 17 08:00:05.902233 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.902190    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t8cs9\" (UniqueName: \"kubernetes.io/projected/ef18a471-e3c5-4baf-ac41-299748334698-kube-api-access-t8cs9\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz\" (UID: \"ef18a471-e3c5-4baf-ac41-299748334698\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"
Apr 17 08:00:05.902233 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.902218    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c818b50-6a17-459e-95f7-998f0e0aab28-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg\" (UID: \"1c818b50-6a17-459e-95f7-998f0e0aab28\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"
Apr 17 08:00:05.902563 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.902381    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef18a471-e3c5-4baf-ac41-299748334698-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz\" (UID: \"ef18a471-e3c5-4baf-ac41-299748334698\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"
Apr 17 08:00:05.902629 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.902583    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef18a471-e3c5-4baf-ac41-299748334698-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz\" (UID: \"ef18a471-e3c5-4baf-ac41-299748334698\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"
Apr 17 08:00:05.911042 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.911015    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8cs9\" (UniqueName: \"kubernetes.io/projected/ef18a471-e3c5-4baf-ac41-299748334698-kube-api-access-t8cs9\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz\" (UID: \"ef18a471-e3c5-4baf-ac41-299748334698\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"
Apr 17 08:00:05.927735 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.927707    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"
Apr 17 08:00:05.946103 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:05.946074    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"]
Apr 17 08:00:05.947496 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:00:05.947470    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41972c34_2df9_40b2_a166_8b34dbb41806.slice/crio-18fd909a39e1e341cfbdb4cd2c13a8e4641ea841db882e216503e75239b3659e WatchSource:0}: Error finding container 18fd909a39e1e341cfbdb4cd2c13a8e4641ea841db882e216503e75239b3659e: Status 404 returned error can't find the container with id 18fd909a39e1e341cfbdb4cd2c13a8e4641ea841db882e216503e75239b3659e
Apr 17 08:00:06.003086 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.003046    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c818b50-6a17-459e-95f7-998f0e0aab28-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg\" (UID: \"1c818b50-6a17-459e-95f7-998f0e0aab28\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"
Apr 17 08:00:06.003197 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.003120    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c818b50-6a17-459e-95f7-998f0e0aab28-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg\" (UID: \"1c818b50-6a17-459e-95f7-998f0e0aab28\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"
Apr 17 08:00:06.003197 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.003153    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9tth2\" (UniqueName: \"kubernetes.io/projected/1c818b50-6a17-459e-95f7-998f0e0aab28-kube-api-access-9tth2\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg\" (UID: \"1c818b50-6a17-459e-95f7-998f0e0aab28\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"
Apr 17 08:00:06.003450 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.003403    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c818b50-6a17-459e-95f7-998f0e0aab28-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg\" (UID: \"1c818b50-6a17-459e-95f7-998f0e0aab28\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"
Apr 17 08:00:06.003663 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.003629    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c818b50-6a17-459e-95f7-998f0e0aab28-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg\" (UID: \"1c818b50-6a17-459e-95f7-998f0e0aab28\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"
Apr 17 08:00:06.011483 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.011461    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tth2\" (UniqueName: \"kubernetes.io/projected/1c818b50-6a17-459e-95f7-998f0e0aab28-kube-api-access-9tth2\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg\" (UID: \"1c818b50-6a17-459e-95f7-998f0e0aab28\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"
Apr 17 08:00:06.029120 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.029087    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"
Apr 17 08:00:06.054417 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.054388    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"]
Apr 17 08:00:06.056498 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:00:06.056469    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5449cdc8_0ead_4013_bbe2_4a73019e2734.slice/crio-172ee0c6200a39fc38b73bc35dc8335a25b5b79090ef49cc1063a7d5afd8bf05 WatchSource:0}: Error finding container 172ee0c6200a39fc38b73bc35dc8335a25b5b79090ef49cc1063a7d5afd8bf05: Status 404 returned error can't find the container with id 172ee0c6200a39fc38b73bc35dc8335a25b5b79090ef49cc1063a7d5afd8bf05
Apr 17 08:00:06.088031 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.087996    2571 generic.go:358] "Generic (PLEG): container finished" podID="41972c34-2df9-40b2-a166-8b34dbb41806" containerID="9c6dedfa5d5ed65356c3bbef94d84aade2b60a45c6518ece872aebdf4bb78e3a" exitCode=0
Apr 17 08:00:06.088146 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.088096    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx" event={"ID":"41972c34-2df9-40b2-a166-8b34dbb41806","Type":"ContainerDied","Data":"9c6dedfa5d5ed65356c3bbef94d84aade2b60a45c6518ece872aebdf4bb78e3a"}
Apr 17 08:00:06.088146 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.088139    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx" event={"ID":"41972c34-2df9-40b2-a166-8b34dbb41806","Type":"ContainerStarted","Data":"18fd909a39e1e341cfbdb4cd2c13a8e4641ea841db882e216503e75239b3659e"}
Apr 17 08:00:06.089554 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.089521    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm" event={"ID":"5449cdc8-0ead-4013-bbe2-4a73019e2734","Type":"ContainerStarted","Data":"172ee0c6200a39fc38b73bc35dc8335a25b5b79090ef49cc1063a7d5afd8bf05"}
Apr 17 08:00:06.131703 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.131412    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"
Apr 17 08:00:06.161484 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.161348    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"]
Apr 17 08:00:06.163959 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:00:06.163926    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef18a471_e3c5_4baf_ac41_299748334698.slice/crio-df0cef474d21a7bd1ee8bcb8812e821467b9d6fc3191176342cadc4f7ca3175e WatchSource:0}: Error finding container df0cef474d21a7bd1ee8bcb8812e821467b9d6fc3191176342cadc4f7ca3175e: Status 404 returned error can't find the container with id df0cef474d21a7bd1ee8bcb8812e821467b9d6fc3191176342cadc4f7ca3175e
Apr 17 08:00:06.259519 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:06.259493    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"]
Apr 17 08:00:06.261064 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:00:06.261039    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c818b50_6a17_459e_95f7_998f0e0aab28.slice/crio-ff34cd3f0aa0443f70ba679a4c85117fb105776ea0363ad7a369116d693cece4 WatchSource:0}: Error finding container ff34cd3f0aa0443f70ba679a4c85117fb105776ea0363ad7a369116d693cece4: Status 404 returned error can't find the container with id ff34cd3f0aa0443f70ba679a4c85117fb105776ea0363ad7a369116d693cece4
Apr 17 08:00:07.093966 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:07.093931    2571 generic.go:358] "Generic (PLEG): container finished" podID="1c818b50-6a17-459e-95f7-998f0e0aab28" containerID="437f9fc10d0110b8af617df8a4a7c513e471549d4a7676c643dd56493deb163a" exitCode=0
Apr 17 08:00:07.094409 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:07.094000    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg" event={"ID":"1c818b50-6a17-459e-95f7-998f0e0aab28","Type":"ContainerDied","Data":"437f9fc10d0110b8af617df8a4a7c513e471549d4a7676c643dd56493deb163a"}
Apr 17 08:00:07.094409 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:07.094028    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg" event={"ID":"1c818b50-6a17-459e-95f7-998f0e0aab28","Type":"ContainerStarted","Data":"ff34cd3f0aa0443f70ba679a4c85117fb105776ea0363ad7a369116d693cece4"}
Apr 17 08:00:07.095601 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:07.095458    2571 generic.go:358] "Generic (PLEG): container finished" podID="ef18a471-e3c5-4baf-ac41-299748334698" containerID="c3444ec7d5abdef72b1e18bc130496c0c52d93419b561d87b1d0eb01a64f5719" exitCode=0
Apr 17 08:00:07.095601 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:07.095540    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz" event={"ID":"ef18a471-e3c5-4baf-ac41-299748334698","Type":"ContainerDied","Data":"c3444ec7d5abdef72b1e18bc130496c0c52d93419b561d87b1d0eb01a64f5719"}
Apr 17 08:00:07.095601 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:07.095569    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz" event={"ID":"ef18a471-e3c5-4baf-ac41-299748334698","Type":"ContainerStarted","Data":"df0cef474d21a7bd1ee8bcb8812e821467b9d6fc3191176342cadc4f7ca3175e"}
Apr 17 08:00:07.096977 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:07.096936    2571 generic.go:358] "Generic (PLEG): container finished" podID="5449cdc8-0ead-4013-bbe2-4a73019e2734" containerID="56e36fe65ab1ac5a6a1490906d9d3c3baca983d0177f968e937f667c119e1169" exitCode=0
Apr 17 08:00:07.097049 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:07.096971    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm" event={"ID":"5449cdc8-0ead-4013-bbe2-4a73019e2734","Type":"ContainerDied","Data":"56e36fe65ab1ac5a6a1490906d9d3c3baca983d0177f968e937f667c119e1169"}
Apr 17 08:00:11.116628 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:11.116592    2571 generic.go:358] "Generic (PLEG): container finished" podID="41972c34-2df9-40b2-a166-8b34dbb41806" containerID="1d6f0bb438b378e43b45ece856e750427fbfc33afdf64c28be7ad6e68b9d4d64" exitCode=0
Apr 17 08:00:11.117090 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:11.116662    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx" event={"ID":"41972c34-2df9-40b2-a166-8b34dbb41806","Type":"ContainerDied","Data":"1d6f0bb438b378e43b45ece856e750427fbfc33afdf64c28be7ad6e68b9d4d64"}
Apr 17 08:00:12.122227 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:12.122192    2571 generic.go:358] "Generic (PLEG): container finished" podID="41972c34-2df9-40b2-a166-8b34dbb41806" containerID="296069e4eb8a3aba97dfdd0b5008bc70379c22302b9124bed0209bf80f1135e4" exitCode=0
Apr 17 08:00:12.122654 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:12.122282    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx" event={"ID":"41972c34-2df9-40b2-a166-8b34dbb41806","Type":"ContainerDied","Data":"296069e4eb8a3aba97dfdd0b5008bc70379c22302b9124bed0209bf80f1135e4"}
Apr 17 08:00:12.123822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:12.123802    2571 generic.go:358] "Generic (PLEG): container finished" podID="ef18a471-e3c5-4baf-ac41-299748334698" containerID="8dfa214802fa4268a2990a619857ab2a1cd1fbc2859cae100cb2b2745e38a209" exitCode=0
Apr 17 08:00:12.123922 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:12.123831    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz" event={"ID":"ef18a471-e3c5-4baf-ac41-299748334698","Type":"ContainerDied","Data":"8dfa214802fa4268a2990a619857ab2a1cd1fbc2859cae100cb2b2745e38a209"}
Apr 17 08:00:13.129172 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.129132    2571 generic.go:358] "Generic (PLEG): container finished" podID="ef18a471-e3c5-4baf-ac41-299748334698" containerID="8ed3308ef4e9cb80a682428875f713a77be31288869f01df5f1d3360517a44cf" exitCode=0
Apr 17 08:00:13.129619 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.129223    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz" event={"ID":"ef18a471-e3c5-4baf-ac41-299748334698","Type":"ContainerDied","Data":"8ed3308ef4e9cb80a682428875f713a77be31288869f01df5f1d3360517a44cf"}
Apr 17 08:00:13.130695 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.130674    2571 generic.go:358] "Generic (PLEG): container finished" podID="5449cdc8-0ead-4013-bbe2-4a73019e2734" containerID="3c0238a3c1c7ffb539054eee7a3733ce7d3b07c4700dd43fe882445ce86678da" exitCode=0
Apr 17 08:00:13.130806 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.130697    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm" event={"ID":"5449cdc8-0ead-4013-bbe2-4a73019e2734","Type":"ContainerDied","Data":"3c0238a3c1c7ffb539054eee7a3733ce7d3b07c4700dd43fe882445ce86678da"}
Apr 17 08:00:13.323083 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.323061    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"
Apr 17 08:00:13.359576 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.359545    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41972c34-2df9-40b2-a166-8b34dbb41806-bundle\") pod \"41972c34-2df9-40b2-a166-8b34dbb41806\" (UID: \"41972c34-2df9-40b2-a166-8b34dbb41806\") "
Apr 17 08:00:13.359713 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.359624    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41972c34-2df9-40b2-a166-8b34dbb41806-util\") pod \"41972c34-2df9-40b2-a166-8b34dbb41806\" (UID: \"41972c34-2df9-40b2-a166-8b34dbb41806\") "
Apr 17 08:00:13.359713 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.359662    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-md7pj\" (UniqueName: \"kubernetes.io/projected/41972c34-2df9-40b2-a166-8b34dbb41806-kube-api-access-md7pj\") pod \"41972c34-2df9-40b2-a166-8b34dbb41806\" (UID: \"41972c34-2df9-40b2-a166-8b34dbb41806\") "
Apr 17 08:00:13.360113 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.360086    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41972c34-2df9-40b2-a166-8b34dbb41806-bundle" (OuterVolumeSpecName: "bundle") pod "41972c34-2df9-40b2-a166-8b34dbb41806" (UID: "41972c34-2df9-40b2-a166-8b34dbb41806"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:00:13.361725 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.361701    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41972c34-2df9-40b2-a166-8b34dbb41806-kube-api-access-md7pj" (OuterVolumeSpecName: "kube-api-access-md7pj") pod "41972c34-2df9-40b2-a166-8b34dbb41806" (UID: "41972c34-2df9-40b2-a166-8b34dbb41806"). InnerVolumeSpecName "kube-api-access-md7pj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:00:13.364131 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.364095    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41972c34-2df9-40b2-a166-8b34dbb41806-util" (OuterVolumeSpecName: "util") pod "41972c34-2df9-40b2-a166-8b34dbb41806" (UID: "41972c34-2df9-40b2-a166-8b34dbb41806"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:00:13.460562 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.460486    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41972c34-2df9-40b2-a166-8b34dbb41806-bundle\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:00:13.460562 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.460513    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41972c34-2df9-40b2-a166-8b34dbb41806-util\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:00:13.460562 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:13.460523    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-md7pj\" (UniqueName: \"kubernetes.io/projected/41972c34-2df9-40b2-a166-8b34dbb41806-kube-api-access-md7pj\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:00:14.136265 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.136229    2571 generic.go:358] "Generic (PLEG): container finished" podID="1c818b50-6a17-459e-95f7-998f0e0aab28" containerID="107a2472a5593c868e1e4cb93a36837b459c94a0e6f04ebf8aed55dfb2725050" exitCode=0
Apr 17 08:00:14.136721 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.136306    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg" event={"ID":"1c818b50-6a17-459e-95f7-998f0e0aab28","Type":"ContainerDied","Data":"107a2472a5593c868e1e4cb93a36837b459c94a0e6f04ebf8aed55dfb2725050"}
Apr 17 08:00:14.138219 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.138197    2571 generic.go:358] "Generic (PLEG): container finished" podID="5449cdc8-0ead-4013-bbe2-4a73019e2734" containerID="e07dcdb3a65a6f18b30be92ce435f78374e81b3ca8a5cb3b39896ef387c20685" exitCode=0
Apr 17 08:00:14.138318 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.138237    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm" event={"ID":"5449cdc8-0ead-4013-bbe2-4a73019e2734","Type":"ContainerDied","Data":"e07dcdb3a65a6f18b30be92ce435f78374e81b3ca8a5cb3b39896ef387c20685"}
Apr 17 08:00:14.139930 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.139874    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx"
Apr 17 08:00:14.140035 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.139890    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5b107722e7a95ad7441cca2226d3d656a8bf76dc65e15066bc01f8e503gxrmx" event={"ID":"41972c34-2df9-40b2-a166-8b34dbb41806","Type":"ContainerDied","Data":"18fd909a39e1e341cfbdb4cd2c13a8e4641ea841db882e216503e75239b3659e"}
Apr 17 08:00:14.140035 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.139952    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18fd909a39e1e341cfbdb4cd2c13a8e4641ea841db882e216503e75239b3659e"
Apr 17 08:00:14.276818 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.276794    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"
Apr 17 08:00:14.368647 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.368620    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8cs9\" (UniqueName: \"kubernetes.io/projected/ef18a471-e3c5-4baf-ac41-299748334698-kube-api-access-t8cs9\") pod \"ef18a471-e3c5-4baf-ac41-299748334698\" (UID: \"ef18a471-e3c5-4baf-ac41-299748334698\") "
Apr 17 08:00:14.368801 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.368689    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef18a471-e3c5-4baf-ac41-299748334698-bundle\") pod \"ef18a471-e3c5-4baf-ac41-299748334698\" (UID: \"ef18a471-e3c5-4baf-ac41-299748334698\") "
Apr 17 08:00:14.368801 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.368710    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef18a471-e3c5-4baf-ac41-299748334698-util\") pod \"ef18a471-e3c5-4baf-ac41-299748334698\" (UID: \"ef18a471-e3c5-4baf-ac41-299748334698\") "
Apr 17 08:00:14.369163 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.369136    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef18a471-e3c5-4baf-ac41-299748334698-bundle" (OuterVolumeSpecName: "bundle") pod "ef18a471-e3c5-4baf-ac41-299748334698" (UID: "ef18a471-e3c5-4baf-ac41-299748334698"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:00:14.370555 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.370532    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef18a471-e3c5-4baf-ac41-299748334698-kube-api-access-t8cs9" (OuterVolumeSpecName: "kube-api-access-t8cs9") pod "ef18a471-e3c5-4baf-ac41-299748334698" (UID: "ef18a471-e3c5-4baf-ac41-299748334698"). InnerVolumeSpecName "kube-api-access-t8cs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:00:14.373675 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.373643    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef18a471-e3c5-4baf-ac41-299748334698-util" (OuterVolumeSpecName: "util") pod "ef18a471-e3c5-4baf-ac41-299748334698" (UID: "ef18a471-e3c5-4baf-ac41-299748334698"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:00:14.469487 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.469383    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef18a471-e3c5-4baf-ac41-299748334698-bundle\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:00:14.469487 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.469412    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef18a471-e3c5-4baf-ac41-299748334698-util\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:00:14.469487 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:14.469422    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8cs9\" (UniqueName: \"kubernetes.io/projected/ef18a471-e3c5-4baf-ac41-299748334698-kube-api-access-t8cs9\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:00:15.144706 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.144674    2571 generic.go:358] "Generic (PLEG): container finished" podID="1c818b50-6a17-459e-95f7-998f0e0aab28" containerID="4a06026075e12dd047fcf13fafd2e0e06f821326d0c832c907e015f3bdd0f63c" exitCode=0
Apr 17 08:00:15.145124 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.144742    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg" event={"ID":"1c818b50-6a17-459e-95f7-998f0e0aab28","Type":"ContainerDied","Data":"4a06026075e12dd047fcf13fafd2e0e06f821326d0c832c907e015f3bdd0f63c"}
Apr 17 08:00:15.146353 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.146328    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz"
Apr 17 08:00:15.146353 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.146343    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bh5ngz" event={"ID":"ef18a471-e3c5-4baf-ac41-299748334698","Type":"ContainerDied","Data":"df0cef474d21a7bd1ee8bcb8812e821467b9d6fc3191176342cadc4f7ca3175e"}
Apr 17 08:00:15.146528 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.146370    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df0cef474d21a7bd1ee8bcb8812e821467b9d6fc3191176342cadc4f7ca3175e"
Apr 17 08:00:15.268370 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.268349    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"
Apr 17 08:00:15.376389 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.376358    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qg5p\" (UniqueName: \"kubernetes.io/projected/5449cdc8-0ead-4013-bbe2-4a73019e2734-kube-api-access-8qg5p\") pod \"5449cdc8-0ead-4013-bbe2-4a73019e2734\" (UID: \"5449cdc8-0ead-4013-bbe2-4a73019e2734\") "
Apr 17 08:00:15.376567 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.376408    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5449cdc8-0ead-4013-bbe2-4a73019e2734-bundle\") pod \"5449cdc8-0ead-4013-bbe2-4a73019e2734\" (UID: \"5449cdc8-0ead-4013-bbe2-4a73019e2734\") "
Apr 17 08:00:15.376567 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.376471    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5449cdc8-0ead-4013-bbe2-4a73019e2734-util\") pod \"5449cdc8-0ead-4013-bbe2-4a73019e2734\" (UID: \"5449cdc8-0ead-4013-bbe2-4a73019e2734\") "
Apr 17 08:00:15.376868 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.376844    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5449cdc8-0ead-4013-bbe2-4a73019e2734-bundle" (OuterVolumeSpecName: "bundle") pod "5449cdc8-0ead-4013-bbe2-4a73019e2734" (UID: "5449cdc8-0ead-4013-bbe2-4a73019e2734"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:00:15.378467 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.378422    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5449cdc8-0ead-4013-bbe2-4a73019e2734-kube-api-access-8qg5p" (OuterVolumeSpecName: "kube-api-access-8qg5p") pod "5449cdc8-0ead-4013-bbe2-4a73019e2734" (UID: "5449cdc8-0ead-4013-bbe2-4a73019e2734"). InnerVolumeSpecName "kube-api-access-8qg5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:00:15.381071 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.381039    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5449cdc8-0ead-4013-bbe2-4a73019e2734-util" (OuterVolumeSpecName: "util") pod "5449cdc8-0ead-4013-bbe2-4a73019e2734" (UID: "5449cdc8-0ead-4013-bbe2-4a73019e2734"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:00:15.477382 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.477315    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qg5p\" (UniqueName: \"kubernetes.io/projected/5449cdc8-0ead-4013-bbe2-4a73019e2734-kube-api-access-8qg5p\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:00:15.477382 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.477342    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5449cdc8-0ead-4013-bbe2-4a73019e2734-bundle\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:00:15.477382 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:15.477351    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5449cdc8-0ead-4013-bbe2-4a73019e2734-util\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:00:16.151418 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:16.151382    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm" event={"ID":"5449cdc8-0ead-4013-bbe2-4a73019e2734","Type":"ContainerDied","Data":"172ee0c6200a39fc38b73bc35dc8335a25b5b79090ef49cc1063a7d5afd8bf05"}
Apr 17 08:00:16.151418 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:16.151410    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec887rlhm"
Apr 17 08:00:16.151418 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:16.151420    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="172ee0c6200a39fc38b73bc35dc8335a25b5b79090ef49cc1063a7d5afd8bf05"
Apr 17 08:00:16.267176 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:16.267154    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"
Apr 17 08:00:16.383215 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:16.383172    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c818b50-6a17-459e-95f7-998f0e0aab28-util\") pod \"1c818b50-6a17-459e-95f7-998f0e0aab28\" (UID: \"1c818b50-6a17-459e-95f7-998f0e0aab28\") "
Apr 17 08:00:16.383358 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:16.383257    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tth2\" (UniqueName: \"kubernetes.io/projected/1c818b50-6a17-459e-95f7-998f0e0aab28-kube-api-access-9tth2\") pod \"1c818b50-6a17-459e-95f7-998f0e0aab28\" (UID: \"1c818b50-6a17-459e-95f7-998f0e0aab28\") "
Apr 17 08:00:16.383358 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:16.383280    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c818b50-6a17-459e-95f7-998f0e0aab28-bundle\") pod \"1c818b50-6a17-459e-95f7-998f0e0aab28\" (UID: \"1c818b50-6a17-459e-95f7-998f0e0aab28\") "
Apr 17 08:00:16.383823 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:16.383795    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c818b50-6a17-459e-95f7-998f0e0aab28-bundle" (OuterVolumeSpecName: "bundle") pod "1c818b50-6a17-459e-95f7-998f0e0aab28" (UID: "1c818b50-6a17-459e-95f7-998f0e0aab28"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:00:16.385399 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:16.385373    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c818b50-6a17-459e-95f7-998f0e0aab28-kube-api-access-9tth2" (OuterVolumeSpecName: "kube-api-access-9tth2") pod "1c818b50-6a17-459e-95f7-998f0e0aab28" (UID: "1c818b50-6a17-459e-95f7-998f0e0aab28"). InnerVolumeSpecName "kube-api-access-9tth2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:00:16.387918 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:16.387893    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c818b50-6a17-459e-95f7-998f0e0aab28-util" (OuterVolumeSpecName: "util") pod "1c818b50-6a17-459e-95f7-998f0e0aab28" (UID: "1c818b50-6a17-459e-95f7-998f0e0aab28"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:00:16.484374 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:16.484305    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tth2\" (UniqueName: \"kubernetes.io/projected/1c818b50-6a17-459e-95f7-998f0e0aab28-kube-api-access-9tth2\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:00:16.484374 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:16.484329    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c818b50-6a17-459e-95f7-998f0e0aab28-bundle\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:00:16.484374 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:16.484339    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c818b50-6a17-459e-95f7-998f0e0aab28-util\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:00:17.156716 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:17.156680    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg" event={"ID":"1c818b50-6a17-459e-95f7-998f0e0aab28","Type":"ContainerDied","Data":"ff34cd3f0aa0443f70ba679a4c85117fb105776ea0363ad7a369116d693cece4"}
Apr 17 08:00:17.156716 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:17.156704    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c308lwvg"
Apr 17 08:00:17.156716 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:17.156713    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff34cd3f0aa0443f70ba679a4c85117fb105776ea0363ad7a369116d693cece4"
Apr 17 08:00:33.651397 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651361    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-4bcjv"]
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651640    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c818b50-6a17-459e-95f7-998f0e0aab28" containerName="extract"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651651    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c818b50-6a17-459e-95f7-998f0e0aab28" containerName="extract"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651664    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41972c34-2df9-40b2-a166-8b34dbb41806" containerName="util"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651670    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="41972c34-2df9-40b2-a166-8b34dbb41806" containerName="util"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651681    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef18a471-e3c5-4baf-ac41-299748334698" containerName="util"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651690    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef18a471-e3c5-4baf-ac41-299748334698" containerName="util"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651698    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef18a471-e3c5-4baf-ac41-299748334698" containerName="extract"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651703    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef18a471-e3c5-4baf-ac41-299748334698" containerName="extract"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651710    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5449cdc8-0ead-4013-bbe2-4a73019e2734" containerName="pull"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651715    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5449cdc8-0ead-4013-bbe2-4a73019e2734" containerName="pull"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651723    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41972c34-2df9-40b2-a166-8b34dbb41806" containerName="pull"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651728    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="41972c34-2df9-40b2-a166-8b34dbb41806" containerName="pull"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651735    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c818b50-6a17-459e-95f7-998f0e0aab28" containerName="util"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651740    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c818b50-6a17-459e-95f7-998f0e0aab28" containerName="util"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651747    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef18a471-e3c5-4baf-ac41-299748334698" containerName="pull"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651752    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef18a471-e3c5-4baf-ac41-299748334698" containerName="pull"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651757    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5449cdc8-0ead-4013-bbe2-4a73019e2734" containerName="extract"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651763    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5449cdc8-0ead-4013-bbe2-4a73019e2734" containerName="extract"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651769    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c818b50-6a17-459e-95f7-998f0e0aab28" containerName="pull"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651773    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c818b50-6a17-459e-95f7-998f0e0aab28" containerName="pull"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651785    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5449cdc8-0ead-4013-bbe2-4a73019e2734" containerName="util"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651790    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5449cdc8-0ead-4013-bbe2-4a73019e2734" containerName="util"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651796    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41972c34-2df9-40b2-a166-8b34dbb41806" containerName="extract"
Apr 17 08:00:33.651822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651801    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="41972c34-2df9-40b2-a166-8b34dbb41806" containerName="extract"
Apr 17 08:00:33.652604 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651843    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="1c818b50-6a17-459e-95f7-998f0e0aab28" containerName="extract"
Apr 17 08:00:33.652604 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651852    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="5449cdc8-0ead-4013-bbe2-4a73019e2734" containerName="extract"
Apr 17 08:00:33.652604 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651860    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="ef18a471-e3c5-4baf-ac41-299748334698" containerName="extract"
Apr 17 08:00:33.652604 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.651867    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="41972c34-2df9-40b2-a166-8b34dbb41806" containerName="extract"
Apr 17 08:00:33.654566 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.654551    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7587b89b76-4bcjv"
Apr 17 08:00:33.657113 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.657083    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Apr 17 08:00:33.657113 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.657103    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Apr 17 08:00:33.657113 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.657104    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-p5phq\""
Apr 17 08:00:33.665576 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.665555    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-4bcjv"]
Apr 17 08:00:33.708554 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.708521    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhmk8\" (UniqueName: \"kubernetes.io/projected/6e531445-d515-4ac6-8d17-bd4ea771e009-kube-api-access-jhmk8\") pod \"authorino-operator-7587b89b76-4bcjv\" (UID: \"6e531445-d515-4ac6-8d17-bd4ea771e009\") " pod="kuadrant-system/authorino-operator-7587b89b76-4bcjv"
Apr 17 08:00:33.809062 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.809026    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jhmk8\" (UniqueName: \"kubernetes.io/projected/6e531445-d515-4ac6-8d17-bd4ea771e009-kube-api-access-jhmk8\") pod \"authorino-operator-7587b89b76-4bcjv\" (UID: \"6e531445-d515-4ac6-8d17-bd4ea771e009\") " pod="kuadrant-system/authorino-operator-7587b89b76-4bcjv"
Apr 17 08:00:33.821117 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.821084    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhmk8\" (UniqueName: \"kubernetes.io/projected/6e531445-d515-4ac6-8d17-bd4ea771e009-kube-api-access-jhmk8\") pod \"authorino-operator-7587b89b76-4bcjv\" (UID: \"6e531445-d515-4ac6-8d17-bd4ea771e009\") " pod="kuadrant-system/authorino-operator-7587b89b76-4bcjv"
Apr 17 08:00:33.964610 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:33.964516    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7587b89b76-4bcjv"
Apr 17 08:00:34.091059 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:34.091034    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-4bcjv"]
Apr 17 08:00:34.093488 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:00:34.093459    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e531445_d515_4ac6_8d17_bd4ea771e009.slice/crio-0ae11d2fdbe738e360bdfed54e6ad8417d8a240758868cf331e182c34fed2265 WatchSource:0}: Error finding container 0ae11d2fdbe738e360bdfed54e6ad8417d8a240758868cf331e182c34fed2265: Status 404 returned error can't find the container with id 0ae11d2fdbe738e360bdfed54e6ad8417d8a240758868cf331e182c34fed2265
Apr 17 08:00:34.213579 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:34.213536    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7587b89b76-4bcjv" event={"ID":"6e531445-d515-4ac6-8d17-bd4ea771e009","Type":"ContainerStarted","Data":"0ae11d2fdbe738e360bdfed54e6ad8417d8a240758868cf331e182c34fed2265"}
Apr 17 08:00:37.226201 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:37.226142    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7587b89b76-4bcjv" event={"ID":"6e531445-d515-4ac6-8d17-bd4ea771e009","Type":"ContainerStarted","Data":"c5d2fb4498871a04e5e4ef4e018e9d41208c6b4df5538c8afc7c1f48a6379567"}
Apr 17 08:00:37.226601 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:37.226232    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7587b89b76-4bcjv"
Apr 17 08:00:37.242554 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:37.242513    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7587b89b76-4bcjv" podStartSLOduration=1.9462325040000001 podStartE2EDuration="4.242500719s" podCreationTimestamp="2026-04-17 08:00:33 +0000 UTC" firstStartedPulling="2026-04-17 08:00:34.095714042 +0000 UTC m=+505.149317236" lastFinishedPulling="2026-04-17 08:00:36.391982259 +0000 UTC m=+507.445585451" observedRunningTime="2026-04-17 08:00:37.242250711 +0000 UTC m=+508.295853920" watchObservedRunningTime="2026-04-17 08:00:37.242500719 +0000 UTC m=+508.296103929"
Apr 17 08:00:48.231248 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:00:48.231216    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7587b89b76-4bcjv"
Apr 17 08:01:16.866770 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:16.866732    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-674b59b84c-gn9tk"]
Apr 17 08:01:16.874997 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:16.874962    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-gn9tk"
Apr 17 08:01:16.877233 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:16.877208    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-25v6g\""
Apr 17 08:01:16.877375 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:16.877309    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674b59b84c-gn9tk"]
Apr 17 08:01:17.005050 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.005022    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79cbc94b89-mljmg"]
Apr 17 08:01:17.009050 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.009034    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-mljmg"
Apr 17 08:01:17.013888 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.013862    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-mljmg"]
Apr 17 08:01:17.036700 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.036677    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x889\" (UniqueName: \"kubernetes.io/projected/b07e0ef7-b859-462e-a70d-fd4f82cc8bb5-kube-api-access-4x889\") pod \"authorino-674b59b84c-gn9tk\" (UID: \"b07e0ef7-b859-462e-a70d-fd4f82cc8bb5\") " pod="kuadrant-system/authorino-674b59b84c-gn9tk"
Apr 17 08:01:17.137508 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.137413    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-598vm\" (UniqueName: \"kubernetes.io/projected/e00b7d23-8c55-4d7b-9fc5-d62e7f60be35-kube-api-access-598vm\") pod \"authorino-79cbc94b89-mljmg\" (UID: \"e00b7d23-8c55-4d7b-9fc5-d62e7f60be35\") " pod="kuadrant-system/authorino-79cbc94b89-mljmg"
Apr 17 08:01:17.137639 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.137526    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4x889\" (UniqueName: \"kubernetes.io/projected/b07e0ef7-b859-462e-a70d-fd4f82cc8bb5-kube-api-access-4x889\") pod \"authorino-674b59b84c-gn9tk\" (UID: \"b07e0ef7-b859-462e-a70d-fd4f82cc8bb5\") " pod="kuadrant-system/authorino-674b59b84c-gn9tk"
Apr 17 08:01:17.146447 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.146409    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x889\" (UniqueName: \"kubernetes.io/projected/b07e0ef7-b859-462e-a70d-fd4f82cc8bb5-kube-api-access-4x889\") pod \"authorino-674b59b84c-gn9tk\" (UID: \"b07e0ef7-b859-462e-a70d-fd4f82cc8bb5\") " pod="kuadrant-system/authorino-674b59b84c-gn9tk"
Apr 17 08:01:17.186304 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.186280    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-gn9tk"
Apr 17 08:01:17.238365 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.238331    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-598vm\" (UniqueName: \"kubernetes.io/projected/e00b7d23-8c55-4d7b-9fc5-d62e7f60be35-kube-api-access-598vm\") pod \"authorino-79cbc94b89-mljmg\" (UID: \"e00b7d23-8c55-4d7b-9fc5-d62e7f60be35\") " pod="kuadrant-system/authorino-79cbc94b89-mljmg"
Apr 17 08:01:17.247563 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.247538    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-598vm\" (UniqueName: \"kubernetes.io/projected/e00b7d23-8c55-4d7b-9fc5-d62e7f60be35-kube-api-access-598vm\") pod \"authorino-79cbc94b89-mljmg\" (UID: \"e00b7d23-8c55-4d7b-9fc5-d62e7f60be35\") " pod="kuadrant-system/authorino-79cbc94b89-mljmg"
Apr 17 08:01:17.302659 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.302631    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674b59b84c-gn9tk"]
Apr 17 08:01:17.305021 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:01:17.304987    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb07e0ef7_b859_462e_a70d_fd4f82cc8bb5.slice/crio-293616d628cf1ff3a32ad9b94c8dfa2b61cd6dcab3fe9b3adcbc609b29ffd925 WatchSource:0}: Error finding container 293616d628cf1ff3a32ad9b94c8dfa2b61cd6dcab3fe9b3adcbc609b29ffd925: Status 404 returned error can't find the container with id 293616d628cf1ff3a32ad9b94c8dfa2b61cd6dcab3fe9b3adcbc609b29ffd925
Apr 17 08:01:17.319173 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.319152    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-mljmg"
Apr 17 08:01:17.365770 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.365739    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-gn9tk" event={"ID":"b07e0ef7-b859-462e-a70d-fd4f82cc8bb5","Type":"ContainerStarted","Data":"293616d628cf1ff3a32ad9b94c8dfa2b61cd6dcab3fe9b3adcbc609b29ffd925"}
Apr 17 08:01:17.433916 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:17.433894    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-mljmg"]
Apr 17 08:01:17.435776 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:01:17.435748    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode00b7d23_8c55_4d7b_9fc5_d62e7f60be35.slice/crio-36a53c9c9a67ab60f5a38e2631585991bbf5e55cc34bb33d8ee9c78fd98718b4 WatchSource:0}: Error finding container 36a53c9c9a67ab60f5a38e2631585991bbf5e55cc34bb33d8ee9c78fd98718b4: Status 404 returned error can't find the container with id 36a53c9c9a67ab60f5a38e2631585991bbf5e55cc34bb33d8ee9c78fd98718b4
Apr 17 08:01:18.370492 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:18.370455    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-mljmg" event={"ID":"e00b7d23-8c55-4d7b-9fc5-d62e7f60be35","Type":"ContainerStarted","Data":"36a53c9c9a67ab60f5a38e2631585991bbf5e55cc34bb33d8ee9c78fd98718b4"}
Apr 17 08:01:20.380523 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:20.380489    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-mljmg" event={"ID":"e00b7d23-8c55-4d7b-9fc5-d62e7f60be35","Type":"ContainerStarted","Data":"31bff94f998eda6169a3b7406196d7fc00d7c9baee4f52646ec5677fd7b53409"}
Apr 17 08:01:20.381945 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:20.381914    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-gn9tk" event={"ID":"b07e0ef7-b859-462e-a70d-fd4f82cc8bb5","Type":"ContainerStarted","Data":"1726ee453857fd8e616c09fe600718b20266649e3141ecf62cb070a5c4509cb8"}
Apr 17 08:01:20.396109 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:20.396068    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-79cbc94b89-mljmg" podStartSLOduration=1.513542532 podStartE2EDuration="4.39605496s" podCreationTimestamp="2026-04-17 08:01:16 +0000 UTC" firstStartedPulling="2026-04-17 08:01:17.437099643 +0000 UTC m=+548.490702832" lastFinishedPulling="2026-04-17 08:01:20.319612072 +0000 UTC m=+551.373215260" observedRunningTime="2026-04-17 08:01:20.39364505 +0000 UTC m=+551.447248261" watchObservedRunningTime="2026-04-17 08:01:20.39605496 +0000 UTC m=+551.449658170"
Apr 17 08:01:20.410418 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:20.410372    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-674b59b84c-gn9tk" podStartSLOduration=1.40708656 podStartE2EDuration="4.410356547s" podCreationTimestamp="2026-04-17 08:01:16 +0000 UTC" firstStartedPulling="2026-04-17 08:01:17.306412931 +0000 UTC m=+548.360016123" lastFinishedPulling="2026-04-17 08:01:20.309682917 +0000 UTC m=+551.363286110" observedRunningTime="2026-04-17 08:01:20.406674018 +0000 UTC m=+551.460277252" watchObservedRunningTime="2026-04-17 08:01:20.410356547 +0000 UTC m=+551.463959757"
Apr 17 08:01:20.434518 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:20.434359    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-674b59b84c-gn9tk"]
Apr 17 08:01:22.388610 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:22.388573    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-674b59b84c-gn9tk" podUID="b07e0ef7-b859-462e-a70d-fd4f82cc8bb5" containerName="authorino" containerID="cri-o://1726ee453857fd8e616c09fe600718b20266649e3141ecf62cb070a5c4509cb8" gracePeriod=30
Apr 17 08:01:22.613743 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:22.613721    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-gn9tk"
Apr 17 08:01:22.780556 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:22.780525    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4x889\" (UniqueName: \"kubernetes.io/projected/b07e0ef7-b859-462e-a70d-fd4f82cc8bb5-kube-api-access-4x889\") pod \"b07e0ef7-b859-462e-a70d-fd4f82cc8bb5\" (UID: \"b07e0ef7-b859-462e-a70d-fd4f82cc8bb5\") "
Apr 17 08:01:22.782677 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:22.782654    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b07e0ef7-b859-462e-a70d-fd4f82cc8bb5-kube-api-access-4x889" (OuterVolumeSpecName: "kube-api-access-4x889") pod "b07e0ef7-b859-462e-a70d-fd4f82cc8bb5" (UID: "b07e0ef7-b859-462e-a70d-fd4f82cc8bb5"). InnerVolumeSpecName "kube-api-access-4x889". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:01:22.881647 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:22.881613    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4x889\" (UniqueName: \"kubernetes.io/projected/b07e0ef7-b859-462e-a70d-fd4f82cc8bb5-kube-api-access-4x889\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:01:23.392824 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:23.392790    2571 generic.go:358] "Generic (PLEG): container finished" podID="b07e0ef7-b859-462e-a70d-fd4f82cc8bb5" containerID="1726ee453857fd8e616c09fe600718b20266649e3141ecf62cb070a5c4509cb8" exitCode=0
Apr 17 08:01:23.393312 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:23.392843    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-gn9tk"
Apr 17 08:01:23.393312 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:23.392867    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-gn9tk" event={"ID":"b07e0ef7-b859-462e-a70d-fd4f82cc8bb5","Type":"ContainerDied","Data":"1726ee453857fd8e616c09fe600718b20266649e3141ecf62cb070a5c4509cb8"}
Apr 17 08:01:23.393312 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:23.392909    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-gn9tk" event={"ID":"b07e0ef7-b859-462e-a70d-fd4f82cc8bb5","Type":"ContainerDied","Data":"293616d628cf1ff3a32ad9b94c8dfa2b61cd6dcab3fe9b3adcbc609b29ffd925"}
Apr 17 08:01:23.393312 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:23.392924    2571 scope.go:117] "RemoveContainer" containerID="1726ee453857fd8e616c09fe600718b20266649e3141ecf62cb070a5c4509cb8"
Apr 17 08:01:23.401156 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:23.401137    2571 scope.go:117] "RemoveContainer" containerID="1726ee453857fd8e616c09fe600718b20266649e3141ecf62cb070a5c4509cb8"
Apr 17 08:01:23.401394 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:01:23.401374    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1726ee453857fd8e616c09fe600718b20266649e3141ecf62cb070a5c4509cb8\": container with ID starting with 1726ee453857fd8e616c09fe600718b20266649e3141ecf62cb070a5c4509cb8 not found: ID does not exist" containerID="1726ee453857fd8e616c09fe600718b20266649e3141ecf62cb070a5c4509cb8"
Apr 17 08:01:23.401458 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:23.401402    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1726ee453857fd8e616c09fe600718b20266649e3141ecf62cb070a5c4509cb8"} err="failed to get container status \"1726ee453857fd8e616c09fe600718b20266649e3141ecf62cb070a5c4509cb8\": rpc error: code = NotFound desc = could not find container \"1726ee453857fd8e616c09fe600718b20266649e3141ecf62cb070a5c4509cb8\": container with ID starting with 1726ee453857fd8e616c09fe600718b20266649e3141ecf62cb070a5c4509cb8 not found: ID does not exist"
Apr 17 08:01:23.418754 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:23.418735    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-674b59b84c-gn9tk"]
Apr 17 08:01:23.425098 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:23.425078    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-674b59b84c-gn9tk"]
Apr 17 08:01:23.543167 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:23.543138    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b07e0ef7-b859-462e-a70d-fd4f82cc8bb5" path="/var/lib/kubelet/pods/b07e0ef7-b859-462e-a70d-fd4f82cc8bb5/volumes"
Apr 17 08:01:41.441310 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.441270    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68bd676465-b87xk"]
Apr 17 08:01:41.441704 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.441595    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b07e0ef7-b859-462e-a70d-fd4f82cc8bb5" containerName="authorino"
Apr 17 08:01:41.441704 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.441607    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b07e0ef7-b859-462e-a70d-fd4f82cc8bb5" containerName="authorino"
Apr 17 08:01:41.441704 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.441658    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="b07e0ef7-b859-462e-a70d-fd4f82cc8bb5" containerName="authorino"
Apr 17 08:01:41.444514 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.444495    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68bd676465-b87xk"
Apr 17 08:01:41.446933 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.446915    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Apr 17 08:01:41.450185 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.450162    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68bd676465-b87xk"]
Apr 17 08:01:41.522963 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.522929    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgtpn\" (UniqueName: \"kubernetes.io/projected/38e5d3d2-6156-4e4a-a543-88a520539a96-kube-api-access-sgtpn\") pod \"authorino-68bd676465-b87xk\" (UID: \"38e5d3d2-6156-4e4a-a543-88a520539a96\") " pod="kuadrant-system/authorino-68bd676465-b87xk"
Apr 17 08:01:41.523120 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.522976    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38e5d3d2-6156-4e4a-a543-88a520539a96-tls-cert\") pod \"authorino-68bd676465-b87xk\" (UID: \"38e5d3d2-6156-4e4a-a543-88a520539a96\") " pod="kuadrant-system/authorino-68bd676465-b87xk"
Apr 17 08:01:41.624227 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.624195    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38e5d3d2-6156-4e4a-a543-88a520539a96-tls-cert\") pod \"authorino-68bd676465-b87xk\" (UID: \"38e5d3d2-6156-4e4a-a543-88a520539a96\") " pod="kuadrant-system/authorino-68bd676465-b87xk"
Apr 17 08:01:41.624393 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.624308    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sgtpn\" (UniqueName: \"kubernetes.io/projected/38e5d3d2-6156-4e4a-a543-88a520539a96-kube-api-access-sgtpn\") pod \"authorino-68bd676465-b87xk\" (UID: \"38e5d3d2-6156-4e4a-a543-88a520539a96\") " pod="kuadrant-system/authorino-68bd676465-b87xk"
Apr 17 08:01:41.627035 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.627007    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38e5d3d2-6156-4e4a-a543-88a520539a96-tls-cert\") pod \"authorino-68bd676465-b87xk\" (UID: \"38e5d3d2-6156-4e4a-a543-88a520539a96\") " pod="kuadrant-system/authorino-68bd676465-b87xk"
Apr 17 08:01:41.632039 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.632013    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgtpn\" (UniqueName: \"kubernetes.io/projected/38e5d3d2-6156-4e4a-a543-88a520539a96-kube-api-access-sgtpn\") pod \"authorino-68bd676465-b87xk\" (UID: \"38e5d3d2-6156-4e4a-a543-88a520539a96\") " pod="kuadrant-system/authorino-68bd676465-b87xk"
Apr 17 08:01:41.755261 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.755176    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68bd676465-b87xk"
Apr 17 08:01:41.869420 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:41.869377    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68bd676465-b87xk"]
Apr 17 08:01:41.871719 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:01:41.871694    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38e5d3d2_6156_4e4a_a543_88a520539a96.slice/crio-257e09c3461815473aca3176e67305ed699e620ec1960786d359c2bfbc81cd34 WatchSource:0}: Error finding container 257e09c3461815473aca3176e67305ed699e620ec1960786d359c2bfbc81cd34: Status 404 returned error can't find the container with id 257e09c3461815473aca3176e67305ed699e620ec1960786d359c2bfbc81cd34
Apr 17 08:01:42.458360 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:42.458315    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68bd676465-b87xk" event={"ID":"38e5d3d2-6156-4e4a-a543-88a520539a96","Type":"ContainerStarted","Data":"257e09c3461815473aca3176e67305ed699e620ec1960786d359c2bfbc81cd34"}
Apr 17 08:01:43.463334 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:43.463294    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68bd676465-b87xk" event={"ID":"38e5d3d2-6156-4e4a-a543-88a520539a96","Type":"ContainerStarted","Data":"8eaee780b560760891fd99c553e54c64f0165d4ae1c9bdf23aa62f8ae1c34a28"}
Apr 17 08:01:43.479395 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:43.479347    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-68bd676465-b87xk" podStartSLOduration=1.78368375 podStartE2EDuration="2.479333568s" podCreationTimestamp="2026-04-17 08:01:41 +0000 UTC" firstStartedPulling="2026-04-17 08:01:41.873410656 +0000 UTC m=+572.927013846" lastFinishedPulling="2026-04-17 08:01:42.569060475 +0000 UTC m=+573.622663664" observedRunningTime="2026-04-17 08:01:43.478725074 +0000 UTC m=+574.532328284" watchObservedRunningTime="2026-04-17 08:01:43.479333568 +0000 UTC m=+574.532936778"
Apr 17 08:01:43.500736 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:43.500702    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-mljmg"]
Apr 17 08:01:43.500982 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:43.500956    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-79cbc94b89-mljmg" podUID="e00b7d23-8c55-4d7b-9fc5-d62e7f60be35" containerName="authorino" containerID="cri-o://31bff94f998eda6169a3b7406196d7fc00d7c9baee4f52646ec5677fd7b53409" gracePeriod=30
Apr 17 08:01:43.733469 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:43.733422    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-mljmg"
Apr 17 08:01:43.840254 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:43.840218    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-598vm\" (UniqueName: \"kubernetes.io/projected/e00b7d23-8c55-4d7b-9fc5-d62e7f60be35-kube-api-access-598vm\") pod \"e00b7d23-8c55-4d7b-9fc5-d62e7f60be35\" (UID: \"e00b7d23-8c55-4d7b-9fc5-d62e7f60be35\") "
Apr 17 08:01:43.842331 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:43.842304    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e00b7d23-8c55-4d7b-9fc5-d62e7f60be35-kube-api-access-598vm" (OuterVolumeSpecName: "kube-api-access-598vm") pod "e00b7d23-8c55-4d7b-9fc5-d62e7f60be35" (UID: "e00b7d23-8c55-4d7b-9fc5-d62e7f60be35"). InnerVolumeSpecName "kube-api-access-598vm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:01:43.941625 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:43.941588    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-598vm\" (UniqueName: \"kubernetes.io/projected/e00b7d23-8c55-4d7b-9fc5-d62e7f60be35-kube-api-access-598vm\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:01:44.467056 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:44.467023    2571 generic.go:358] "Generic (PLEG): container finished" podID="e00b7d23-8c55-4d7b-9fc5-d62e7f60be35" containerID="31bff94f998eda6169a3b7406196d7fc00d7c9baee4f52646ec5677fd7b53409" exitCode=0
Apr 17 08:01:44.467534 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:44.467068    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-mljmg"
Apr 17 08:01:44.467534 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:44.467115    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-mljmg" event={"ID":"e00b7d23-8c55-4d7b-9fc5-d62e7f60be35","Type":"ContainerDied","Data":"31bff94f998eda6169a3b7406196d7fc00d7c9baee4f52646ec5677fd7b53409"}
Apr 17 08:01:44.467534 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:44.467155    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-mljmg" event={"ID":"e00b7d23-8c55-4d7b-9fc5-d62e7f60be35","Type":"ContainerDied","Data":"36a53c9c9a67ab60f5a38e2631585991bbf5e55cc34bb33d8ee9c78fd98718b4"}
Apr 17 08:01:44.467534 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:44.467176    2571 scope.go:117] "RemoveContainer" containerID="31bff94f998eda6169a3b7406196d7fc00d7c9baee4f52646ec5677fd7b53409"
Apr 17 08:01:44.475335 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:44.475319    2571 scope.go:117] "RemoveContainer" containerID="31bff94f998eda6169a3b7406196d7fc00d7c9baee4f52646ec5677fd7b53409"
Apr 17 08:01:44.475587 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:01:44.475567    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31bff94f998eda6169a3b7406196d7fc00d7c9baee4f52646ec5677fd7b53409\": container with ID starting with 31bff94f998eda6169a3b7406196d7fc00d7c9baee4f52646ec5677fd7b53409 not found: ID does not exist" containerID="31bff94f998eda6169a3b7406196d7fc00d7c9baee4f52646ec5677fd7b53409"
Apr 17 08:01:44.475639 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:44.475595    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31bff94f998eda6169a3b7406196d7fc00d7c9baee4f52646ec5677fd7b53409"} err="failed to get container status \"31bff94f998eda6169a3b7406196d7fc00d7c9baee4f52646ec5677fd7b53409\": rpc error: code = NotFound desc = could not find container \"31bff94f998eda6169a3b7406196d7fc00d7c9baee4f52646ec5677fd7b53409\": container with ID starting with 31bff94f998eda6169a3b7406196d7fc00d7c9baee4f52646ec5677fd7b53409 not found: ID does not exist"
Apr 17 08:01:44.491701 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:44.491651    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-mljmg"]
Apr 17 08:01:44.494085 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:44.494066    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-mljmg"]
Apr 17 08:01:45.542498 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:01:45.542464    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e00b7d23-8c55-4d7b-9fc5-d62e7f60be35" path="/var/lib/kubelet/pods/e00b7d23-8c55-4d7b-9fc5-d62e7f60be35/volumes"
Apr 17 08:02:09.455896 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:02:09.455871    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 08:02:09.459221 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:02:09.459200    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 08:03:09.889404 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.889370    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-86f7b4b499-6rv84"]
Apr 17 08:03:09.889839 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.889671    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e00b7d23-8c55-4d7b-9fc5-d62e7f60be35" containerName="authorino"
Apr 17 08:03:09.889839 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.889685    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e00b7d23-8c55-4d7b-9fc5-d62e7f60be35" containerName="authorino"
Apr 17 08:03:09.889839 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.889743    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e00b7d23-8c55-4d7b-9fc5-d62e7f60be35" containerName="authorino"
Apr 17 08:03:09.892506 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.892487    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-86f7b4b499-6rv84"
Apr 17 08:03:09.895070 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.895052    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Apr 17 08:03:09.895178 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.895108    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Apr 17 08:03:09.895227 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.895174    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Apr 17 08:03:09.896174 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.896150    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-jnnlq\""
Apr 17 08:03:09.901759 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.901734    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-86f7b4b499-6rv84"]
Apr 17 08:03:09.905389 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.905369    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-696fc77849-xc7vc"]
Apr 17 08:03:09.908300 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.908283    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-xc7vc"
Apr 17 08:03:09.910828 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.910803    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Apr 17 08:03:09.911034 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.911014    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-c8ldm\""
Apr 17 08:03:09.923938 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.923906    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-xc7vc"]
Apr 17 08:03:09.984995 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.984957    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a22c9ec0-07ac-4327-934a-12d476ca5771-cert\") pod \"odh-model-controller-696fc77849-xc7vc\" (UID: \"a22c9ec0-07ac-4327-934a-12d476ca5771\") " pod="kserve/odh-model-controller-696fc77849-xc7vc"
Apr 17 08:03:09.984995 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.984994    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tgfd\" (UniqueName: \"kubernetes.io/projected/a22c9ec0-07ac-4327-934a-12d476ca5771-kube-api-access-5tgfd\") pod \"odh-model-controller-696fc77849-xc7vc\" (UID: \"a22c9ec0-07ac-4327-934a-12d476ca5771\") " pod="kserve/odh-model-controller-696fc77849-xc7vc"
Apr 17 08:03:09.985203 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.985073    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4355290c-c266-4053-9da1-71ad9182681a-tls-certs\") pod \"model-serving-api-86f7b4b499-6rv84\" (UID: \"4355290c-c266-4053-9da1-71ad9182681a\") " pod="kserve/model-serving-api-86f7b4b499-6rv84"
Apr 17 08:03:09.985203 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:09.985114    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkc45\" (UniqueName: \"kubernetes.io/projected/4355290c-c266-4053-9da1-71ad9182681a-kube-api-access-tkc45\") pod \"model-serving-api-86f7b4b499-6rv84\" (UID: \"4355290c-c266-4053-9da1-71ad9182681a\") " pod="kserve/model-serving-api-86f7b4b499-6rv84"
Apr 17 08:03:10.086204 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.086170    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a22c9ec0-07ac-4327-934a-12d476ca5771-cert\") pod \"odh-model-controller-696fc77849-xc7vc\" (UID: \"a22c9ec0-07ac-4327-934a-12d476ca5771\") " pod="kserve/odh-model-controller-696fc77849-xc7vc"
Apr 17 08:03:10.086204 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.086209    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5tgfd\" (UniqueName: \"kubernetes.io/projected/a22c9ec0-07ac-4327-934a-12d476ca5771-kube-api-access-5tgfd\") pod \"odh-model-controller-696fc77849-xc7vc\" (UID: \"a22c9ec0-07ac-4327-934a-12d476ca5771\") " pod="kserve/odh-model-controller-696fc77849-xc7vc"
Apr 17 08:03:10.086463 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.086268    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4355290c-c266-4053-9da1-71ad9182681a-tls-certs\") pod \"model-serving-api-86f7b4b499-6rv84\" (UID: \"4355290c-c266-4053-9da1-71ad9182681a\") " pod="kserve/model-serving-api-86f7b4b499-6rv84"
Apr 17 08:03:10.086463 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.086295    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tkc45\" (UniqueName: \"kubernetes.io/projected/4355290c-c266-4053-9da1-71ad9182681a-kube-api-access-tkc45\") pod \"model-serving-api-86f7b4b499-6rv84\" (UID: \"4355290c-c266-4053-9da1-71ad9182681a\") " pod="kserve/model-serving-api-86f7b4b499-6rv84"
Apr 17 08:03:10.086463 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:03:10.086350    2571 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Apr 17 08:03:10.086463 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:03:10.086388    2571 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found
Apr 17 08:03:10.086463 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:03:10.086457    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a22c9ec0-07ac-4327-934a-12d476ca5771-cert podName:a22c9ec0-07ac-4327-934a-12d476ca5771 nodeName:}" failed. No retries permitted until 2026-04-17 08:03:10.586410672 +0000 UTC m=+661.640013860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a22c9ec0-07ac-4327-934a-12d476ca5771-cert") pod "odh-model-controller-696fc77849-xc7vc" (UID: "a22c9ec0-07ac-4327-934a-12d476ca5771") : secret "odh-model-controller-webhook-cert" not found
Apr 17 08:03:10.086650 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:03:10.086478    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4355290c-c266-4053-9da1-71ad9182681a-tls-certs podName:4355290c-c266-4053-9da1-71ad9182681a nodeName:}" failed. No retries permitted until 2026-04-17 08:03:10.586468446 +0000 UTC m=+661.640071635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/4355290c-c266-4053-9da1-71ad9182681a-tls-certs") pod "model-serving-api-86f7b4b499-6rv84" (UID: "4355290c-c266-4053-9da1-71ad9182681a") : secret "model-serving-api-tls" not found
Apr 17 08:03:10.097778 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.097740    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tgfd\" (UniqueName: \"kubernetes.io/projected/a22c9ec0-07ac-4327-934a-12d476ca5771-kube-api-access-5tgfd\") pod \"odh-model-controller-696fc77849-xc7vc\" (UID: \"a22c9ec0-07ac-4327-934a-12d476ca5771\") " pod="kserve/odh-model-controller-696fc77849-xc7vc"
Apr 17 08:03:10.097911 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.097798    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkc45\" (UniqueName: \"kubernetes.io/projected/4355290c-c266-4053-9da1-71ad9182681a-kube-api-access-tkc45\") pod \"model-serving-api-86f7b4b499-6rv84\" (UID: \"4355290c-c266-4053-9da1-71ad9182681a\") " pod="kserve/model-serving-api-86f7b4b499-6rv84"
Apr 17 08:03:10.591202 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.591148    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a22c9ec0-07ac-4327-934a-12d476ca5771-cert\") pod \"odh-model-controller-696fc77849-xc7vc\" (UID: \"a22c9ec0-07ac-4327-934a-12d476ca5771\") " pod="kserve/odh-model-controller-696fc77849-xc7vc"
Apr 17 08:03:10.591465 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.591234    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4355290c-c266-4053-9da1-71ad9182681a-tls-certs\") pod \"model-serving-api-86f7b4b499-6rv84\" (UID: \"4355290c-c266-4053-9da1-71ad9182681a\") " pod="kserve/model-serving-api-86f7b4b499-6rv84"
Apr 17 08:03:10.593757 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.593730    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a22c9ec0-07ac-4327-934a-12d476ca5771-cert\") pod \"odh-model-controller-696fc77849-xc7vc\" (UID: \"a22c9ec0-07ac-4327-934a-12d476ca5771\") " pod="kserve/odh-model-controller-696fc77849-xc7vc"
Apr 17 08:03:10.593853 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.593778    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4355290c-c266-4053-9da1-71ad9182681a-tls-certs\") pod \"model-serving-api-86f7b4b499-6rv84\" (UID: \"4355290c-c266-4053-9da1-71ad9182681a\") " pod="kserve/model-serving-api-86f7b4b499-6rv84"
Apr 17 08:03:10.803424 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.803380    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-86f7b4b499-6rv84"
Apr 17 08:03:10.823613 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.823577    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-xc7vc"
Apr 17 08:03:10.936986 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.936960    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-86f7b4b499-6rv84"]
Apr 17 08:03:10.939422 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:03:10.939388    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4355290c_c266_4053_9da1_71ad9182681a.slice/crio-f5f36ad1dfad8a6c0ccbc9c2eda918be4a54b12373a789c674a6817038db76d4 WatchSource:0}: Error finding container f5f36ad1dfad8a6c0ccbc9c2eda918be4a54b12373a789c674a6817038db76d4: Status 404 returned error can't find the container with id f5f36ad1dfad8a6c0ccbc9c2eda918be4a54b12373a789c674a6817038db76d4
Apr 17 08:03:10.941189 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.941172    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 17 08:03:10.960177 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:10.960154    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-xc7vc"]
Apr 17 08:03:10.961991 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:03:10.961964    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda22c9ec0_07ac_4327_934a_12d476ca5771.slice/crio-95ce740d73a4b20ee07dd8ca14e2a92ee1b092e3115bf4dae2d9d0319315ba1c WatchSource:0}: Error finding container 95ce740d73a4b20ee07dd8ca14e2a92ee1b092e3115bf4dae2d9d0319315ba1c: Status 404 returned error can't find the container with id 95ce740d73a4b20ee07dd8ca14e2a92ee1b092e3115bf4dae2d9d0319315ba1c
Apr 17 08:03:11.775253 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:11.775186    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-86f7b4b499-6rv84" event={"ID":"4355290c-c266-4053-9da1-71ad9182681a","Type":"ContainerStarted","Data":"f5f36ad1dfad8a6c0ccbc9c2eda918be4a54b12373a789c674a6817038db76d4"}
Apr 17 08:03:11.776940 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:11.776911    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-xc7vc" event={"ID":"a22c9ec0-07ac-4327-934a-12d476ca5771","Type":"ContainerStarted","Data":"95ce740d73a4b20ee07dd8ca14e2a92ee1b092e3115bf4dae2d9d0319315ba1c"}
Apr 17 08:03:14.789580 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:14.789545    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-xc7vc" event={"ID":"a22c9ec0-07ac-4327-934a-12d476ca5771","Type":"ContainerStarted","Data":"52d61994ad351c08b8f017edb0dfe67f6e1b269ce4a371ee3a53fcd6e4ba1e09"}
Apr 17 08:03:14.789892 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:14.789655    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-696fc77849-xc7vc"
Apr 17 08:03:14.804143 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:14.804094    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/odh-model-controller-696fc77849-xc7vc" podStartSLOduration=2.083085221 podStartE2EDuration="5.804078728s" podCreationTimestamp="2026-04-17 08:03:09 +0000 UTC" firstStartedPulling="2026-04-17 08:03:10.963156368 +0000 UTC m=+662.016759557" lastFinishedPulling="2026-04-17 08:03:14.684149875 +0000 UTC m=+665.737753064" observedRunningTime="2026-04-17 08:03:14.803653964 +0000 UTC m=+665.857257200" watchObservedRunningTime="2026-04-17 08:03:14.804078728 +0000 UTC m=+665.857681941"
Apr 17 08:03:15.793925 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:15.793891    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-86f7b4b499-6rv84" event={"ID":"4355290c-c266-4053-9da1-71ad9182681a","Type":"ContainerStarted","Data":"9784a24ac6c5e6db23f6f90dd384ce86adb3ffa21a2f3a3e0888223ffff6edde"}
Apr 17 08:03:15.794379 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:15.793974    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-86f7b4b499-6rv84"
Apr 17 08:03:15.812724 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:15.812671    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/model-serving-api-86f7b4b499-6rv84" podStartSLOduration=3.015996497 podStartE2EDuration="6.812656562s" podCreationTimestamp="2026-04-17 08:03:09 +0000 UTC" firstStartedPulling="2026-04-17 08:03:10.941309529 +0000 UTC m=+661.994912722" lastFinishedPulling="2026-04-17 08:03:14.737969598 +0000 UTC m=+665.791572787" observedRunningTime="2026-04-17 08:03:15.810190506 +0000 UTC m=+666.863793717" watchObservedRunningTime="2026-04-17 08:03:15.812656562 +0000 UTC m=+666.866259772"
Apr 17 08:03:25.796641 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:25.796614    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-696fc77849-xc7vc"
Apr 17 08:03:26.601226 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:26.601196    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/s3-init-gx66p"]
Apr 17 08:03:26.604610 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:26.604593    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-gx66p"
Apr 17 08:03:26.607036 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:26.607016    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Apr 17 08:03:26.607131 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:26.607074    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-zgnsg\""
Apr 17 08:03:26.610664 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:26.610642    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-gx66p"]
Apr 17 08:03:26.724615 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:26.724574    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmllq\" (UniqueName: \"kubernetes.io/projected/639bb21e-fc99-4584-9644-b218441f2001-kube-api-access-nmllq\") pod \"s3-init-gx66p\" (UID: \"639bb21e-fc99-4584-9644-b218441f2001\") " pod="kserve/s3-init-gx66p"
Apr 17 08:03:26.802192 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:26.802164    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-86f7b4b499-6rv84"
Apr 17 08:03:26.825599 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:26.825572    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nmllq\" (UniqueName: \"kubernetes.io/projected/639bb21e-fc99-4584-9644-b218441f2001-kube-api-access-nmllq\") pod \"s3-init-gx66p\" (UID: \"639bb21e-fc99-4584-9644-b218441f2001\") " pod="kserve/s3-init-gx66p"
Apr 17 08:03:26.833619 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:26.833588    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmllq\" (UniqueName: \"kubernetes.io/projected/639bb21e-fc99-4584-9644-b218441f2001-kube-api-access-nmllq\") pod \"s3-init-gx66p\" (UID: \"639bb21e-fc99-4584-9644-b218441f2001\") " pod="kserve/s3-init-gx66p"
Apr 17 08:03:26.914991 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:26.914910    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-gx66p"
Apr 17 08:03:27.031071 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:27.031044    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-gx66p"]
Apr 17 08:03:27.033732 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:03:27.033701    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod639bb21e_fc99_4584_9644_b218441f2001.slice/crio-4ef82590e42fce090c961cdf878ecf1c23ef53096547699724149403e36298d9 WatchSource:0}: Error finding container 4ef82590e42fce090c961cdf878ecf1c23ef53096547699724149403e36298d9: Status 404 returned error can't find the container with id 4ef82590e42fce090c961cdf878ecf1c23ef53096547699724149403e36298d9
Apr 17 08:03:27.843466 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:27.843399    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-gx66p" event={"ID":"639bb21e-fc99-4584-9644-b218441f2001","Type":"ContainerStarted","Data":"4ef82590e42fce090c961cdf878ecf1c23ef53096547699724149403e36298d9"}
Apr 17 08:03:31.860751 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:31.860715    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-gx66p" event={"ID":"639bb21e-fc99-4584-9644-b218441f2001","Type":"ContainerStarted","Data":"666d5d23afa575392fdc83bb49829d9a9c6fad49b9e4e172243b13f5febcc19b"}
Apr 17 08:03:31.875727 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:31.875680    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/s3-init-gx66p" podStartSLOduration=1.253223971 podStartE2EDuration="5.875665393s" podCreationTimestamp="2026-04-17 08:03:26 +0000 UTC" firstStartedPulling="2026-04-17 08:03:27.035486497 +0000 UTC m=+678.089089688" lastFinishedPulling="2026-04-17 08:03:31.657927922 +0000 UTC m=+682.711531110" observedRunningTime="2026-04-17 08:03:31.873754695 +0000 UTC m=+682.927357905" watchObservedRunningTime="2026-04-17 08:03:31.875665393 +0000 UTC m=+682.929268604"
Apr 17 08:03:34.872616 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:34.872581    2571 generic.go:358] "Generic (PLEG): container finished" podID="639bb21e-fc99-4584-9644-b218441f2001" containerID="666d5d23afa575392fdc83bb49829d9a9c6fad49b9e4e172243b13f5febcc19b" exitCode=0
Apr 17 08:03:34.872978 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:34.872640    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-gx66p" event={"ID":"639bb21e-fc99-4584-9644-b218441f2001","Type":"ContainerDied","Data":"666d5d23afa575392fdc83bb49829d9a9c6fad49b9e4e172243b13f5febcc19b"}
Apr 17 08:03:35.992907 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:35.992876    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-gx66p"
Apr 17 08:03:36.108561 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:36.108529    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmllq\" (UniqueName: \"kubernetes.io/projected/639bb21e-fc99-4584-9644-b218441f2001-kube-api-access-nmllq\") pod \"639bb21e-fc99-4584-9644-b218441f2001\" (UID: \"639bb21e-fc99-4584-9644-b218441f2001\") "
Apr 17 08:03:36.110557 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:36.110533    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/639bb21e-fc99-4584-9644-b218441f2001-kube-api-access-nmllq" (OuterVolumeSpecName: "kube-api-access-nmllq") pod "639bb21e-fc99-4584-9644-b218441f2001" (UID: "639bb21e-fc99-4584-9644-b218441f2001"). InnerVolumeSpecName "kube-api-access-nmllq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:03:36.209789 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:36.209710    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmllq\" (UniqueName: \"kubernetes.io/projected/639bb21e-fc99-4584-9644-b218441f2001-kube-api-access-nmllq\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:03:36.880446 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:36.880399    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-gx66p"
Apr 17 08:03:36.880622 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:36.880394    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-gx66p" event={"ID":"639bb21e-fc99-4584-9644-b218441f2001","Type":"ContainerDied","Data":"4ef82590e42fce090c961cdf878ecf1c23ef53096547699724149403e36298d9"}
Apr 17 08:03:36.880622 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:03:36.880495    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ef82590e42fce090c961cdf878ecf1c23ef53096547699724149403e36298d9"
Apr 17 08:04:14.894980 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:14.894949    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"]
Apr 17 08:04:14.895590 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:14.895323    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="639bb21e-fc99-4584-9644-b218441f2001" containerName="s3-init"
Apr 17 08:04:14.895590 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:14.895336    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="639bb21e-fc99-4584-9644-b218441f2001" containerName="s3-init"
Apr 17 08:04:14.895590 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:14.895407    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="639bb21e-fc99-4584-9644-b218441f2001" containerName="s3-init"
Apr 17 08:04:14.915200 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:14.915159    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"]
Apr 17 08:04:14.915329 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:14.915304    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:14.918937 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:14.918910    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 17 08:04:14.918937 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:14.918923    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jj9s6\""
Apr 17 08:04:14.919136 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:14.918923    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Apr 17 08:04:14.919273 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:14.919255    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 17 08:04:15.009054 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.009011    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.009244 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.009060    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-home\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.009244 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.009117    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-model-cache\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.009244 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.009153    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.009244 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.009215    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-dshm\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.009384 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.009267    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbcfb\" (UniqueName: \"kubernetes.io/projected/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-kube-api-access-rbcfb\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.110263 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.110232    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-dshm\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.110457 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.110272    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rbcfb\" (UniqueName: \"kubernetes.io/projected/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-kube-api-access-rbcfb\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.110457 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.110311    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.110457 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.110328    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-home\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.110457 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.110359    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-model-cache\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.110457 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.110376    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.110858 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.110829    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.110858 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.110847    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-home\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.111009 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.110858    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-model-cache\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.112575 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.112554    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-dshm\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.112988 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.112967    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.118873 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.118852    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbcfb\" (UniqueName: \"kubernetes.io/projected/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-kube-api-access-rbcfb\") pod \"scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.226422 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.226320    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:15.345473 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:15.345447    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"]
Apr 17 08:04:15.347270 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:04:15.347245    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod585bc1e7_fbc0_4ddc_841d_b94dd1a0f6de.slice/crio-4561c5583ee2f974de521b4cf97fd598a4b1e6a57c7ebab3dbd6c46976afe6ad WatchSource:0}: Error finding container 4561c5583ee2f974de521b4cf97fd598a4b1e6a57c7ebab3dbd6c46976afe6ad: Status 404 returned error can't find the container with id 4561c5583ee2f974de521b4cf97fd598a4b1e6a57c7ebab3dbd6c46976afe6ad
Apr 17 08:04:16.021757 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:16.021720    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9" event={"ID":"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de","Type":"ContainerStarted","Data":"4561c5583ee2f974de521b4cf97fd598a4b1e6a57c7ebab3dbd6c46976afe6ad"}
Apr 17 08:04:20.038586 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:20.038510    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9" event={"ID":"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de","Type":"ContainerStarted","Data":"69519b53bd0d6b7c549138e853bd8a06f9f2c608c594245dc82b7243bc83e6ae"}
Apr 17 08:04:23.051102 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:23.051075    2571 generic.go:358] "Generic (PLEG): container finished" podID="585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" containerID="69519b53bd0d6b7c549138e853bd8a06f9f2c608c594245dc82b7243bc83e6ae" exitCode=0
Apr 17 08:04:23.051404 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:23.051146    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9" event={"ID":"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de","Type":"ContainerDied","Data":"69519b53bd0d6b7c549138e853bd8a06f9f2c608c594245dc82b7243bc83e6ae"}
Apr 17 08:04:25.060045 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:25.060001    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9" event={"ID":"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de","Type":"ContainerStarted","Data":"b19025b8ef8a7dc540ffb9c642d31dd233307fa39c608c34de19bd9108d2a2bf"}
Apr 17 08:04:25.079628 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:25.079581    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9" podStartSLOduration=2.226994576 podStartE2EDuration="11.079566947s" podCreationTimestamp="2026-04-17 08:04:14 +0000 UTC" firstStartedPulling="2026-04-17 08:04:15.349144566 +0000 UTC m=+726.402747770" lastFinishedPulling="2026-04-17 08:04:24.201716949 +0000 UTC m=+735.255320141" observedRunningTime="2026-04-17 08:04:25.077554205 +0000 UTC m=+736.131157438" watchObservedRunningTime="2026-04-17 08:04:25.079566947 +0000 UTC m=+736.133170158"
Apr 17 08:04:25.226639 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:25.226591    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:25.226639 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:25.226643    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:25.239200 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:25.239179    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:04:26.074725 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:04:26.074696    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:05:13.710502 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.710471    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"]
Apr 17 08:05:13.714475 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.714456    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.718335 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.718312    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvdde380eaa9fe1facad32d45131f9e34d-kserve-self-signed-certs\""
Apr 17 08:05:13.718468 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.718451    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-sa-dockercfg-2gdbt\""
Apr 17 08:05:13.724592 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.724570    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"]
Apr 17 08:05:13.769038 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.769005    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e684c614-2482-4d10-9064-cfc4f2c83fb1-tls-certs\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.769175 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.769052    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-kserve-provision-location\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.769175 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.769077    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-cache\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.769175 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.769134    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-uds\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.769175 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.769163    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-tmp\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.769312 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.769190    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6v65\" (UniqueName: \"kubernetes.io/projected/e684c614-2482-4d10-9064-cfc4f2c83fb1-kube-api-access-p6v65\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.870280 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.870242    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-cache\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.870280 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.870285    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-uds\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.870536 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.870307    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-tmp\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.870536 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.870339    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p6v65\" (UniqueName: \"kubernetes.io/projected/e684c614-2482-4d10-9064-cfc4f2c83fb1-kube-api-access-p6v65\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.870536 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.870370    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e684c614-2482-4d10-9064-cfc4f2c83fb1-tls-certs\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.870536 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.870413    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-kserve-provision-location\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.870720 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.870674    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-uds\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.870772 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.870720    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-cache\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.870810 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.870761    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-tmp\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.870846 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.870807    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-kserve-provision-location\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.872913 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.872892    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e684c614-2482-4d10-9064-cfc4f2c83fb1-tls-certs\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:13.879342 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:13.879323    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6v65\" (UniqueName: \"kubernetes.io/projected/e684c614-2482-4d10-9064-cfc4f2c83fb1-kube-api-access-p6v65\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:14.025676 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:14.025573    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:14.152136 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:14.152113    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"]
Apr 17 08:05:14.154069 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:05:14.154039    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode684c614_2482_4d10_9064_cfc4f2c83fb1.slice/crio-9c313635f7e7f6d23ba1cd0cbdd933ef4e3d60d7eeb2efca15a2778da8c938c0 WatchSource:0}: Error finding container 9c313635f7e7f6d23ba1cd0cbdd933ef4e3d60d7eeb2efca15a2778da8c938c0: Status 404 returned error can't find the container with id 9c313635f7e7f6d23ba1cd0cbdd933ef4e3d60d7eeb2efca15a2778da8c938c0
Apr 17 08:05:14.230405 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:14.230381    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff" event={"ID":"e684c614-2482-4d10-9064-cfc4f2c83fb1","Type":"ContainerStarted","Data":"6645c78a755f0452b8ee5592cbdfc5a320109abb42ec7d060fa016aa516071d0"}
Apr 17 08:05:14.230548 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:14.230411    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff" event={"ID":"e684c614-2482-4d10-9064-cfc4f2c83fb1","Type":"ContainerStarted","Data":"9c313635f7e7f6d23ba1cd0cbdd933ef4e3d60d7eeb2efca15a2778da8c938c0"}
Apr 17 08:05:15.234870 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:15.234786    2571 generic.go:358] "Generic (PLEG): container finished" podID="e684c614-2482-4d10-9064-cfc4f2c83fb1" containerID="6645c78a755f0452b8ee5592cbdfc5a320109abb42ec7d060fa016aa516071d0" exitCode=0
Apr 17 08:05:15.235200 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:15.234871    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff" event={"ID":"e684c614-2482-4d10-9064-cfc4f2c83fb1","Type":"ContainerDied","Data":"6645c78a755f0452b8ee5592cbdfc5a320109abb42ec7d060fa016aa516071d0"}
Apr 17 08:05:17.245851 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:17.245793    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff" event={"ID":"e684c614-2482-4d10-9064-cfc4f2c83fb1","Type":"ContainerStarted","Data":"a98e0eb75d0a37d22f56cfd5a308c2457ffe5ec36420e8a02fe51ebb31ca2c4a"}
Apr 17 08:05:27.464273 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.464136    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"]
Apr 17 08:05:27.464817 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.464776    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9" podUID="585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" containerName="main" containerID="cri-o://b19025b8ef8a7dc540ffb9c642d31dd233307fa39c608c34de19bd9108d2a2bf" gracePeriod=30
Apr 17 08:05:27.724934 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.724855    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:05:27.795257 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.795220    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-home\") pod \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") "
Apr 17 08:05:27.795450 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.795323    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbcfb\" (UniqueName: \"kubernetes.io/projected/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-kube-api-access-rbcfb\") pod \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") "
Apr 17 08:05:27.795450 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.795347    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-kserve-provision-location\") pod \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") "
Apr 17 08:05:27.795450 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.795386    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-dshm\") pod \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") "
Apr 17 08:05:27.795450 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.795415    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-tls-certs\") pod \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") "
Apr 17 08:05:27.795792 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.795453    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-model-cache\") pod \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\" (UID: \"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de\") "
Apr 17 08:05:27.795792 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.795540    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-home" (OuterVolumeSpecName: "home") pod "585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" (UID: "585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:05:27.795792 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.795748    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-model-cache" (OuterVolumeSpecName: "model-cache") pod "585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" (UID: "585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:05:27.796679 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.796655    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-home\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:05:27.798257 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.798224    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" (UID: "585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 08:05:27.798372 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.798256    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-kube-api-access-rbcfb" (OuterVolumeSpecName: "kube-api-access-rbcfb") pod "585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" (UID: "585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de"). InnerVolumeSpecName "kube-api-access-rbcfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:05:27.798512 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.798487    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-dshm" (OuterVolumeSpecName: "dshm") pod "585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" (UID: "585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:05:27.870707 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.870662    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" (UID: "585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:05:27.898087 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.898050    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbcfb\" (UniqueName: \"kubernetes.io/projected/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-kube-api-access-rbcfb\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:05:27.898087 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.898080    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-kserve-provision-location\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:05:27.898087 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.898094    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-dshm\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:05:27.898351 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.898104    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-tls-certs\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:05:27.898351 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:27.898118    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de-model-cache\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:05:28.291235 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:28.291197    2571 generic.go:358] "Generic (PLEG): container finished" podID="585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" containerID="b19025b8ef8a7dc540ffb9c642d31dd233307fa39c608c34de19bd9108d2a2bf" exitCode=0
Apr 17 08:05:28.291420 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:28.291271    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"
Apr 17 08:05:28.291420 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:28.291281    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9" event={"ID":"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de","Type":"ContainerDied","Data":"b19025b8ef8a7dc540ffb9c642d31dd233307fa39c608c34de19bd9108d2a2bf"}
Apr 17 08:05:28.291420 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:28.291321    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9" event={"ID":"585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de","Type":"ContainerDied","Data":"4561c5583ee2f974de521b4cf97fd598a4b1e6a57c7ebab3dbd6c46976afe6ad"}
Apr 17 08:05:28.291420 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:28.291342    2571 scope.go:117] "RemoveContainer" containerID="b19025b8ef8a7dc540ffb9c642d31dd233307fa39c608c34de19bd9108d2a2bf"
Apr 17 08:05:28.303029 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:28.303008    2571 scope.go:117] "RemoveContainer" containerID="69519b53bd0d6b7c549138e853bd8a06f9f2c608c594245dc82b7243bc83e6ae"
Apr 17 08:05:28.317423 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:28.317398    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"]
Apr 17 08:05:28.319738 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:28.319713    2571 scope.go:117] "RemoveContainer" containerID="b19025b8ef8a7dc540ffb9c642d31dd233307fa39c608c34de19bd9108d2a2bf"
Apr 17 08:05:28.320143 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:05:28.320089    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b19025b8ef8a7dc540ffb9c642d31dd233307fa39c608c34de19bd9108d2a2bf\": container with ID starting with b19025b8ef8a7dc540ffb9c642d31dd233307fa39c608c34de19bd9108d2a2bf not found: ID does not exist" containerID="b19025b8ef8a7dc540ffb9c642d31dd233307fa39c608c34de19bd9108d2a2bf"
Apr 17 08:05:28.320242 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:28.320155    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b19025b8ef8a7dc540ffb9c642d31dd233307fa39c608c34de19bd9108d2a2bf"} err="failed to get container status \"b19025b8ef8a7dc540ffb9c642d31dd233307fa39c608c34de19bd9108d2a2bf\": rpc error: code = NotFound desc = could not find container \"b19025b8ef8a7dc540ffb9c642d31dd233307fa39c608c34de19bd9108d2a2bf\": container with ID starting with b19025b8ef8a7dc540ffb9c642d31dd233307fa39c608c34de19bd9108d2a2bf not found: ID does not exist"
Apr 17 08:05:28.320242 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:28.320191    2571 scope.go:117] "RemoveContainer" containerID="69519b53bd0d6b7c549138e853bd8a06f9f2c608c594245dc82b7243bc83e6ae"
Apr 17 08:05:28.320518 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:05:28.320487    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69519b53bd0d6b7c549138e853bd8a06f9f2c608c594245dc82b7243bc83e6ae\": container with ID starting with 69519b53bd0d6b7c549138e853bd8a06f9f2c608c594245dc82b7243bc83e6ae not found: ID does not exist" containerID="69519b53bd0d6b7c549138e853bd8a06f9f2c608c594245dc82b7243bc83e6ae"
Apr 17 08:05:28.320634 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:28.320526    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69519b53bd0d6b7c549138e853bd8a06f9f2c608c594245dc82b7243bc83e6ae"} err="failed to get container status \"69519b53bd0d6b7c549138e853bd8a06f9f2c608c594245dc82b7243bc83e6ae\": rpc error: code = NotFound desc = could not find container \"69519b53bd0d6b7c549138e853bd8a06f9f2c608c594245dc82b7243bc83e6ae\": container with ID starting with 69519b53bd0d6b7c549138e853bd8a06f9f2c608c594245dc82b7243bc83e6ae not found: ID does not exist"
Apr 17 08:05:28.322222 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:28.322200    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5657c85cdf-4xqr9"]
Apr 17 08:05:29.543621 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:29.543563    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" path="/var/lib/kubelet/pods/585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de/volumes"
Apr 17 08:05:47.374410 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:47.374373    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff" event={"ID":"e684c614-2482-4d10-9064-cfc4f2c83fb1","Type":"ContainerStarted","Data":"ba9107e2c038cafc604a08d22fde1b2e7f54c09b0d5f898639e0274c29a1a26f"}
Apr 17 08:05:47.374827 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:47.374664    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:47.377106 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:47.377080    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:47.395481 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:47.395414    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff" podStartSLOduration=3.154030601 podStartE2EDuration="34.395402738s" podCreationTimestamp="2026-04-17 08:05:13 +0000 UTC" firstStartedPulling="2026-04-17 08:05:15.236119466 +0000 UTC m=+786.289722662" lastFinishedPulling="2026-04-17 08:05:46.477491606 +0000 UTC m=+817.531094799" observedRunningTime="2026-04-17 08:05:47.394003802 +0000 UTC m=+818.447607003" watchObservedRunningTime="2026-04-17 08:05:47.395402738 +0000 UTC m=+818.449005949"
Apr 17 08:05:54.026444 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:54.026401    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:54.026858 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:54.026470    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:54.027973 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:54.027945    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:05:54.406158 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:05:54.406131    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:06:27.812183 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.812150    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"]
Apr 17 08:06:27.812649 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.812465    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" containerName="main"
Apr 17 08:06:27.812649 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.812477    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" containerName="main"
Apr 17 08:06:27.812649 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.812487    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" containerName="storage-initializer"
Apr 17 08:06:27.812649 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.812492    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" containerName="storage-initializer"
Apr 17 08:06:27.812649 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.812553    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="585bc1e7-fbc0-4ddc-841d-b94dd1a0f6de" containerName="main"
Apr 17 08:06:27.876494 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.876456    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"]
Apr 17 08:06:27.876678 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.876610    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:27.879246 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.879223    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-kserve-self-signed-certs\""
Apr 17 08:06:27.979556 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.979530    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7eec93e-7809-42ea-98a3-638e09c2163d-tls-certs\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:27.979679 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.979562    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-home\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:27.979679 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.979590    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-model-cache\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:27.979755 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.979686    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm75t\" (UniqueName: \"kubernetes.io/projected/c7eec93e-7809-42ea-98a3-638e09c2163d-kube-api-access-pm75t\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:27.979755 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.979734    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-dshm\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:27.979817 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:27.979764    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.080568 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.080497    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-dshm\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.080568 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.080538    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.080763 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.080569    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7eec93e-7809-42ea-98a3-638e09c2163d-tls-certs\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.080763 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.080598    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-home\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.080763 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.080618    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-model-cache\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.080763 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.080645    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pm75t\" (UniqueName: \"kubernetes.io/projected/c7eec93e-7809-42ea-98a3-638e09c2163d-kube-api-access-pm75t\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.081028 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.081008    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.081107 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.081081    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-home\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.081185 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.081165    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-model-cache\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.083381 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.083355    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-dshm\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.083604 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.083588    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7eec93e-7809-42ea-98a3-638e09c2163d-tls-certs\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.096376 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.096351    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm75t\" (UniqueName: \"kubernetes.io/projected/c7eec93e-7809-42ea-98a3-638e09c2163d-kube-api-access-pm75t\") pod \"precise-prefix-cache-test-kserve-c44d4d4cc-d59pp\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.187540 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.187507    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:28.309386 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.309359    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"]
Apr 17 08:06:28.310402 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:06:28.310376    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7eec93e_7809_42ea_98a3_638e09c2163d.slice/crio-c263282825b0db4d5d4572f3f70f7782a8c5fcd5042ba21be04bef106d81488f WatchSource:0}: Error finding container c263282825b0db4d5d4572f3f70f7782a8c5fcd5042ba21be04bef106d81488f: Status 404 returned error can't find the container with id c263282825b0db4d5d4572f3f70f7782a8c5fcd5042ba21be04bef106d81488f
Apr 17 08:06:28.524123 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.524088    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp" event={"ID":"c7eec93e-7809-42ea-98a3-638e09c2163d","Type":"ContainerStarted","Data":"c463653f7b6f48bbdb4b21c12f04592f9d82d4102689348a440f1652057d81de"}
Apr 17 08:06:28.524123 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:28.524123    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp" event={"ID":"c7eec93e-7809-42ea-98a3-638e09c2163d","Type":"ContainerStarted","Data":"c263282825b0db4d5d4572f3f70f7782a8c5fcd5042ba21be04bef106d81488f"}
Apr 17 08:06:33.543248 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:33.543215    2571 generic.go:358] "Generic (PLEG): container finished" podID="c7eec93e-7809-42ea-98a3-638e09c2163d" containerID="c463653f7b6f48bbdb4b21c12f04592f9d82d4102689348a440f1652057d81de" exitCode=0
Apr 17 08:06:33.543684 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:33.543287    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp" event={"ID":"c7eec93e-7809-42ea-98a3-638e09c2163d","Type":"ContainerDied","Data":"c463653f7b6f48bbdb4b21c12f04592f9d82d4102689348a440f1652057d81de"}
Apr 17 08:06:34.549027 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:34.548992    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp" event={"ID":"c7eec93e-7809-42ea-98a3-638e09c2163d","Type":"ContainerStarted","Data":"311ea36e7ecba691537cda90e8373c79958f7127de083ab360f631b2321f363a"}
Apr 17 08:06:34.568497 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:34.568452    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp" podStartSLOduration=7.568417944 podStartE2EDuration="7.568417944s" podCreationTimestamp="2026-04-17 08:06:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-17 08:06:34.56634131 +0000 UTC m=+865.619944520" watchObservedRunningTime="2026-04-17 08:06:34.568417944 +0000 UTC m=+865.622021158"
Apr 17 08:06:38.188170 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:38.188133    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:38.188170 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:38.188173    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:38.200778 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:38.200751    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:06:38.573105 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:06:38.573075    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:07:02.434495 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.434393    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"]
Apr 17 08:07:02.435056 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.434801    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp" podUID="c7eec93e-7809-42ea-98a3-638e09c2163d" containerName="main" containerID="cri-o://311ea36e7ecba691537cda90e8373c79958f7127de083ab360f631b2321f363a" gracePeriod=30
Apr 17 08:07:02.646115 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.646083    2571 generic.go:358] "Generic (PLEG): container finished" podID="c7eec93e-7809-42ea-98a3-638e09c2163d" containerID="311ea36e7ecba691537cda90e8373c79958f7127de083ab360f631b2321f363a" exitCode=0
Apr 17 08:07:02.646269 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.646151    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp" event={"ID":"c7eec93e-7809-42ea-98a3-638e09c2163d","Type":"ContainerDied","Data":"311ea36e7ecba691537cda90e8373c79958f7127de083ab360f631b2321f363a"}
Apr 17 08:07:02.689949 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.689887    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:07:02.776421 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.776383    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7eec93e-7809-42ea-98a3-638e09c2163d-tls-certs\") pod \"c7eec93e-7809-42ea-98a3-638e09c2163d\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") "
Apr 17 08:07:02.776641 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.776455    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-kserve-provision-location\") pod \"c7eec93e-7809-42ea-98a3-638e09c2163d\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") "
Apr 17 08:07:02.776641 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.776500    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-home\") pod \"c7eec93e-7809-42ea-98a3-638e09c2163d\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") "
Apr 17 08:07:02.776641 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.776537    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pm75t\" (UniqueName: \"kubernetes.io/projected/c7eec93e-7809-42ea-98a3-638e09c2163d-kube-api-access-pm75t\") pod \"c7eec93e-7809-42ea-98a3-638e09c2163d\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") "
Apr 17 08:07:02.776641 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.776575    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-model-cache\") pod \"c7eec93e-7809-42ea-98a3-638e09c2163d\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") "
Apr 17 08:07:02.776641 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.776613    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-dshm\") pod \"c7eec93e-7809-42ea-98a3-638e09c2163d\" (UID: \"c7eec93e-7809-42ea-98a3-638e09c2163d\") "
Apr 17 08:07:02.776896 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.776730    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-home" (OuterVolumeSpecName: "home") pod "c7eec93e-7809-42ea-98a3-638e09c2163d" (UID: "c7eec93e-7809-42ea-98a3-638e09c2163d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:07:02.776896 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.776846    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-home\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:07:02.776896 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.776863    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-model-cache" (OuterVolumeSpecName: "model-cache") pod "c7eec93e-7809-42ea-98a3-638e09c2163d" (UID: "c7eec93e-7809-42ea-98a3-638e09c2163d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:07:02.778781 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.778750    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7eec93e-7809-42ea-98a3-638e09c2163d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c7eec93e-7809-42ea-98a3-638e09c2163d" (UID: "c7eec93e-7809-42ea-98a3-638e09c2163d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 08:07:02.778781 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.778750    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-dshm" (OuterVolumeSpecName: "dshm") pod "c7eec93e-7809-42ea-98a3-638e09c2163d" (UID: "c7eec93e-7809-42ea-98a3-638e09c2163d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:07:02.778935 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.778856    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7eec93e-7809-42ea-98a3-638e09c2163d-kube-api-access-pm75t" (OuterVolumeSpecName: "kube-api-access-pm75t") pod "c7eec93e-7809-42ea-98a3-638e09c2163d" (UID: "c7eec93e-7809-42ea-98a3-638e09c2163d"). InnerVolumeSpecName "kube-api-access-pm75t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:07:02.830218 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.830155    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c7eec93e-7809-42ea-98a3-638e09c2163d" (UID: "c7eec93e-7809-42ea-98a3-638e09c2163d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:07:02.877648 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.877606    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7eec93e-7809-42ea-98a3-638e09c2163d-tls-certs\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:07:02.877648 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.877641    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-kserve-provision-location\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:07:02.877648 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.877651    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm75t\" (UniqueName: \"kubernetes.io/projected/c7eec93e-7809-42ea-98a3-638e09c2163d-kube-api-access-pm75t\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:07:02.877906 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.877660    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-model-cache\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:07:02.877906 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:02.877669    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7eec93e-7809-42ea-98a3-638e09c2163d-dshm\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:07:03.652047 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:03.652010    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp" event={"ID":"c7eec93e-7809-42ea-98a3-638e09c2163d","Type":"ContainerDied","Data":"c263282825b0db4d5d4572f3f70f7782a8c5fcd5042ba21be04bef106d81488f"}
Apr 17 08:07:03.652047 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:03.652044    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"
Apr 17 08:07:03.652047 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:03.652055    2571 scope.go:117] "RemoveContainer" containerID="311ea36e7ecba691537cda90e8373c79958f7127de083ab360f631b2321f363a"
Apr 17 08:07:03.661015 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:03.660997    2571 scope.go:117] "RemoveContainer" containerID="c463653f7b6f48bbdb4b21c12f04592f9d82d4102689348a440f1652057d81de"
Apr 17 08:07:03.670733 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:03.670711    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"]
Apr 17 08:07:03.674846 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:03.674826    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-c44d4d4cc-d59pp"]
Apr 17 08:07:05.542822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:05.542786    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7eec93e-7809-42ea-98a3-638e09c2163d" path="/var/lib/kubelet/pods/c7eec93e-7809-42ea-98a3-638e09c2163d/volumes"
Apr 17 08:07:09.488524 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:09.488486    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 08:07:09.489043 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:09.488872    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 08:07:52.784999 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:52.784967    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"]
Apr 17 08:07:52.785443 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:52.785251    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff" podUID="e684c614-2482-4d10-9064-cfc4f2c83fb1" containerName="main" containerID="cri-o://a98e0eb75d0a37d22f56cfd5a308c2457ffe5ec36420e8a02fe51ebb31ca2c4a" gracePeriod=30
Apr 17 08:07:52.785443 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:52.785296    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff" podUID="e684c614-2482-4d10-9064-cfc4f2c83fb1" containerName="tokenizer" containerID="cri-o://ba9107e2c038cafc604a08d22fde1b2e7f54c09b0d5f898639e0274c29a1a26f" gracePeriod=30
Apr 17 08:07:53.836946 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:53.836904    2571 generic.go:358] "Generic (PLEG): container finished" podID="e684c614-2482-4d10-9064-cfc4f2c83fb1" containerID="ba9107e2c038cafc604a08d22fde1b2e7f54c09b0d5f898639e0274c29a1a26f" exitCode=0
Apr 17 08:07:53.836946 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:53.836940    2571 generic.go:358] "Generic (PLEG): container finished" podID="e684c614-2482-4d10-9064-cfc4f2c83fb1" containerID="a98e0eb75d0a37d22f56cfd5a308c2457ffe5ec36420e8a02fe51ebb31ca2c4a" exitCode=0
Apr 17 08:07:53.837454 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:53.836960    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff" event={"ID":"e684c614-2482-4d10-9064-cfc4f2c83fb1","Type":"ContainerDied","Data":"ba9107e2c038cafc604a08d22fde1b2e7f54c09b0d5f898639e0274c29a1a26f"}
Apr 17 08:07:53.837454 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:53.836984    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff" event={"ID":"e684c614-2482-4d10-9064-cfc4f2c83fb1","Type":"ContainerDied","Data":"a98e0eb75d0a37d22f56cfd5a308c2457ffe5ec36420e8a02fe51ebb31ca2c4a"}
Apr 17 08:07:53.933625 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:53.933603    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:07:54.087276 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.087246    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6v65\" (UniqueName: \"kubernetes.io/projected/e684c614-2482-4d10-9064-cfc4f2c83fb1-kube-api-access-p6v65\") pod \"e684c614-2482-4d10-9064-cfc4f2c83fb1\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") "
Apr 17 08:07:54.087491 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.087290    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-tmp\") pod \"e684c614-2482-4d10-9064-cfc4f2c83fb1\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") "
Apr 17 08:07:54.087491 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.087318    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-uds\") pod \"e684c614-2482-4d10-9064-cfc4f2c83fb1\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") "
Apr 17 08:07:54.087491 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.087373    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e684c614-2482-4d10-9064-cfc4f2c83fb1-tls-certs\") pod \"e684c614-2482-4d10-9064-cfc4f2c83fb1\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") "
Apr 17 08:07:54.087491 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.087427    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-kserve-provision-location\") pod \"e684c614-2482-4d10-9064-cfc4f2c83fb1\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") "
Apr 17 08:07:54.087714 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.087520    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-cache\") pod \"e684c614-2482-4d10-9064-cfc4f2c83fb1\" (UID: \"e684c614-2482-4d10-9064-cfc4f2c83fb1\") "
Apr 17 08:07:54.087714 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.087666    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "e684c614-2482-4d10-9064-cfc4f2c83fb1" (UID: "e684c614-2482-4d10-9064-cfc4f2c83fb1"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:07:54.087819 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.087701    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "e684c614-2482-4d10-9064-cfc4f2c83fb1" (UID: "e684c614-2482-4d10-9064-cfc4f2c83fb1"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:07:54.087858 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.087812    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "e684c614-2482-4d10-9064-cfc4f2c83fb1" (UID: "e684c614-2482-4d10-9064-cfc4f2c83fb1"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:07:54.087858 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.087824    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-tmp\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:07:54.087931 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.087867    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-uds\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:07:54.088148 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.088129    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "e684c614-2482-4d10-9064-cfc4f2c83fb1" (UID: "e684c614-2482-4d10-9064-cfc4f2c83fb1"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:07:54.089446 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.089403    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e684c614-2482-4d10-9064-cfc4f2c83fb1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e684c614-2482-4d10-9064-cfc4f2c83fb1" (UID: "e684c614-2482-4d10-9064-cfc4f2c83fb1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 08:07:54.089570 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.089454    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e684c614-2482-4d10-9064-cfc4f2c83fb1-kube-api-access-p6v65" (OuterVolumeSpecName: "kube-api-access-p6v65") pod "e684c614-2482-4d10-9064-cfc4f2c83fb1" (UID: "e684c614-2482-4d10-9064-cfc4f2c83fb1"). InnerVolumeSpecName "kube-api-access-p6v65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:07:54.188795 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.188763    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6v65\" (UniqueName: \"kubernetes.io/projected/e684c614-2482-4d10-9064-cfc4f2c83fb1-kube-api-access-p6v65\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:07:54.188795 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.188789    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e684c614-2482-4d10-9064-cfc4f2c83fb1-tls-certs\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:07:54.188795 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.188800    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-kserve-provision-location\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:07:54.189018 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.188809    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/e684c614-2482-4d10-9064-cfc4f2c83fb1-tokenizer-cache\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:07:54.841864 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.841823    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff" event={"ID":"e684c614-2482-4d10-9064-cfc4f2c83fb1","Type":"ContainerDied","Data":"9c313635f7e7f6d23ba1cd0cbdd933ef4e3d60d7eeb2efca15a2778da8c938c0"}
Apr 17 08:07:54.841864 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.841870    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"
Apr 17 08:07:54.842291 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.841881    2571 scope.go:117] "RemoveContainer" containerID="ba9107e2c038cafc604a08d22fde1b2e7f54c09b0d5f898639e0274c29a1a26f"
Apr 17 08:07:54.850548 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.850465    2571 scope.go:117] "RemoveContainer" containerID="a98e0eb75d0a37d22f56cfd5a308c2457ffe5ec36420e8a02fe51ebb31ca2c4a"
Apr 17 08:07:54.857689 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.857669    2571 scope.go:117] "RemoveContainer" containerID="6645c78a755f0452b8ee5592cbdfc5a320109abb42ec7d060fa016aa516071d0"
Apr 17 08:07:54.864932 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.864904    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"]
Apr 17 08:07:54.869484 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:54.869408    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schecmdff"]
Apr 17 08:07:55.543150 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:07:55.543115    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e684c614-2482-4d10-9064-cfc4f2c83fb1" path="/var/lib/kubelet/pods/e684c614-2482-4d10-9064-cfc4f2c83fb1/volumes"
Apr 17 08:08:02.100532 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100491    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"]
Apr 17 08:08:02.101008 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100787    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e684c614-2482-4d10-9064-cfc4f2c83fb1" containerName="storage-initializer"
Apr 17 08:08:02.101008 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100798    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e684c614-2482-4d10-9064-cfc4f2c83fb1" containerName="storage-initializer"
Apr 17 08:08:02.101008 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100814    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7eec93e-7809-42ea-98a3-638e09c2163d" containerName="main"
Apr 17 08:08:02.101008 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100821    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7eec93e-7809-42ea-98a3-638e09c2163d" containerName="main"
Apr 17 08:08:02.101008 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100833    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e684c614-2482-4d10-9064-cfc4f2c83fb1" containerName="main"
Apr 17 08:08:02.101008 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100839    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e684c614-2482-4d10-9064-cfc4f2c83fb1" containerName="main"
Apr 17 08:08:02.101008 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100847    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e684c614-2482-4d10-9064-cfc4f2c83fb1" containerName="tokenizer"
Apr 17 08:08:02.101008 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100852    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e684c614-2482-4d10-9064-cfc4f2c83fb1" containerName="tokenizer"
Apr 17 08:08:02.101008 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100859    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7eec93e-7809-42ea-98a3-638e09c2163d" containerName="storage-initializer"
Apr 17 08:08:02.101008 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100864    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7eec93e-7809-42ea-98a3-638e09c2163d" containerName="storage-initializer"
Apr 17 08:08:02.101008 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100909    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e684c614-2482-4d10-9064-cfc4f2c83fb1" containerName="tokenizer"
Apr 17 08:08:02.101008 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100929    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e684c614-2482-4d10-9064-cfc4f2c83fb1" containerName="main"
Apr 17 08:08:02.101008 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.100936    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="c7eec93e-7809-42ea-98a3-638e09c2163d" containerName="main"
Apr 17 08:08:02.103789 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.103767    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.106274 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.106245    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 17 08:08:02.106452 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.106304    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 17 08:08:02.107314 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.107290    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-epp-sa-dockercfg-k2dm6\""
Apr 17 08:08:02.107416 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.107398    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jj9s6\""
Apr 17 08:08:02.107493 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.107469    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Apr 17 08:08:02.113949 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.113921    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"]
Apr 17 08:08:02.260570 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.260537    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.260747 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.260578    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.260747 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.260600    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-cache\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.260747 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.260654    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk89h\" (UniqueName: \"kubernetes.io/projected/d82906c6-b2d4-4f72-966f-92872e02370a-kube-api-access-gk89h\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.260747 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.260692    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d82906c6-b2d4-4f72-966f-92872e02370a-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.260747 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.260710    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-tmp\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.361670 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.361584    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.361670 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.361624    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.361670 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.361643    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-cache\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.361926 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.361676    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gk89h\" (UniqueName: \"kubernetes.io/projected/d82906c6-b2d4-4f72-966f-92872e02370a-kube-api-access-gk89h\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.361926 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.361717    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d82906c6-b2d4-4f72-966f-92872e02370a-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.361926 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.361733    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-tmp\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.362110 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.361976    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.362110 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.362056    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-cache\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.362110 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.362075    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.362236 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.362131    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-tmp\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.364198 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.364176    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d82906c6-b2d4-4f72-966f-92872e02370a-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.370422 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.370401    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk89h\" (UniqueName: \"kubernetes.io/projected/d82906c6-b2d4-4f72-966f-92872e02370a-kube-api-access-gk89h\") pod \"custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.414019 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.413991    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:02.539956 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.539928    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"]
Apr 17 08:08:02.541579 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:08:02.541553    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd82906c6_b2d4_4f72_966f_92872e02370a.slice/crio-2ed2b56943cfb2fe9ee70729df784036bcecdaa599741d9d4e613c4d5cabcaa6 WatchSource:0}: Error finding container 2ed2b56943cfb2fe9ee70729df784036bcecdaa599741d9d4e613c4d5cabcaa6: Status 404 returned error can't find the container with id 2ed2b56943cfb2fe9ee70729df784036bcecdaa599741d9d4e613c4d5cabcaa6
Apr 17 08:08:02.871257 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.871219    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw" event={"ID":"d82906c6-b2d4-4f72-966f-92872e02370a","Type":"ContainerStarted","Data":"41735a113f9e1bdf71902e4d1f06a298340de7b76a0a944fa82236de0ab20c03"}
Apr 17 08:08:02.871257 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:02.871258    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw" event={"ID":"d82906c6-b2d4-4f72-966f-92872e02370a","Type":"ContainerStarted","Data":"2ed2b56943cfb2fe9ee70729df784036bcecdaa599741d9d4e613c4d5cabcaa6"}
Apr 17 08:08:03.879098 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:03.877295    2571 generic.go:358] "Generic (PLEG): container finished" podID="d82906c6-b2d4-4f72-966f-92872e02370a" containerID="41735a113f9e1bdf71902e4d1f06a298340de7b76a0a944fa82236de0ab20c03" exitCode=0
Apr 17 08:08:03.879098 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:03.877420    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw" event={"ID":"d82906c6-b2d4-4f72-966f-92872e02370a","Type":"ContainerDied","Data":"41735a113f9e1bdf71902e4d1f06a298340de7b76a0a944fa82236de0ab20c03"}
Apr 17 08:08:04.883095 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:04.883057    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw" event={"ID":"d82906c6-b2d4-4f72-966f-92872e02370a","Type":"ContainerStarted","Data":"45848fe9d7b9632e5c5cb7d6469f3529475d756258126cb53fc7ba83a5002571"}
Apr 17 08:08:04.883095 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:04.883094    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw" event={"ID":"d82906c6-b2d4-4f72-966f-92872e02370a","Type":"ContainerStarted","Data":"f783bf3620d03eb76d95fb004d2d668bb51d991a461346c06fe363135fda615b"}
Apr 17 08:08:04.883508 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:04.883247    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:04.903467 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:04.903388    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw" podStartSLOduration=2.903371364 podStartE2EDuration="2.903371364s" podCreationTimestamp="2026-04-17 08:08:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-17 08:08:04.902892278 +0000 UTC m=+955.956495518" watchObservedRunningTime="2026-04-17 08:08:04.903371364 +0000 UTC m=+955.956974576"
Apr 17 08:08:12.414350 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:12.414310    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:12.414350 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:12.414357    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:12.416910 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:12.416885    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:12.914485 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:12.914444    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:08:33.919271 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:08:33.919195    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:09:58.407222 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:58.407145    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"]
Apr 17 08:09:58.407698 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:58.407452    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw" podUID="d82906c6-b2d4-4f72-966f-92872e02370a" containerName="main" containerID="cri-o://f783bf3620d03eb76d95fb004d2d668bb51d991a461346c06fe363135fda615b" gracePeriod=30
Apr 17 08:09:58.407698 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:58.407503    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw" podUID="d82906c6-b2d4-4f72-966f-92872e02370a" containerName="tokenizer" containerID="cri-o://45848fe9d7b9632e5c5cb7d6469f3529475d756258126cb53fc7ba83a5002571" gracePeriod=30
Apr 17 08:09:59.295351 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.295316    2571 generic.go:358] "Generic (PLEG): container finished" podID="d82906c6-b2d4-4f72-966f-92872e02370a" containerID="f783bf3620d03eb76d95fb004d2d668bb51d991a461346c06fe363135fda615b" exitCode=0
Apr 17 08:09:59.295552 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.295388    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw" event={"ID":"d82906c6-b2d4-4f72-966f-92872e02370a","Type":"ContainerDied","Data":"f783bf3620d03eb76d95fb004d2d668bb51d991a461346c06fe363135fda615b"}
Apr 17 08:09:59.553730 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.553677    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:09:59.624365 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.624332    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-cache\") pod \"d82906c6-b2d4-4f72-966f-92872e02370a\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") "
Apr 17 08:09:59.624550 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.624407    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-uds\") pod \"d82906c6-b2d4-4f72-966f-92872e02370a\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") "
Apr 17 08:09:59.624550 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.624461    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d82906c6-b2d4-4f72-966f-92872e02370a-tls-certs\") pod \"d82906c6-b2d4-4f72-966f-92872e02370a\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") "
Apr 17 08:09:59.624550 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.624517    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-tmp\") pod \"d82906c6-b2d4-4f72-966f-92872e02370a\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") "
Apr 17 08:09:59.624683 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.624555    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gk89h\" (UniqueName: \"kubernetes.io/projected/d82906c6-b2d4-4f72-966f-92872e02370a-kube-api-access-gk89h\") pod \"d82906c6-b2d4-4f72-966f-92872e02370a\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") "
Apr 17 08:09:59.624683 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.624560    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "d82906c6-b2d4-4f72-966f-92872e02370a" (UID: "d82906c6-b2d4-4f72-966f-92872e02370a"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:09:59.624683 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.624574    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-kserve-provision-location\") pod \"d82906c6-b2d4-4f72-966f-92872e02370a\" (UID: \"d82906c6-b2d4-4f72-966f-92872e02370a\") "
Apr 17 08:09:59.624795 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.624693    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "d82906c6-b2d4-4f72-966f-92872e02370a" (UID: "d82906c6-b2d4-4f72-966f-92872e02370a"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:09:59.624795 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.624726    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-cache\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:09:59.624874 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.624845    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "d82906c6-b2d4-4f72-966f-92872e02370a" (UID: "d82906c6-b2d4-4f72-966f-92872e02370a"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:09:59.625302 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.625282    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d82906c6-b2d4-4f72-966f-92872e02370a" (UID: "d82906c6-b2d4-4f72-966f-92872e02370a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:09:59.626622 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.626594    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d82906c6-b2d4-4f72-966f-92872e02370a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d82906c6-b2d4-4f72-966f-92872e02370a" (UID: "d82906c6-b2d4-4f72-966f-92872e02370a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 08:09:59.626724 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.626626    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d82906c6-b2d4-4f72-966f-92872e02370a-kube-api-access-gk89h" (OuterVolumeSpecName: "kube-api-access-gk89h") pod "d82906c6-b2d4-4f72-966f-92872e02370a" (UID: "d82906c6-b2d4-4f72-966f-92872e02370a"). InnerVolumeSpecName "kube-api-access-gk89h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:09:59.725421 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.725389    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-tmp\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:09:59.725421 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.725414    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gk89h\" (UniqueName: \"kubernetes.io/projected/d82906c6-b2d4-4f72-966f-92872e02370a-kube-api-access-gk89h\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:09:59.725421 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.725423    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-kserve-provision-location\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:09:59.725685 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.725466    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d82906c6-b2d4-4f72-966f-92872e02370a-tokenizer-uds\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:09:59.725685 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:09:59.725482    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d82906c6-b2d4-4f72-966f-92872e02370a-tls-certs\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:10:00.300257 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.300223    2571 generic.go:358] "Generic (PLEG): container finished" podID="d82906c6-b2d4-4f72-966f-92872e02370a" containerID="45848fe9d7b9632e5c5cb7d6469f3529475d756258126cb53fc7ba83a5002571" exitCode=0
Apr 17 08:10:00.300477 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.300309    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"
Apr 17 08:10:00.300477 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.300314    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw" event={"ID":"d82906c6-b2d4-4f72-966f-92872e02370a","Type":"ContainerDied","Data":"45848fe9d7b9632e5c5cb7d6469f3529475d756258126cb53fc7ba83a5002571"}
Apr 17 08:10:00.300477 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.300358    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw" event={"ID":"d82906c6-b2d4-4f72-966f-92872e02370a","Type":"ContainerDied","Data":"2ed2b56943cfb2fe9ee70729df784036bcecdaa599741d9d4e613c4d5cabcaa6"}
Apr 17 08:10:00.300477 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.300378    2571 scope.go:117] "RemoveContainer" containerID="45848fe9d7b9632e5c5cb7d6469f3529475d756258126cb53fc7ba83a5002571"
Apr 17 08:10:00.309542 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.309523    2571 scope.go:117] "RemoveContainer" containerID="f783bf3620d03eb76d95fb004d2d668bb51d991a461346c06fe363135fda615b"
Apr 17 08:10:00.317341 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.317326    2571 scope.go:117] "RemoveContainer" containerID="41735a113f9e1bdf71902e4d1f06a298340de7b76a0a944fa82236de0ab20c03"
Apr 17 08:10:00.323082 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.323062    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"]
Apr 17 08:10:00.324841 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.324827    2571 scope.go:117] "RemoveContainer" containerID="45848fe9d7b9632e5c5cb7d6469f3529475d756258126cb53fc7ba83a5002571"
Apr 17 08:10:00.325093 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:10:00.325074    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45848fe9d7b9632e5c5cb7d6469f3529475d756258126cb53fc7ba83a5002571\": container with ID starting with 45848fe9d7b9632e5c5cb7d6469f3529475d756258126cb53fc7ba83a5002571 not found: ID does not exist" containerID="45848fe9d7b9632e5c5cb7d6469f3529475d756258126cb53fc7ba83a5002571"
Apr 17 08:10:00.325173 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.325107    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45848fe9d7b9632e5c5cb7d6469f3529475d756258126cb53fc7ba83a5002571"} err="failed to get container status \"45848fe9d7b9632e5c5cb7d6469f3529475d756258126cb53fc7ba83a5002571\": rpc error: code = NotFound desc = could not find container \"45848fe9d7b9632e5c5cb7d6469f3529475d756258126cb53fc7ba83a5002571\": container with ID starting with 45848fe9d7b9632e5c5cb7d6469f3529475d756258126cb53fc7ba83a5002571 not found: ID does not exist"
Apr 17 08:10:00.325173 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.325131    2571 scope.go:117] "RemoveContainer" containerID="f783bf3620d03eb76d95fb004d2d668bb51d991a461346c06fe363135fda615b"
Apr 17 08:10:00.325352 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:10:00.325329    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f783bf3620d03eb76d95fb004d2d668bb51d991a461346c06fe363135fda615b\": container with ID starting with f783bf3620d03eb76d95fb004d2d668bb51d991a461346c06fe363135fda615b not found: ID does not exist" containerID="f783bf3620d03eb76d95fb004d2d668bb51d991a461346c06fe363135fda615b"
Apr 17 08:10:00.325415 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.325361    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f783bf3620d03eb76d95fb004d2d668bb51d991a461346c06fe363135fda615b"} err="failed to get container status \"f783bf3620d03eb76d95fb004d2d668bb51d991a461346c06fe363135fda615b\": rpc error: code = NotFound desc = could not find container \"f783bf3620d03eb76d95fb004d2d668bb51d991a461346c06fe363135fda615b\": container with ID starting with f783bf3620d03eb76d95fb004d2d668bb51d991a461346c06fe363135fda615b not found: ID does not exist"
Apr 17 08:10:00.325415 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.325383    2571 scope.go:117] "RemoveContainer" containerID="41735a113f9e1bdf71902e4d1f06a298340de7b76a0a944fa82236de0ab20c03"
Apr 17 08:10:00.325617 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:10:00.325594    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41735a113f9e1bdf71902e4d1f06a298340de7b76a0a944fa82236de0ab20c03\": container with ID starting with 41735a113f9e1bdf71902e4d1f06a298340de7b76a0a944fa82236de0ab20c03 not found: ID does not exist" containerID="41735a113f9e1bdf71902e4d1f06a298340de7b76a0a944fa82236de0ab20c03"
Apr 17 08:10:00.325662 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.325624    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41735a113f9e1bdf71902e4d1f06a298340de7b76a0a944fa82236de0ab20c03"} err="failed to get container status \"41735a113f9e1bdf71902e4d1f06a298340de7b76a0a944fa82236de0ab20c03\": rpc error: code = NotFound desc = could not find container \"41735a113f9e1bdf71902e4d1f06a298340de7b76a0a944fa82236de0ab20c03\": container with ID starting with 41735a113f9e1bdf71902e4d1f06a298340de7b76a0a944fa82236de0ab20c03 not found: ID does not exist"
Apr 17 08:10:00.328604 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:00.328584    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-6884c984zq4xw"]
Apr 17 08:10:01.543822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:01.543789    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d82906c6-b2d4-4f72-966f-92872e02370a" path="/var/lib/kubelet/pods/d82906c6-b2d4-4f72-966f-92872e02370a/volumes"
Apr 17 08:10:09.775250 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.775215    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"]
Apr 17 08:10:09.775707 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.775689    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d82906c6-b2d4-4f72-966f-92872e02370a" containerName="main"
Apr 17 08:10:09.775760 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.775711    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d82906c6-b2d4-4f72-966f-92872e02370a" containerName="main"
Apr 17 08:10:09.775760 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.775723    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d82906c6-b2d4-4f72-966f-92872e02370a" containerName="storage-initializer"
Apr 17 08:10:09.775760 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.775732    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d82906c6-b2d4-4f72-966f-92872e02370a" containerName="storage-initializer"
Apr 17 08:10:09.775760 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.775755    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d82906c6-b2d4-4f72-966f-92872e02370a" containerName="tokenizer"
Apr 17 08:10:09.775902 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.775764    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d82906c6-b2d4-4f72-966f-92872e02370a" containerName="tokenizer"
Apr 17 08:10:09.775902 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.775848    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="d82906c6-b2d4-4f72-966f-92872e02370a" containerName="tokenizer"
Apr 17 08:10:09.775902 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.775863    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="d82906c6-b2d4-4f72-966f-92872e02370a" containerName="main"
Apr 17 08:10:09.780541 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.780516    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.783377 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.783315    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 17 08:10:09.784183 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.784162    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 17 08:10:09.784337 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.784320    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-test-epp-sa-dockercfg-bzjz9\""
Apr 17 08:10:09.784459 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.784332    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-test-kserve-self-signed-certs\""
Apr 17 08:10:09.784459 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.784292    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jj9s6\""
Apr 17 08:10:09.790587 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.790557    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"]
Apr 17 08:10:09.888213 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.888185    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-tmp\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.888378 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.888246    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.888378 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.888272    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.888378 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.888287    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.888378 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.888303    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knbjl\" (UniqueName: \"kubernetes.io/projected/b496ceba-4b4c-4f52-a4aa-bb20547e894a-kube-api-access-knbjl\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.888378 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.888324    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-cache\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.988822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.988792    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.988965 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.988831    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.988965 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.988855    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.988965 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.988881    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-knbjl\" (UniqueName: \"kubernetes.io/projected/b496ceba-4b4c-4f52-a4aa-bb20547e894a-kube-api-access-knbjl\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.988965 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.988914    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-cache\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.989156 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.989006    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-tmp\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.989238 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.989220    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.989329 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.989312    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-cache\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.989383 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.989337    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.989419 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.989378    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-tmp\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.991294 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.991272    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:09.995905 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:09.995887    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knbjl\" (UniqueName: \"kubernetes.io/projected/b496ceba-4b4c-4f52-a4aa-bb20547e894a-kube-api-access-knbjl\") pod \"router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:10.093496 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:10.093403    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:10.214740 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:10.214715    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"]
Apr 17 08:10:10.216295 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:10:10.216270    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb496ceba_4b4c_4f52_a4aa_bb20547e894a.slice/crio-f7b9dd72e489651bd4a1f77d8f5a6ddd4503a8798e86a7ff8b0157420c3de396 WatchSource:0}: Error finding container f7b9dd72e489651bd4a1f77d8f5a6ddd4503a8798e86a7ff8b0157420c3de396: Status 404 returned error can't find the container with id f7b9dd72e489651bd4a1f77d8f5a6ddd4503a8798e86a7ff8b0157420c3de396
Apr 17 08:10:10.220733 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:10.220716    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 17 08:10:10.336393 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:10.336350    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb" event={"ID":"b496ceba-4b4c-4f52-a4aa-bb20547e894a","Type":"ContainerStarted","Data":"c3484440f1113e964a01ee4e9ce4c797088855bd160786813563b8cd6c21d862"}
Apr 17 08:10:10.336393 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:10.336397    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb" event={"ID":"b496ceba-4b4c-4f52-a4aa-bb20547e894a","Type":"ContainerStarted","Data":"f7b9dd72e489651bd4a1f77d8f5a6ddd4503a8798e86a7ff8b0157420c3de396"}
Apr 17 08:10:11.341347 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:11.341310    2571 generic.go:358] "Generic (PLEG): container finished" podID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" containerID="c3484440f1113e964a01ee4e9ce4c797088855bd160786813563b8cd6c21d862" exitCode=0
Apr 17 08:10:11.341840 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:11.341395    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb" event={"ID":"b496ceba-4b4c-4f52-a4aa-bb20547e894a","Type":"ContainerDied","Data":"c3484440f1113e964a01ee4e9ce4c797088855bd160786813563b8cd6c21d862"}
Apr 17 08:10:12.346412 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:12.346373    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb" event={"ID":"b496ceba-4b4c-4f52-a4aa-bb20547e894a","Type":"ContainerStarted","Data":"2d339b4d5a719b1bb65afaf1ebc3bc40cca3c55eb365fb3367a058e494e6bbe0"}
Apr 17 08:10:12.346412 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:12.346415    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb" event={"ID":"b496ceba-4b4c-4f52-a4aa-bb20547e894a","Type":"ContainerStarted","Data":"2bf50f2151def3333ced059e1b79d9c0d16866d5441519972c7b9d5fae3e8d24"}
Apr 17 08:10:12.346837 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:12.346477    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:20.093630 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:20.093594    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:20.093630 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:20.093643    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:20.096123 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:20.096097    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:20.116679 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:20.116626    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb" podStartSLOduration=11.11661078 podStartE2EDuration="11.11661078s" podCreationTimestamp="2026-04-17 08:10:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-17 08:10:12.367046012 +0000 UTC m=+1083.420649223" watchObservedRunningTime="2026-04-17 08:10:20.11661078 +0000 UTC m=+1091.170213991"
Apr 17 08:10:20.375054 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:20.374970    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:10:41.379528 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:10:41.379500    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:12:05.523237 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:05.523204    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"]
Apr 17 08:12:05.523702 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:05.523531    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb" podUID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" containerName="main" containerID="cri-o://2bf50f2151def3333ced059e1b79d9c0d16866d5441519972c7b9d5fae3e8d24" gracePeriod=30
Apr 17 08:12:05.523702 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:05.523585    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb" podUID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" containerName="tokenizer" containerID="cri-o://2d339b4d5a719b1bb65afaf1ebc3bc40cca3c55eb365fb3367a058e494e6bbe0" gracePeriod=30
Apr 17 08:12:05.747531 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:05.747500    2571 generic.go:358] "Generic (PLEG): container finished" podID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" containerID="2bf50f2151def3333ced059e1b79d9c0d16866d5441519972c7b9d5fae3e8d24" exitCode=0
Apr 17 08:12:05.747713 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:05.747573    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb" event={"ID":"b496ceba-4b4c-4f52-a4aa-bb20547e894a","Type":"ContainerDied","Data":"2bf50f2151def3333ced059e1b79d9c0d16866d5441519972c7b9d5fae3e8d24"}
Apr 17 08:12:06.676035 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.676015    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:12:06.752310 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.752278    2571 generic.go:358] "Generic (PLEG): container finished" podID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" containerID="2d339b4d5a719b1bb65afaf1ebc3bc40cca3c55eb365fb3367a058e494e6bbe0" exitCode=0
Apr 17 08:12:06.752519 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.752327    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb" event={"ID":"b496ceba-4b4c-4f52-a4aa-bb20547e894a","Type":"ContainerDied","Data":"2d339b4d5a719b1bb65afaf1ebc3bc40cca3c55eb365fb3367a058e494e6bbe0"}
Apr 17 08:12:06.752519 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.752345    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"
Apr 17 08:12:06.752519 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.752360    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb" event={"ID":"b496ceba-4b4c-4f52-a4aa-bb20547e894a","Type":"ContainerDied","Data":"f7b9dd72e489651bd4a1f77d8f5a6ddd4503a8798e86a7ff8b0157420c3de396"}
Apr 17 08:12:06.752519 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.752381    2571 scope.go:117] "RemoveContainer" containerID="2d339b4d5a719b1bb65afaf1ebc3bc40cca3c55eb365fb3367a058e494e6bbe0"
Apr 17 08:12:06.760341 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.760324    2571 scope.go:117] "RemoveContainer" containerID="2bf50f2151def3333ced059e1b79d9c0d16866d5441519972c7b9d5fae3e8d24"
Apr 17 08:12:06.767299 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.767284    2571 scope.go:117] "RemoveContainer" containerID="c3484440f1113e964a01ee4e9ce4c797088855bd160786813563b8cd6c21d862"
Apr 17 08:12:06.774363 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.774347    2571 scope.go:117] "RemoveContainer" containerID="2d339b4d5a719b1bb65afaf1ebc3bc40cca3c55eb365fb3367a058e494e6bbe0"
Apr 17 08:12:06.774674 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:12:06.774647    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d339b4d5a719b1bb65afaf1ebc3bc40cca3c55eb365fb3367a058e494e6bbe0\": container with ID starting with 2d339b4d5a719b1bb65afaf1ebc3bc40cca3c55eb365fb3367a058e494e6bbe0 not found: ID does not exist" containerID="2d339b4d5a719b1bb65afaf1ebc3bc40cca3c55eb365fb3367a058e494e6bbe0"
Apr 17 08:12:06.774750 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.774686    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d339b4d5a719b1bb65afaf1ebc3bc40cca3c55eb365fb3367a058e494e6bbe0"} err="failed to get container status \"2d339b4d5a719b1bb65afaf1ebc3bc40cca3c55eb365fb3367a058e494e6bbe0\": rpc error: code = NotFound desc = could not find container \"2d339b4d5a719b1bb65afaf1ebc3bc40cca3c55eb365fb3367a058e494e6bbe0\": container with ID starting with 2d339b4d5a719b1bb65afaf1ebc3bc40cca3c55eb365fb3367a058e494e6bbe0 not found: ID does not exist"
Apr 17 08:12:06.774750 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.774710    2571 scope.go:117] "RemoveContainer" containerID="2bf50f2151def3333ced059e1b79d9c0d16866d5441519972c7b9d5fae3e8d24"
Apr 17 08:12:06.774952 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:12:06.774935    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bf50f2151def3333ced059e1b79d9c0d16866d5441519972c7b9d5fae3e8d24\": container with ID starting with 2bf50f2151def3333ced059e1b79d9c0d16866d5441519972c7b9d5fae3e8d24 not found: ID does not exist" containerID="2bf50f2151def3333ced059e1b79d9c0d16866d5441519972c7b9d5fae3e8d24"
Apr 17 08:12:06.774997 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.774958    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bf50f2151def3333ced059e1b79d9c0d16866d5441519972c7b9d5fae3e8d24"} err="failed to get container status \"2bf50f2151def3333ced059e1b79d9c0d16866d5441519972c7b9d5fae3e8d24\": rpc error: code = NotFound desc = could not find container \"2bf50f2151def3333ced059e1b79d9c0d16866d5441519972c7b9d5fae3e8d24\": container with ID starting with 2bf50f2151def3333ced059e1b79d9c0d16866d5441519972c7b9d5fae3e8d24 not found: ID does not exist"
Apr 17 08:12:06.774997 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.774975    2571 scope.go:117] "RemoveContainer" containerID="c3484440f1113e964a01ee4e9ce4c797088855bd160786813563b8cd6c21d862"
Apr 17 08:12:06.775223 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:12:06.775206    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3484440f1113e964a01ee4e9ce4c797088855bd160786813563b8cd6c21d862\": container with ID starting with c3484440f1113e964a01ee4e9ce4c797088855bd160786813563b8cd6c21d862 not found: ID does not exist" containerID="c3484440f1113e964a01ee4e9ce4c797088855bd160786813563b8cd6c21d862"
Apr 17 08:12:06.775265 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.775228    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3484440f1113e964a01ee4e9ce4c797088855bd160786813563b8cd6c21d862"} err="failed to get container status \"c3484440f1113e964a01ee4e9ce4c797088855bd160786813563b8cd6c21d862\": rpc error: code = NotFound desc = could not find container \"c3484440f1113e964a01ee4e9ce4c797088855bd160786813563b8cd6c21d862\": container with ID starting with c3484440f1113e964a01ee4e9ce4c797088855bd160786813563b8cd6c21d862 not found: ID does not exist"
Apr 17 08:12:06.804872 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.804850    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knbjl\" (UniqueName: \"kubernetes.io/projected/b496ceba-4b4c-4f52-a4aa-bb20547e894a-kube-api-access-knbjl\") pod \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") "
Apr 17 08:12:06.804990 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.804891    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tls-certs\") pod \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") "
Apr 17 08:12:06.804990 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.804933    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-uds\") pod \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") "
Apr 17 08:12:06.804990 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.804984    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-cache\") pod \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") "
Apr 17 08:12:06.805149 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.805029    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-kserve-provision-location\") pod \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") "
Apr 17 08:12:06.805149 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.805104    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-tmp\") pod \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\" (UID: \"b496ceba-4b4c-4f52-a4aa-bb20547e894a\") "
Apr 17 08:12:06.805240 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.805200    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "b496ceba-4b4c-4f52-a4aa-bb20547e894a" (UID: "b496ceba-4b4c-4f52-a4aa-bb20547e894a"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:12:06.805310 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.805284    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "b496ceba-4b4c-4f52-a4aa-bb20547e894a" (UID: "b496ceba-4b4c-4f52-a4aa-bb20547e894a"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:12:06.805359 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.805349    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-uds\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:12:06.805494 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.805472    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "b496ceba-4b4c-4f52-a4aa-bb20547e894a" (UID: "b496ceba-4b4c-4f52-a4aa-bb20547e894a"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:12:06.805726 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.805706    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b496ceba-4b4c-4f52-a4aa-bb20547e894a" (UID: "b496ceba-4b4c-4f52-a4aa-bb20547e894a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:12:06.806814 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.806794    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b496ceba-4b4c-4f52-a4aa-bb20547e894a" (UID: "b496ceba-4b4c-4f52-a4aa-bb20547e894a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 08:12:06.806903 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.806865    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b496ceba-4b4c-4f52-a4aa-bb20547e894a-kube-api-access-knbjl" (OuterVolumeSpecName: "kube-api-access-knbjl") pod "b496ceba-4b4c-4f52-a4aa-bb20547e894a" (UID: "b496ceba-4b4c-4f52-a4aa-bb20547e894a"). InnerVolumeSpecName "kube-api-access-knbjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:12:06.906514 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.906466    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-cache\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:12:06.906514 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.906506    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-kserve-provision-location\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:12:06.906514 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.906517    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tokenizer-tmp\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:12:06.906514 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.906527    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knbjl\" (UniqueName: \"kubernetes.io/projected/b496ceba-4b4c-4f52-a4aa-bb20547e894a-kube-api-access-knbjl\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:12:06.906767 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:06.906539    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b496ceba-4b4c-4f52-a4aa-bb20547e894a-tls-certs\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:12:07.074503 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:07.074464    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"]
Apr 17 08:12:07.078358 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:07.078333    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-59dbb66c9d-rpcbb"]
Apr 17 08:12:07.543185 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:07.543151    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" path="/var/lib/kubelet/pods/b496ceba-4b4c-4f52-a4aa-bb20547e894a/volumes"
Apr 17 08:12:09.519662 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:09.519634    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 08:12:09.520061 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:12:09.519779    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 08:16:07.280116 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.280082    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"]
Apr 17 08:16:07.280531 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.280372    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" containerName="tokenizer"
Apr 17 08:16:07.280531 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.280383    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" containerName="tokenizer"
Apr 17 08:16:07.280531 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.280393    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" containerName="storage-initializer"
Apr 17 08:16:07.280531 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.280400    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" containerName="storage-initializer"
Apr 17 08:16:07.280531 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.280408    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" containerName="main"
Apr 17 08:16:07.280531 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.280414    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" containerName="main"
Apr 17 08:16:07.280531 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.280490    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" containerName="tokenizer"
Apr 17 08:16:07.280531 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.280499    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="b496ceba-4b4c-4f52-a4aa-bb20547e894a" containerName="main"
Apr 17 08:16:07.283742 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.283717    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.287593 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.287569    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-f312f5ec-epp-sa-dockercfg-24zkz\""
Apr 17 08:16:07.288689 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.288667    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs\""
Apr 17 08:16:07.288815 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.288676    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jj9s6\""
Apr 17 08:16:07.288815 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.288698    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 17 08:16:07.288815 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.288698    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 17 08:16:07.296978 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.296955    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"]
Apr 17 08:16:07.316290 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.316266    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
Apr 17 08:16:07.319489 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.319474    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.321991 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.321973    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-f312f5-cb7fb8cf-dockercfg-nxt4g\""
Apr 17 08:16:07.327319 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.327300    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
Apr 17 08:16:07.358058 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.358029    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-cache\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.358058 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.358058    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqk7b\" (UniqueName: \"kubernetes.io/projected/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-kube-api-access-jqk7b\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.358227 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.358080    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-tmp\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.358227 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.358140    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-uds\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.358227 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.358168    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tls-certs\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.358343 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.358229    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-kserve-provision-location\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.459303 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.459267    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdsms\" (UniqueName: \"kubernetes.io/projected/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-kube-api-access-bdsms\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.459303 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.459309    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tls-certs\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.459567 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.459338    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.459567 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.459481    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.459567 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.459547    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.459730 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.459612    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-kserve-provision-location\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.459730 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.459653    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.459730 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.459701    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-cache\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.459881 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.459738    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jqk7b\" (UniqueName: \"kubernetes.io/projected/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-kube-api-access-jqk7b\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.459881 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.459777    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-tmp\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.459881 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.459807    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.459881 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.459858    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-uds\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.460078 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.459940    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-kserve-provision-location\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.460078 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.460028    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-cache\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.460190 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.460083    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-tmp\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.460239 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.460187    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-uds\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.461782 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.461761    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tls-certs\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.467937 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.467917    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqk7b\" (UniqueName: \"kubernetes.io/projected/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-kube-api-access-jqk7b\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.560737 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.560661    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.560737 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.560707    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.560737 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.560735    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.560964 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.560767    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.560964 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.560795    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bdsms\" (UniqueName: \"kubernetes.io/projected/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-kube-api-access-bdsms\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.560964 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.560816    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.561067 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.561042    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.561116 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.561093    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.561174 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.561157    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.562984 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.562962    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.563194 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.563177    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.568256 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.568236    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdsms\" (UniqueName: \"kubernetes.io/projected/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-kube-api-access-bdsms\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.595766 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.595739    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:07.630455 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.630408    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:16:07.725635 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.725606    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"]
Apr 17 08:16:07.727716 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:16:07.727689    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc707f0_4cc4_48a9_acd7_8f18c6592c14.slice/crio-3abdfff22468205a737ca388a6ee78dc02c9c37bb1f225c7e4b21517f6dc6bc5 WatchSource:0}: Error finding container 3abdfff22468205a737ca388a6ee78dc02c9c37bb1f225c7e4b21517f6dc6bc5: Status 404 returned error can't find the container with id 3abdfff22468205a737ca388a6ee78dc02c9c37bb1f225c7e4b21517f6dc6bc5
Apr 17 08:16:07.729867 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.729841    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 17 08:16:07.764340 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:07.764315    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
Apr 17 08:16:07.766193 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:16:07.766169    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ff5342e_9907_4c06_82b3_c2d9ef94bea7.slice/crio-654a9263671b220277e468dbbede994e8c347c9e214ba9fdea21aa45faefb42c WatchSource:0}: Error finding container 654a9263671b220277e468dbbede994e8c347c9e214ba9fdea21aa45faefb42c: Status 404 returned error can't find the container with id 654a9263671b220277e468dbbede994e8c347c9e214ba9fdea21aa45faefb42c
Apr 17 08:16:08.605921 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:08.605885    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"8ff5342e-9907-4c06-82b3-c2d9ef94bea7","Type":"ContainerStarted","Data":"5d8c5c1ff64baf15f04afd1906707a1816355484bcd08a9f034e81717eea1817"}
Apr 17 08:16:08.605921 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:08.605924    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"8ff5342e-9907-4c06-82b3-c2d9ef94bea7","Type":"ContainerStarted","Data":"654a9263671b220277e468dbbede994e8c347c9e214ba9fdea21aa45faefb42c"}
Apr 17 08:16:08.607235 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:08.607209    2571 generic.go:358] "Generic (PLEG): container finished" podID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" containerID="4c827f88363bd9e5ee5a399d23ce6dbec1d705fdb9332fe38f3631595cd5d878" exitCode=0
Apr 17 08:16:08.607385 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:08.607258    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2" event={"ID":"8bc707f0-4cc4-48a9-acd7-8f18c6592c14","Type":"ContainerDied","Data":"4c827f88363bd9e5ee5a399d23ce6dbec1d705fdb9332fe38f3631595cd5d878"}
Apr 17 08:16:08.607385 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:08.607287    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2" event={"ID":"8bc707f0-4cc4-48a9-acd7-8f18c6592c14","Type":"ContainerStarted","Data":"3abdfff22468205a737ca388a6ee78dc02c9c37bb1f225c7e4b21517f6dc6bc5"}
Apr 17 08:16:09.612574 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:09.612531    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2" event={"ID":"8bc707f0-4cc4-48a9-acd7-8f18c6592c14","Type":"ContainerStarted","Data":"a7de9c7704dc0df049b4923d69e9bbcdbab3f21d5262f4427085f14ecf164ed7"}
Apr 17 08:16:09.612918 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:09.612586    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2" event={"ID":"8bc707f0-4cc4-48a9-acd7-8f18c6592c14","Type":"ContainerStarted","Data":"1e2e8bb3af53f96b2f08c42c012b3e41a5d7e87676f9e0e444a45bc39c79c97f"}
Apr 17 08:16:09.612918 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:09.612608    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:09.636219 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:09.636150    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2" podStartSLOduration=2.636131754 podStartE2EDuration="2.636131754s" podCreationTimestamp="2026-04-17 08:16:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-17 08:16:09.631657987 +0000 UTC m=+1440.685261197" watchObservedRunningTime="2026-04-17 08:16:09.636131754 +0000 UTC m=+1440.689734966"
Apr 17 08:16:12.626597 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:12.626561    2571 generic.go:358] "Generic (PLEG): container finished" podID="8ff5342e-9907-4c06-82b3-c2d9ef94bea7" containerID="5d8c5c1ff64baf15f04afd1906707a1816355484bcd08a9f034e81717eea1817" exitCode=0
Apr 17 08:16:12.626597 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:12.626595    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"8ff5342e-9907-4c06-82b3-c2d9ef94bea7","Type":"ContainerDied","Data":"5d8c5c1ff64baf15f04afd1906707a1816355484bcd08a9f034e81717eea1817"}
Apr 17 08:16:17.596133 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:17.596096    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:17.596611 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:17.596257    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:17.598943 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:17.598922    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:17.649882 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:17.649790    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:39.657612 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:39.657582    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:16:59.812872 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:59.812834    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"8ff5342e-9907-4c06-82b3-c2d9ef94bea7","Type":"ContainerStarted","Data":"113db1c57538cb860200cedd208bc25ee0bf7add367b82b9f17b279f38d50a1b"}
Apr 17 08:16:59.835503 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:16:59.835445    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" podStartSLOduration=6.360802254 podStartE2EDuration="52.835412779s" podCreationTimestamp="2026-04-17 08:16:07 +0000 UTC" firstStartedPulling="2026-04-17 08:16:12.627803847 +0000 UTC m=+1443.681407036" lastFinishedPulling="2026-04-17 08:16:59.102414373 +0000 UTC m=+1490.156017561" observedRunningTime="2026-04-17 08:16:59.8318863 +0000 UTC m=+1490.885489512" watchObservedRunningTime="2026-04-17 08:16:59.835412779 +0000 UTC m=+1490.889015990"
Apr 17 08:17:09.552579 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:17:09.552551    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 08:17:09.553074 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:17:09.552594    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 08:18:42.948897 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:42.948846    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"]
Apr 17 08:18:42.949491 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:42.949244    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2" podUID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" containerName="main" containerID="cri-o://1e2e8bb3af53f96b2f08c42c012b3e41a5d7e87676f9e0e444a45bc39c79c97f" gracePeriod=30
Apr 17 08:18:42.949573 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:42.949479    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2" podUID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" containerName="tokenizer" containerID="cri-o://a7de9c7704dc0df049b4923d69e9bbcdbab3f21d5262f4427085f14ecf164ed7" gracePeriod=30
Apr 17 08:18:43.207074 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:43.206981    2571 generic.go:358] "Generic (PLEG): container finished" podID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" containerID="1e2e8bb3af53f96b2f08c42c012b3e41a5d7e87676f9e0e444a45bc39c79c97f" exitCode=0
Apr 17 08:18:43.207074 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:43.207057    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2" event={"ID":"8bc707f0-4cc4-48a9-acd7-8f18c6592c14","Type":"ContainerDied","Data":"1e2e8bb3af53f96b2f08c42c012b3e41a5d7e87676f9e0e444a45bc39c79c97f"}
Apr 17 08:18:44.093399 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.093378    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:18:44.161024 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.160985    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-kserve-provision-location\") pod \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") "
Apr 17 08:18:44.161192 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.161052    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-uds\") pod \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") "
Apr 17 08:18:44.161192 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.161082    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tls-certs\") pod \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") "
Apr 17 08:18:44.161192 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.161115    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-cache\") pod \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") "
Apr 17 08:18:44.161192 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.161164    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqk7b\" (UniqueName: \"kubernetes.io/projected/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-kube-api-access-jqk7b\") pod \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") "
Apr 17 08:18:44.161405 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.161213    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-tmp\") pod \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\" (UID: \"8bc707f0-4cc4-48a9-acd7-8f18c6592c14\") "
Apr 17 08:18:44.161405 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.161329    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "8bc707f0-4cc4-48a9-acd7-8f18c6592c14" (UID: "8bc707f0-4cc4-48a9-acd7-8f18c6592c14"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:18:44.161531 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.161428    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-uds\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:18:44.161585 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.161502    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "8bc707f0-4cc4-48a9-acd7-8f18c6592c14" (UID: "8bc707f0-4cc4-48a9-acd7-8f18c6592c14"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:18:44.161657 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.161636    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "8bc707f0-4cc4-48a9-acd7-8f18c6592c14" (UID: "8bc707f0-4cc4-48a9-acd7-8f18c6592c14"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:18:44.161934 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.161915    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8bc707f0-4cc4-48a9-acd7-8f18c6592c14" (UID: "8bc707f0-4cc4-48a9-acd7-8f18c6592c14"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:18:44.163128 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.163104    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8bc707f0-4cc4-48a9-acd7-8f18c6592c14" (UID: "8bc707f0-4cc4-48a9-acd7-8f18c6592c14"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 08:18:44.163214 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.163144    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-kube-api-access-jqk7b" (OuterVolumeSpecName: "kube-api-access-jqk7b") pod "8bc707f0-4cc4-48a9-acd7-8f18c6592c14" (UID: "8bc707f0-4cc4-48a9-acd7-8f18c6592c14"). InnerVolumeSpecName "kube-api-access-jqk7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:18:44.212544 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.212469    2571 generic.go:358] "Generic (PLEG): container finished" podID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" containerID="a7de9c7704dc0df049b4923d69e9bbcdbab3f21d5262f4427085f14ecf164ed7" exitCode=0
Apr 17 08:18:44.212544 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.212512    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2" event={"ID":"8bc707f0-4cc4-48a9-acd7-8f18c6592c14","Type":"ContainerDied","Data":"a7de9c7704dc0df049b4923d69e9bbcdbab3f21d5262f4427085f14ecf164ed7"}
Apr 17 08:18:44.212544 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.212535    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2" event={"ID":"8bc707f0-4cc4-48a9-acd7-8f18c6592c14","Type":"ContainerDied","Data":"3abdfff22468205a737ca388a6ee78dc02c9c37bb1f225c7e4b21517f6dc6bc5"}
Apr 17 08:18:44.212544 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.212549    2571 scope.go:117] "RemoveContainer" containerID="a7de9c7704dc0df049b4923d69e9bbcdbab3f21d5262f4427085f14ecf164ed7"
Apr 17 08:18:44.212828 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.212586    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"
Apr 17 08:18:44.221310 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.221291    2571 scope.go:117] "RemoveContainer" containerID="1e2e8bb3af53f96b2f08c42c012b3e41a5d7e87676f9e0e444a45bc39c79c97f"
Apr 17 08:18:44.228871 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.228855    2571 scope.go:117] "RemoveContainer" containerID="4c827f88363bd9e5ee5a399d23ce6dbec1d705fdb9332fe38f3631595cd5d878"
Apr 17 08:18:44.234339 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.234318    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"]
Apr 17 08:18:44.236708 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.236690    2571 scope.go:117] "RemoveContainer" containerID="a7de9c7704dc0df049b4923d69e9bbcdbab3f21d5262f4427085f14ecf164ed7"
Apr 17 08:18:44.236952 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:18:44.236934    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7de9c7704dc0df049b4923d69e9bbcdbab3f21d5262f4427085f14ecf164ed7\": container with ID starting with a7de9c7704dc0df049b4923d69e9bbcdbab3f21d5262f4427085f14ecf164ed7 not found: ID does not exist" containerID="a7de9c7704dc0df049b4923d69e9bbcdbab3f21d5262f4427085f14ecf164ed7"
Apr 17 08:18:44.237002 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.236961    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7de9c7704dc0df049b4923d69e9bbcdbab3f21d5262f4427085f14ecf164ed7"} err="failed to get container status \"a7de9c7704dc0df049b4923d69e9bbcdbab3f21d5262f4427085f14ecf164ed7\": rpc error: code = NotFound desc = could not find container \"a7de9c7704dc0df049b4923d69e9bbcdbab3f21d5262f4427085f14ecf164ed7\": container with ID starting with a7de9c7704dc0df049b4923d69e9bbcdbab3f21d5262f4427085f14ecf164ed7 not found: ID does not exist"
Apr 17 08:18:44.237002 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.236978    2571 scope.go:117] "RemoveContainer" containerID="1e2e8bb3af53f96b2f08c42c012b3e41a5d7e87676f9e0e444a45bc39c79c97f"
Apr 17 08:18:44.237223 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:18:44.237205    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e2e8bb3af53f96b2f08c42c012b3e41a5d7e87676f9e0e444a45bc39c79c97f\": container with ID starting with 1e2e8bb3af53f96b2f08c42c012b3e41a5d7e87676f9e0e444a45bc39c79c97f not found: ID does not exist" containerID="1e2e8bb3af53f96b2f08c42c012b3e41a5d7e87676f9e0e444a45bc39c79c97f"
Apr 17 08:18:44.237272 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.237228    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e2e8bb3af53f96b2f08c42c012b3e41a5d7e87676f9e0e444a45bc39c79c97f"} err="failed to get container status \"1e2e8bb3af53f96b2f08c42c012b3e41a5d7e87676f9e0e444a45bc39c79c97f\": rpc error: code = NotFound desc = could not find container \"1e2e8bb3af53f96b2f08c42c012b3e41a5d7e87676f9e0e444a45bc39c79c97f\": container with ID starting with 1e2e8bb3af53f96b2f08c42c012b3e41a5d7e87676f9e0e444a45bc39c79c97f not found: ID does not exist"
Apr 17 08:18:44.237272 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.237244    2571 scope.go:117] "RemoveContainer" containerID="4c827f88363bd9e5ee5a399d23ce6dbec1d705fdb9332fe38f3631595cd5d878"
Apr 17 08:18:44.237379 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.237359    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-sche9fkn2"]
Apr 17 08:18:44.237527 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:18:44.237507    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c827f88363bd9e5ee5a399d23ce6dbec1d705fdb9332fe38f3631595cd5d878\": container with ID starting with 4c827f88363bd9e5ee5a399d23ce6dbec1d705fdb9332fe38f3631595cd5d878 not found: ID does not exist" containerID="4c827f88363bd9e5ee5a399d23ce6dbec1d705fdb9332fe38f3631595cd5d878"
Apr 17 08:18:44.237577 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.237544    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c827f88363bd9e5ee5a399d23ce6dbec1d705fdb9332fe38f3631595cd5d878"} err="failed to get container status \"4c827f88363bd9e5ee5a399d23ce6dbec1d705fdb9332fe38f3631595cd5d878\": rpc error: code = NotFound desc = could not find container \"4c827f88363bd9e5ee5a399d23ce6dbec1d705fdb9332fe38f3631595cd5d878\": container with ID starting with 4c827f88363bd9e5ee5a399d23ce6dbec1d705fdb9332fe38f3631595cd5d878 not found: ID does not exist"
Apr 17 08:18:44.261916 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.261893    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqk7b\" (UniqueName: \"kubernetes.io/projected/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-kube-api-access-jqk7b\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:18:44.261916 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.261916    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-tmp\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:18:44.262029 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.261926    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-kserve-provision-location\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:18:44.262029 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.261935    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tls-certs\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:18:44.262029 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.261945    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bc707f0-4cc4-48a9-acd7-8f18c6592c14-tokenizer-cache\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:18:44.694397 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.694364    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
Apr 17 08:18:44.694685 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:44.694646    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" podUID="8ff5342e-9907-4c06-82b3-c2d9ef94bea7" containerName="main" containerID="cri-o://113db1c57538cb860200cedd208bc25ee0bf7add367b82b9f17b279f38d50a1b" gracePeriod=30
Apr 17 08:18:45.531045 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.531020    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:18:45.543122 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.543095    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" path="/var/lib/kubelet/pods/8bc707f0-4cc4-48a9-acd7-8f18c6592c14/volumes"
Apr 17 08:18:45.574057 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.573974    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-kserve-provision-location\") pod \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") "
Apr 17 08:18:45.574057 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.574018    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-model-cache\") pod \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") "
Apr 17 08:18:45.574057 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.574044    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-dshm\") pod \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") "
Apr 17 08:18:45.574276 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.574070    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-home\") pod \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") "
Apr 17 08:18:45.574276 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.574148    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-tls-certs\") pod \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") "
Apr 17 08:18:45.574276 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.574198    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdsms\" (UniqueName: \"kubernetes.io/projected/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-kube-api-access-bdsms\") pod \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\" (UID: \"8ff5342e-9907-4c06-82b3-c2d9ef94bea7\") "
Apr 17 08:18:45.574421 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.574329    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-model-cache" (OuterVolumeSpecName: "model-cache") pod "8ff5342e-9907-4c06-82b3-c2d9ef94bea7" (UID: "8ff5342e-9907-4c06-82b3-c2d9ef94bea7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:18:45.574517 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.574446    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-model-cache\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:18:45.574577 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.574520    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-home" (OuterVolumeSpecName: "home") pod "8ff5342e-9907-4c06-82b3-c2d9ef94bea7" (UID: "8ff5342e-9907-4c06-82b3-c2d9ef94bea7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:18:45.576331 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.576302    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8ff5342e-9907-4c06-82b3-c2d9ef94bea7" (UID: "8ff5342e-9907-4c06-82b3-c2d9ef94bea7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 08:18:45.576521 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.576494    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-dshm" (OuterVolumeSpecName: "dshm") pod "8ff5342e-9907-4c06-82b3-c2d9ef94bea7" (UID: "8ff5342e-9907-4c06-82b3-c2d9ef94bea7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:18:45.576882 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.576860    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-kube-api-access-bdsms" (OuterVolumeSpecName: "kube-api-access-bdsms") pod "8ff5342e-9907-4c06-82b3-c2d9ef94bea7" (UID: "8ff5342e-9907-4c06-82b3-c2d9ef94bea7"). InnerVolumeSpecName "kube-api-access-bdsms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:18:45.631112 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.631059    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8ff5342e-9907-4c06-82b3-c2d9ef94bea7" (UID: "8ff5342e-9907-4c06-82b3-c2d9ef94bea7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:18:45.674914 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.674876    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-kserve-provision-location\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:18:45.674914 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.674913    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-dshm\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:18:45.675077 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.674926    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-home\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:18:45.675077 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.674937    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-tls-certs\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:18:45.675077 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:45.674952    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdsms\" (UniqueName: \"kubernetes.io/projected/8ff5342e-9907-4c06-82b3-c2d9ef94bea7-kube-api-access-bdsms\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:18:46.223953 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:46.223913    2571 generic.go:358] "Generic (PLEG): container finished" podID="8ff5342e-9907-4c06-82b3-c2d9ef94bea7" containerID="113db1c57538cb860200cedd208bc25ee0bf7add367b82b9f17b279f38d50a1b" exitCode=0
Apr 17 08:18:46.224118 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:46.223985    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
Apr 17 08:18:46.224118 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:46.223999    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"8ff5342e-9907-4c06-82b3-c2d9ef94bea7","Type":"ContainerDied","Data":"113db1c57538cb860200cedd208bc25ee0bf7add367b82b9f17b279f38d50a1b"}
Apr 17 08:18:46.224118 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:46.224041    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"8ff5342e-9907-4c06-82b3-c2d9ef94bea7","Type":"ContainerDied","Data":"654a9263671b220277e468dbbede994e8c347c9e214ba9fdea21aa45faefb42c"}
Apr 17 08:18:46.224118 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:46.224056    2571 scope.go:117] "RemoveContainer" containerID="113db1c57538cb860200cedd208bc25ee0bf7add367b82b9f17b279f38d50a1b"
Apr 17 08:18:46.243375 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:46.243350    2571 scope.go:117] "RemoveContainer" containerID="5d8c5c1ff64baf15f04afd1906707a1816355484bcd08a9f034e81717eea1817"
Apr 17 08:18:46.245889 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:46.245866    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
Apr 17 08:18:46.249330 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:46.249310    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
Apr 17 08:18:46.306780 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:46.303932    2571 scope.go:117] "RemoveContainer" containerID="113db1c57538cb860200cedd208bc25ee0bf7add367b82b9f17b279f38d50a1b"
Apr 17 08:18:46.306780 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:18:46.304213    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"113db1c57538cb860200cedd208bc25ee0bf7add367b82b9f17b279f38d50a1b\": container with ID starting with 113db1c57538cb860200cedd208bc25ee0bf7add367b82b9f17b279f38d50a1b not found: ID does not exist" containerID="113db1c57538cb860200cedd208bc25ee0bf7add367b82b9f17b279f38d50a1b"
Apr 17 08:18:46.306780 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:46.304238    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113db1c57538cb860200cedd208bc25ee0bf7add367b82b9f17b279f38d50a1b"} err="failed to get container status \"113db1c57538cb860200cedd208bc25ee0bf7add367b82b9f17b279f38d50a1b\": rpc error: code = NotFound desc = could not find container \"113db1c57538cb860200cedd208bc25ee0bf7add367b82b9f17b279f38d50a1b\": container with ID starting with 113db1c57538cb860200cedd208bc25ee0bf7add367b82b9f17b279f38d50a1b not found: ID does not exist"
Apr 17 08:18:46.306780 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:46.304256    2571 scope.go:117] "RemoveContainer" containerID="5d8c5c1ff64baf15f04afd1906707a1816355484bcd08a9f034e81717eea1817"
Apr 17 08:18:46.306780 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:18:46.304478    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d8c5c1ff64baf15f04afd1906707a1816355484bcd08a9f034e81717eea1817\": container with ID starting with 5d8c5c1ff64baf15f04afd1906707a1816355484bcd08a9f034e81717eea1817 not found: ID does not exist" containerID="5d8c5c1ff64baf15f04afd1906707a1816355484bcd08a9f034e81717eea1817"
Apr 17 08:18:46.306780 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:46.304501    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d8c5c1ff64baf15f04afd1906707a1816355484bcd08a9f034e81717eea1817"} err="failed to get container status \"5d8c5c1ff64baf15f04afd1906707a1816355484bcd08a9f034e81717eea1817\": rpc error: code = NotFound desc = could not find container \"5d8c5c1ff64baf15f04afd1906707a1816355484bcd08a9f034e81717eea1817\": container with ID starting with 5d8c5c1ff64baf15f04afd1906707a1816355484bcd08a9f034e81717eea1817 not found: ID does not exist"
Apr 17 08:18:47.543640 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:47.543612    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ff5342e-9907-4c06-82b3-c2d9ef94bea7" path="/var/lib/kubelet/pods/8ff5342e-9907-4c06-82b3-c2d9ef94bea7/volumes"
Apr 17 08:18:48.599342 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599308    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"]
Apr 17 08:18:48.599726 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599615    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" containerName="tokenizer"
Apr 17 08:18:48.599726 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599626    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" containerName="tokenizer"
Apr 17 08:18:48.599726 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599638    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" containerName="storage-initializer"
Apr 17 08:18:48.599726 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599644    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" containerName="storage-initializer"
Apr 17 08:18:48.599726 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599654    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" containerName="main"
Apr 17 08:18:48.599726 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599661    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" containerName="main"
Apr 17 08:18:48.599726 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599668    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff5342e-9907-4c06-82b3-c2d9ef94bea7" containerName="main"
Apr 17 08:18:48.599726 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599673    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff5342e-9907-4c06-82b3-c2d9ef94bea7" containerName="main"
Apr 17 08:18:48.599726 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599683    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff5342e-9907-4c06-82b3-c2d9ef94bea7" containerName="storage-initializer"
Apr 17 08:18:48.599726 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599688    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff5342e-9907-4c06-82b3-c2d9ef94bea7" containerName="storage-initializer"
Apr 17 08:18:48.600252 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599790    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" containerName="tokenizer"
Apr 17 08:18:48.600252 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599803    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="8ff5342e-9907-4c06-82b3-c2d9ef94bea7" containerName="main"
Apr 17 08:18:48.600252 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.599816    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="8bc707f0-4cc4-48a9-acd7-8f18c6592c14" containerName="main"
Apr 17 08:18:48.603164 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.603143    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.605729 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.605703    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 17 08:18:48.607172 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.606770    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jj9s6\""
Apr 17 08:18:48.607172 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.606802    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Apr 17 08:18:48.607172 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.607067    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 17 08:18:48.614620 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.614598    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"]
Apr 17 08:18:48.699055 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.699017    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-home\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.699231 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.699091    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.699231 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.699123    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-model-cache\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.699231 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.699153    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rx6s\" (UniqueName: \"kubernetes.io/projected/5e179593-72e6-491d-8ab3-2a57d8ef6090-kube-api-access-7rx6s\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.699231 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.699180    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e179593-72e6-491d-8ab3-2a57d8ef6090-tls-certs\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.699231 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.699204    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-dshm\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.799696 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.799657    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7rx6s\" (UniqueName: \"kubernetes.io/projected/5e179593-72e6-491d-8ab3-2a57d8ef6090-kube-api-access-7rx6s\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.799826 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.799699    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e179593-72e6-491d-8ab3-2a57d8ef6090-tls-certs\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.799907 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.799817    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-dshm\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.799907 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.799876    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-home\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.800009 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.799975    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.800073 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.800019    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-model-cache\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.800286 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.800266    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-home\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.800376 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.800290    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.800376 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.800344    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-model-cache\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.802027 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.802001    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-dshm\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.802187 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.802172    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e179593-72e6-491d-8ab3-2a57d8ef6090-tls-certs\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.811991 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.811972    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rx6s\" (UniqueName: \"kubernetes.io/projected/5e179593-72e6-491d-8ab3-2a57d8ef6090-kube-api-access-7rx6s\") pod \"scheduler-inline-config-test-kserve-5b774f6465-lfm4d\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:48.833143 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.833114    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"]
Apr 17 08:18:48.838406 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.838380    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:48.840904 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.840874    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-epp-sa-dockercfg-t8fw4\""
Apr 17 08:18:48.846687 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.846665    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"]
Apr 17 08:18:48.900966 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.900886    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-cache\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:48.900966 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.900929    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgrs7\" (UniqueName: \"kubernetes.io/projected/88a932e3-2abd-450d-a94b-539737ffb84d-kube-api-access-hgrs7\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:48.900966 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.900952    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-tmp\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:48.901170 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.901015    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a932e3-2abd-450d-a94b-539737ffb84d-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:48.901170 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.901032    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:48.901170 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.901058    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:48.914868 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:48.914840    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:49.001945 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.001909    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-cache\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:49.002089 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.001953    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hgrs7\" (UniqueName: \"kubernetes.io/projected/88a932e3-2abd-450d-a94b-539737ffb84d-kube-api-access-hgrs7\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:49.002089 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.001988    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-tmp\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:49.002089 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.002030    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a932e3-2abd-450d-a94b-539737ffb84d-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:49.002089 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.002053    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:49.002089 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.002084    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:49.002397 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.002374    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-cache\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:49.002480 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.002407    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-tmp\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:49.002480 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.002459    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:49.002672 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.002652    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:49.004821 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.004800    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a932e3-2abd-450d-a94b-539737ffb84d-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:49.010081 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.010025    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgrs7\" (UniqueName: \"kubernetes.io/projected/88a932e3-2abd-450d-a94b-539737ffb84d-kube-api-access-hgrs7\") pod \"scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:49.032467 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.032446    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"]
Apr 17 08:18:49.148612 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.148576    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:49.237023 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.236989    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d" event={"ID":"5e179593-72e6-491d-8ab3-2a57d8ef6090","Type":"ContainerStarted","Data":"677045d37eacf553449696ee54157434b5721f0ec5bfb016e9b04e6b871195ef"}
Apr 17 08:18:49.237175 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.237030    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d" event={"ID":"5e179593-72e6-491d-8ab3-2a57d8ef6090","Type":"ContainerStarted","Data":"7a793dc9fd8b319129958b173d7240add619bdd3708a3139ee1aa2c4ab6cd477"}
Apr 17 08:18:49.277479 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:49.277450    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"]
Apr 17 08:18:49.279069 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:18:49.279039    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88a932e3_2abd_450d_a94b_539737ffb84d.slice/crio-afdeb8495497ae4adf6e6717e2a03e029bfa1af68838f5e00b1192b315521aa2 WatchSource:0}: Error finding container afdeb8495497ae4adf6e6717e2a03e029bfa1af68838f5e00b1192b315521aa2: Status 404 returned error can't find the container with id afdeb8495497ae4adf6e6717e2a03e029bfa1af68838f5e00b1192b315521aa2
Apr 17 08:18:50.242498 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:50.242406    2571 generic.go:358] "Generic (PLEG): container finished" podID="88a932e3-2abd-450d-a94b-539737ffb84d" containerID="420424a70b15693af4bc699506f9d555f74906d45912025c143b8653b98eab82" exitCode=0
Apr 17 08:18:50.242855 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:50.242494    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp" event={"ID":"88a932e3-2abd-450d-a94b-539737ffb84d","Type":"ContainerDied","Data":"420424a70b15693af4bc699506f9d555f74906d45912025c143b8653b98eab82"}
Apr 17 08:18:50.242855 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:50.242542    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp" event={"ID":"88a932e3-2abd-450d-a94b-539737ffb84d","Type":"ContainerStarted","Data":"afdeb8495497ae4adf6e6717e2a03e029bfa1af68838f5e00b1192b315521aa2"}
Apr 17 08:18:51.258064 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:51.258029    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp" event={"ID":"88a932e3-2abd-450d-a94b-539737ffb84d","Type":"ContainerStarted","Data":"cf413527fa25beb5a3ab163f4209159b3e2be5a5c31b23ae8101949ba8545f8b"}
Apr 17 08:18:51.258064 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:51.258065    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp" event={"ID":"88a932e3-2abd-450d-a94b-539737ffb84d","Type":"ContainerStarted","Data":"d4770bd1ff6cabfcdc97401cf1bbf3bba78006a452c9bddc8ba1dff939eed2a0"}
Apr 17 08:18:51.258497 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:51.258155    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:51.279572 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:51.279509    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp" podStartSLOduration=3.279487736 podStartE2EDuration="3.279487736s" podCreationTimestamp="2026-04-17 08:18:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-17 08:18:51.276875096 +0000 UTC m=+1602.330478318" watchObservedRunningTime="2026-04-17 08:18:51.279487736 +0000 UTC m=+1602.333090971"
Apr 17 08:18:53.267906 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:53.267876    2571 generic.go:358] "Generic (PLEG): container finished" podID="5e179593-72e6-491d-8ab3-2a57d8ef6090" containerID="677045d37eacf553449696ee54157434b5721f0ec5bfb016e9b04e6b871195ef" exitCode=0
Apr 17 08:18:53.268296 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:53.267952    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d" event={"ID":"5e179593-72e6-491d-8ab3-2a57d8ef6090","Type":"ContainerDied","Data":"677045d37eacf553449696ee54157434b5721f0ec5bfb016e9b04e6b871195ef"}
Apr 17 08:18:54.273549 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:54.273507    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d" event={"ID":"5e179593-72e6-491d-8ab3-2a57d8ef6090","Type":"ContainerStarted","Data":"5c343819a13ed8d055037884a24fd60275c9b1e77a186656c13e386bbbf509de"}
Apr 17 08:18:54.291396 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:54.291344    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d" podStartSLOduration=6.291330568 podStartE2EDuration="6.291330568s" podCreationTimestamp="2026-04-17 08:18:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-17 08:18:54.290464141 +0000 UTC m=+1605.344067352" watchObservedRunningTime="2026-04-17 08:18:54.291330568 +0000 UTC m=+1605.344933781"
Apr 17 08:18:58.915489 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:58.915390    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:58.915489 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:58.915454    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:58.927772 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:58.927747    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:18:59.149578 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:59.149539    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:59.149578 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:59.149580    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:59.152252 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:59.152227    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:59.292287 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:59.292252    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:18:59.302255 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:18:59.302236    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:19:20.296457 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:20.296405    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:19:21.634196 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.634162    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"]
Apr 17 08:19:21.634695 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.634477    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d" podUID="5e179593-72e6-491d-8ab3-2a57d8ef6090" containerName="main" containerID="cri-o://5c343819a13ed8d055037884a24fd60275c9b1e77a186656c13e386bbbf509de" gracePeriod=30
Apr 17 08:19:21.636278 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.636253    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"]
Apr 17 08:19:21.636600 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.636574    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp" podUID="88a932e3-2abd-450d-a94b-539737ffb84d" containerName="main" containerID="cri-o://d4770bd1ff6cabfcdc97401cf1bbf3bba78006a452c9bddc8ba1dff939eed2a0" gracePeriod=30
Apr 17 08:19:21.636694 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.636597    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp" podUID="88a932e3-2abd-450d-a94b-539737ffb84d" containerName="tokenizer" containerID="cri-o://cf413527fa25beb5a3ab163f4209159b3e2be5a5c31b23ae8101949ba8545f8b" gracePeriod=30
Apr 17 08:19:21.885501 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.885446    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:19:21.982808 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.982768    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-kserve-provision-location\") pod \"5e179593-72e6-491d-8ab3-2a57d8ef6090\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") "
Apr 17 08:19:21.983006 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.982816    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rx6s\" (UniqueName: \"kubernetes.io/projected/5e179593-72e6-491d-8ab3-2a57d8ef6090-kube-api-access-7rx6s\") pod \"5e179593-72e6-491d-8ab3-2a57d8ef6090\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") "
Apr 17 08:19:21.983006 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.982848    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-model-cache\") pod \"5e179593-72e6-491d-8ab3-2a57d8ef6090\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") "
Apr 17 08:19:21.983006 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.982905    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-dshm\") pod \"5e179593-72e6-491d-8ab3-2a57d8ef6090\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") "
Apr 17 08:19:21.983189 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.983060    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e179593-72e6-491d-8ab3-2a57d8ef6090-tls-certs\") pod \"5e179593-72e6-491d-8ab3-2a57d8ef6090\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") "
Apr 17 08:19:21.983189 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.983104    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-model-cache" (OuterVolumeSpecName: "model-cache") pod "5e179593-72e6-491d-8ab3-2a57d8ef6090" (UID: "5e179593-72e6-491d-8ab3-2a57d8ef6090"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:19:21.983189 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.983139    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-home\") pod \"5e179593-72e6-491d-8ab3-2a57d8ef6090\" (UID: \"5e179593-72e6-491d-8ab3-2a57d8ef6090\") "
Apr 17 08:19:21.983378 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.983349    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-home" (OuterVolumeSpecName: "home") pod "5e179593-72e6-491d-8ab3-2a57d8ef6090" (UID: "5e179593-72e6-491d-8ab3-2a57d8ef6090"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:19:21.983474 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.983375    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-model-cache\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:19:21.985007 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.984980    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e179593-72e6-491d-8ab3-2a57d8ef6090-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e179593-72e6-491d-8ab3-2a57d8ef6090" (UID: "5e179593-72e6-491d-8ab3-2a57d8ef6090"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 08:19:21.985144 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.985104    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e179593-72e6-491d-8ab3-2a57d8ef6090-kube-api-access-7rx6s" (OuterVolumeSpecName: "kube-api-access-7rx6s") pod "5e179593-72e6-491d-8ab3-2a57d8ef6090" (UID: "5e179593-72e6-491d-8ab3-2a57d8ef6090"). InnerVolumeSpecName "kube-api-access-7rx6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:19:21.985234 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:21.985220    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-dshm" (OuterVolumeSpecName: "dshm") pod "5e179593-72e6-491d-8ab3-2a57d8ef6090" (UID: "5e179593-72e6-491d-8ab3-2a57d8ef6090"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:19:22.040816 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.040751    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "5e179593-72e6-491d-8ab3-2a57d8ef6090" (UID: "5e179593-72e6-491d-8ab3-2a57d8ef6090"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:19:22.083950 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.083911    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-dshm\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:19:22.083950 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.083943    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e179593-72e6-491d-8ab3-2a57d8ef6090-tls-certs\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:19:22.083950 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.083959    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-home\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:19:22.084184 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.083972    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5e179593-72e6-491d-8ab3-2a57d8ef6090-kserve-provision-location\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:19:22.084184 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.083985    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rx6s\" (UniqueName: \"kubernetes.io/projected/5e179593-72e6-491d-8ab3-2a57d8ef6090-kube-api-access-7rx6s\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:19:22.374979 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.374938    2571 generic.go:358] "Generic (PLEG): container finished" podID="88a932e3-2abd-450d-a94b-539737ffb84d" containerID="d4770bd1ff6cabfcdc97401cf1bbf3bba78006a452c9bddc8ba1dff939eed2a0" exitCode=0
Apr 17 08:19:22.375154 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.375004    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp" event={"ID":"88a932e3-2abd-450d-a94b-539737ffb84d","Type":"ContainerDied","Data":"d4770bd1ff6cabfcdc97401cf1bbf3bba78006a452c9bddc8ba1dff939eed2a0"}
Apr 17 08:19:22.376477 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.376452    2571 generic.go:358] "Generic (PLEG): container finished" podID="5e179593-72e6-491d-8ab3-2a57d8ef6090" containerID="5c343819a13ed8d055037884a24fd60275c9b1e77a186656c13e386bbbf509de" exitCode=0
Apr 17 08:19:22.376602 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.376521    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d" event={"ID":"5e179593-72e6-491d-8ab3-2a57d8ef6090","Type":"ContainerDied","Data":"5c343819a13ed8d055037884a24fd60275c9b1e77a186656c13e386bbbf509de"}
Apr 17 08:19:22.376602 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.376531    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"
Apr 17 08:19:22.376602 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.376544    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d" event={"ID":"5e179593-72e6-491d-8ab3-2a57d8ef6090","Type":"ContainerDied","Data":"7a793dc9fd8b319129958b173d7240add619bdd3708a3139ee1aa2c4ab6cd477"}
Apr 17 08:19:22.376602 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.376560    2571 scope.go:117] "RemoveContainer" containerID="5c343819a13ed8d055037884a24fd60275c9b1e77a186656c13e386bbbf509de"
Apr 17 08:19:22.386858 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.386841    2571 scope.go:117] "RemoveContainer" containerID="677045d37eacf553449696ee54157434b5721f0ec5bfb016e9b04e6b871195ef"
Apr 17 08:19:22.399986 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.399958    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"]
Apr 17 08:19:22.401264 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.401244    2571 scope.go:117] "RemoveContainer" containerID="5c343819a13ed8d055037884a24fd60275c9b1e77a186656c13e386bbbf509de"
Apr 17 08:19:22.401827 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:19:22.401803    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c343819a13ed8d055037884a24fd60275c9b1e77a186656c13e386bbbf509de\": container with ID starting with 5c343819a13ed8d055037884a24fd60275c9b1e77a186656c13e386bbbf509de not found: ID does not exist" containerID="5c343819a13ed8d055037884a24fd60275c9b1e77a186656c13e386bbbf509de"
Apr 17 08:19:22.401938 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.401833    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c343819a13ed8d055037884a24fd60275c9b1e77a186656c13e386bbbf509de"} err="failed to get container status \"5c343819a13ed8d055037884a24fd60275c9b1e77a186656c13e386bbbf509de\": rpc error: code = NotFound desc = could not find container \"5c343819a13ed8d055037884a24fd60275c9b1e77a186656c13e386bbbf509de\": container with ID starting with 5c343819a13ed8d055037884a24fd60275c9b1e77a186656c13e386bbbf509de not found: ID does not exist"
Apr 17 08:19:22.401938 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.401852    2571 scope.go:117] "RemoveContainer" containerID="677045d37eacf553449696ee54157434b5721f0ec5bfb016e9b04e6b871195ef"
Apr 17 08:19:22.402168 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:19:22.402134    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"677045d37eacf553449696ee54157434b5721f0ec5bfb016e9b04e6b871195ef\": container with ID starting with 677045d37eacf553449696ee54157434b5721f0ec5bfb016e9b04e6b871195ef not found: ID does not exist" containerID="677045d37eacf553449696ee54157434b5721f0ec5bfb016e9b04e6b871195ef"
Apr 17 08:19:22.402231 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.402177    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"677045d37eacf553449696ee54157434b5721f0ec5bfb016e9b04e6b871195ef"} err="failed to get container status \"677045d37eacf553449696ee54157434b5721f0ec5bfb016e9b04e6b871195ef\": rpc error: code = NotFound desc = could not find container \"677045d37eacf553449696ee54157434b5721f0ec5bfb016e9b04e6b871195ef\": container with ID starting with 677045d37eacf553449696ee54157434b5721f0ec5bfb016e9b04e6b871195ef not found: ID does not exist"
Apr 17 08:19:22.403766 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.403745    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5b774f6465-lfm4d"]
Apr 17 08:19:22.886348 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.886324    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:19:22.990400 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.990327    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgrs7\" (UniqueName: \"kubernetes.io/projected/88a932e3-2abd-450d-a94b-539737ffb84d-kube-api-access-hgrs7\") pod \"88a932e3-2abd-450d-a94b-539737ffb84d\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") "
Apr 17 08:19:22.990586 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.990403    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-cache\") pod \"88a932e3-2abd-450d-a94b-539737ffb84d\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") "
Apr 17 08:19:22.990586 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.990445    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a932e3-2abd-450d-a94b-539737ffb84d-tls-certs\") pod \"88a932e3-2abd-450d-a94b-539737ffb84d\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") "
Apr 17 08:19:22.990586 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.990467    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-uds\") pod \"88a932e3-2abd-450d-a94b-539737ffb84d\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") "
Apr 17 08:19:22.990586 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.990485    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-kserve-provision-location\") pod \"88a932e3-2abd-450d-a94b-539737ffb84d\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") "
Apr 17 08:19:22.990586 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.990515    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-tmp\") pod \"88a932e3-2abd-450d-a94b-539737ffb84d\" (UID: \"88a932e3-2abd-450d-a94b-539737ffb84d\") "
Apr 17 08:19:22.990872 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.990656    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "88a932e3-2abd-450d-a94b-539737ffb84d" (UID: "88a932e3-2abd-450d-a94b-539737ffb84d"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:19:22.990872 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.990749    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "88a932e3-2abd-450d-a94b-539737ffb84d" (UID: "88a932e3-2abd-450d-a94b-539737ffb84d"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:19:22.991028 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.990882    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "88a932e3-2abd-450d-a94b-539737ffb84d" (UID: "88a932e3-2abd-450d-a94b-539737ffb84d"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:19:22.991197 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.991174    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "88a932e3-2abd-450d-a94b-539737ffb84d" (UID: "88a932e3-2abd-450d-a94b-539737ffb84d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:19:22.992402 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.992373    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88a932e3-2abd-450d-a94b-539737ffb84d-kube-api-access-hgrs7" (OuterVolumeSpecName: "kube-api-access-hgrs7") pod "88a932e3-2abd-450d-a94b-539737ffb84d" (UID: "88a932e3-2abd-450d-a94b-539737ffb84d"). InnerVolumeSpecName "kube-api-access-hgrs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:19:22.992548 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:22.992483    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88a932e3-2abd-450d-a94b-539737ffb84d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "88a932e3-2abd-450d-a94b-539737ffb84d" (UID: "88a932e3-2abd-450d-a94b-539737ffb84d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 08:19:23.091746 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.091714    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-cache\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:19:23.091746 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.091738    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a932e3-2abd-450d-a94b-539737ffb84d-tls-certs\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:19:23.091746 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.091749    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-uds\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:19:23.091942 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.091758    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-kserve-provision-location\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:19:23.091942 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.091767    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/88a932e3-2abd-450d-a94b-539737ffb84d-tokenizer-tmp\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:19:23.091942 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.091778    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgrs7\" (UniqueName: \"kubernetes.io/projected/88a932e3-2abd-450d-a94b-539737ffb84d-kube-api-access-hgrs7\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:19:23.382520 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.382479    2571 generic.go:358] "Generic (PLEG): container finished" podID="88a932e3-2abd-450d-a94b-539737ffb84d" containerID="cf413527fa25beb5a3ab163f4209159b3e2be5a5c31b23ae8101949ba8545f8b" exitCode=0
Apr 17 08:19:23.382667 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.382561    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"
Apr 17 08:19:23.382667 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.382565    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp" event={"ID":"88a932e3-2abd-450d-a94b-539737ffb84d","Type":"ContainerDied","Data":"cf413527fa25beb5a3ab163f4209159b3e2be5a5c31b23ae8101949ba8545f8b"}
Apr 17 08:19:23.382667 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.382601    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp" event={"ID":"88a932e3-2abd-450d-a94b-539737ffb84d","Type":"ContainerDied","Data":"afdeb8495497ae4adf6e6717e2a03e029bfa1af68838f5e00b1192b315521aa2"}
Apr 17 08:19:23.382667 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.382617    2571 scope.go:117] "RemoveContainer" containerID="cf413527fa25beb5a3ab163f4209159b3e2be5a5c31b23ae8101949ba8545f8b"
Apr 17 08:19:23.391366 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.391349    2571 scope.go:117] "RemoveContainer" containerID="d4770bd1ff6cabfcdc97401cf1bbf3bba78006a452c9bddc8ba1dff939eed2a0"
Apr 17 08:19:23.398447 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.398411    2571 scope.go:117] "RemoveContainer" containerID="420424a70b15693af4bc699506f9d555f74906d45912025c143b8653b98eab82"
Apr 17 08:19:23.404813 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.404790    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"]
Apr 17 08:19:23.406023 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.406010    2571 scope.go:117] "RemoveContainer" containerID="cf413527fa25beb5a3ab163f4209159b3e2be5a5c31b23ae8101949ba8545f8b"
Apr 17 08:19:23.406267 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:19:23.406238    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf413527fa25beb5a3ab163f4209159b3e2be5a5c31b23ae8101949ba8545f8b\": container with ID starting with cf413527fa25beb5a3ab163f4209159b3e2be5a5c31b23ae8101949ba8545f8b not found: ID does not exist" containerID="cf413527fa25beb5a3ab163f4209159b3e2be5a5c31b23ae8101949ba8545f8b"
Apr 17 08:19:23.406344 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.406274    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf413527fa25beb5a3ab163f4209159b3e2be5a5c31b23ae8101949ba8545f8b"} err="failed to get container status \"cf413527fa25beb5a3ab163f4209159b3e2be5a5c31b23ae8101949ba8545f8b\": rpc error: code = NotFound desc = could not find container \"cf413527fa25beb5a3ab163f4209159b3e2be5a5c31b23ae8101949ba8545f8b\": container with ID starting with cf413527fa25beb5a3ab163f4209159b3e2be5a5c31b23ae8101949ba8545f8b not found: ID does not exist"
Apr 17 08:19:23.406344 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.406290    2571 scope.go:117] "RemoveContainer" containerID="d4770bd1ff6cabfcdc97401cf1bbf3bba78006a452c9bddc8ba1dff939eed2a0"
Apr 17 08:19:23.406564 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:19:23.406540    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4770bd1ff6cabfcdc97401cf1bbf3bba78006a452c9bddc8ba1dff939eed2a0\": container with ID starting with d4770bd1ff6cabfcdc97401cf1bbf3bba78006a452c9bddc8ba1dff939eed2a0 not found: ID does not exist" containerID="d4770bd1ff6cabfcdc97401cf1bbf3bba78006a452c9bddc8ba1dff939eed2a0"
Apr 17 08:19:23.406695 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.406569    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4770bd1ff6cabfcdc97401cf1bbf3bba78006a452c9bddc8ba1dff939eed2a0"} err="failed to get container status \"d4770bd1ff6cabfcdc97401cf1bbf3bba78006a452c9bddc8ba1dff939eed2a0\": rpc error: code = NotFound desc = could not find container \"d4770bd1ff6cabfcdc97401cf1bbf3bba78006a452c9bddc8ba1dff939eed2a0\": container with ID starting with d4770bd1ff6cabfcdc97401cf1bbf3bba78006a452c9bddc8ba1dff939eed2a0 not found: ID does not exist"
Apr 17 08:19:23.406695 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.406587    2571 scope.go:117] "RemoveContainer" containerID="420424a70b15693af4bc699506f9d555f74906d45912025c143b8653b98eab82"
Apr 17 08:19:23.407125 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:19:23.407095    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"420424a70b15693af4bc699506f9d555f74906d45912025c143b8653b98eab82\": container with ID starting with 420424a70b15693af4bc699506f9d555f74906d45912025c143b8653b98eab82 not found: ID does not exist" containerID="420424a70b15693af4bc699506f9d555f74906d45912025c143b8653b98eab82"
Apr 17 08:19:23.407209 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.407131    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420424a70b15693af4bc699506f9d555f74906d45912025c143b8653b98eab82"} err="failed to get container status \"420424a70b15693af4bc699506f9d555f74906d45912025c143b8653b98eab82\": rpc error: code = NotFound desc = could not find container \"420424a70b15693af4bc699506f9d555f74906d45912025c143b8653b98eab82\": container with ID starting with 420424a70b15693af4bc699506f9d555f74906d45912025c143b8653b98eab82 not found: ID does not exist"
Apr 17 08:19:23.409169 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.409148    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-549c8nk5vp"]
Apr 17 08:19:23.542998 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.542969    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e179593-72e6-491d-8ab3-2a57d8ef6090" path="/var/lib/kubelet/pods/5e179593-72e6-491d-8ab3-2a57d8ef6090/volumes"
Apr 17 08:19:23.543379 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:23.543366    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88a932e3-2abd-450d-a94b-539737ffb84d" path="/var/lib/kubelet/pods/88a932e3-2abd-450d-a94b-539737ffb84d/volumes"
Apr 17 08:19:27.229401 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229365    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"]
Apr 17 08:19:27.229825 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229684    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e179593-72e6-491d-8ab3-2a57d8ef6090" containerName="storage-initializer"
Apr 17 08:19:27.229825 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229697    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e179593-72e6-491d-8ab3-2a57d8ef6090" containerName="storage-initializer"
Apr 17 08:19:27.229825 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229705    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a932e3-2abd-450d-a94b-539737ffb84d" containerName="tokenizer"
Apr 17 08:19:27.229825 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229711    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a932e3-2abd-450d-a94b-539737ffb84d" containerName="tokenizer"
Apr 17 08:19:27.229825 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229724    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a932e3-2abd-450d-a94b-539737ffb84d" containerName="storage-initializer"
Apr 17 08:19:27.229825 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229730    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a932e3-2abd-450d-a94b-539737ffb84d" containerName="storage-initializer"
Apr 17 08:19:27.229825 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229738    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a932e3-2abd-450d-a94b-539737ffb84d" containerName="main"
Apr 17 08:19:27.229825 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229747    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a932e3-2abd-450d-a94b-539737ffb84d" containerName="main"
Apr 17 08:19:27.229825 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229765    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e179593-72e6-491d-8ab3-2a57d8ef6090" containerName="main"
Apr 17 08:19:27.229825 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229770    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e179593-72e6-491d-8ab3-2a57d8ef6090" containerName="main"
Apr 17 08:19:27.230176 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229855    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="88a932e3-2abd-450d-a94b-539737ffb84d" containerName="tokenizer"
Apr 17 08:19:27.230176 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229863    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="88a932e3-2abd-450d-a94b-539737ffb84d" containerName="main"
Apr 17 08:19:27.230176 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.229872    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="5e179593-72e6-491d-8ab3-2a57d8ef6090" containerName="main"
Apr 17 08:19:27.234734 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.234709    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.237135 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.237097    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-2-openshift-default-dockercfg-vxdsx\""
Apr 17 08:19:27.237135 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.237125    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"istio-ca-root-cert\""
Apr 17 08:19:27.237413 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.237398    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Apr 17 08:19:27.237501 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.237447    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Apr 17 08:19:27.246651 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.246631    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"]
Apr 17 08:19:27.324978 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.324949    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istio-token\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.325115 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.325003    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istio-podinfo\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.325115 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.325084    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istio-envoy\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.325212 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.325123    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istio-data\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.325212 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.325142    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-workload-certs\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.325212 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.325187    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-workload-socket\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.325331 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.325249    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.325331 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.325310    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-credential-socket\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.325411 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.325332    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnkk4\" (UniqueName: \"kubernetes.io/projected/cec87044-ce22-4b2a-af90-ffdfcb72ae20-kube-api-access-vnkk4\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.425656 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.425612    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-workload-socket\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.425656 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.425659    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.425903 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.425680    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-credential-socket\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.425903 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.425707    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vnkk4\" (UniqueName: \"kubernetes.io/projected/cec87044-ce22-4b2a-af90-ffdfcb72ae20-kube-api-access-vnkk4\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.425903 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.425729    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istio-token\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.425903 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.425759    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istio-podinfo\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.425903 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.425792    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istio-envoy\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.425903 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.425830    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istio-data\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.425903 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.425856    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-workload-certs\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.426498 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.426046    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-workload-socket\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.426498 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.426209    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-workload-certs\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.426498 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.426235    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-credential-socket\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.426498 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.426255    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istio-data\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.426498 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.426383    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.428037 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.428018    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istio-envoy\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.428160 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.428141    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istio-podinfo\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.433399 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.433371    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cec87044-ce22-4b2a-af90-ffdfcb72ae20-istio-token\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.433525 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.433476    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnkk4\" (UniqueName: \"kubernetes.io/projected/cec87044-ce22-4b2a-af90-ffdfcb72ae20-kube-api-access-vnkk4\") pod \"router-gateway-2-openshift-default-6866b85949-89mvx\" (UID: \"cec87044-ce22-4b2a-af90-ffdfcb72ae20\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.548233 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.548209    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:27.671674 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:27.671642    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"]
Apr 17 08:19:27.672870 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:19:27.672849    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcec87044_ce22_4b2a_af90_ffdfcb72ae20.slice/crio-3e3006b25ff0c8c5eaf17cc0bfbdb7f703dfe7f9fdb1668df07a246a26529e49 WatchSource:0}: Error finding container 3e3006b25ff0c8c5eaf17cc0bfbdb7f703dfe7f9fdb1668df07a246a26529e49: Status 404 returned error can't find the container with id 3e3006b25ff0c8c5eaf17cc0bfbdb7f703dfe7f9fdb1668df07a246a26529e49
Apr 17 08:19:28.403718 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:28.403678    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx" event={"ID":"cec87044-ce22-4b2a-af90-ffdfcb72ae20","Type":"ContainerStarted","Data":"3e3006b25ff0c8c5eaf17cc0bfbdb7f703dfe7f9fdb1668df07a246a26529e49"}
Apr 17 08:19:30.166156 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:30.166118    2571 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Apr 17 08:19:30.166450 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:30.166185    2571 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Apr 17 08:19:30.166450 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:30.166213    2571 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Apr 17 08:19:30.413480 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:30.413442    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx" event={"ID":"cec87044-ce22-4b2a-af90-ffdfcb72ae20","Type":"ContainerStarted","Data":"5f9def88edfccbb2efa03276c8448ab14292e4ffa8e43a9823d6b26f9ecd57c9"}
Apr 17 08:19:30.433837 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:30.433747    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx" podStartSLOduration=0.942599082 podStartE2EDuration="3.433733311s" podCreationTimestamp="2026-04-17 08:19:27 +0000 UTC" firstStartedPulling="2026-04-17 08:19:27.674786404 +0000 UTC m=+1638.728389594" lastFinishedPulling="2026-04-17 08:19:30.165920629 +0000 UTC m=+1641.219523823" observedRunningTime="2026-04-17 08:19:30.43281258 +0000 UTC m=+1641.486415801" watchObservedRunningTime="2026-04-17 08:19:30.433733311 +0000 UTC m=+1641.487336521"
Apr 17 08:19:30.548945 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:30.548912    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:31.550476 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:31.550418    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx" podUID="cec87044-ce22-4b2a-af90-ffdfcb72ae20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Apr 17 08:19:32.549322 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:32.549282    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx" podUID="cec87044-ce22-4b2a-af90-ffdfcb72ae20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Apr 17 08:19:33.553235 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:33.553206    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:33.553596 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:33.553463    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:33.554174 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:33.554157    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-89mvx"
Apr 17 08:19:37.314292 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.314261    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"]
Apr 17 08:19:37.317531 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.317515    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.319914 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.319875    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-kserve-dockercfg-f498q\""
Apr 17 08:19:37.320043 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.319993    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jj9s6\""
Apr 17 08:19:37.320043 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.320031    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Apr 17 08:19:37.329404 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.329381    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"]
Apr 17 08:19:37.344776 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.344756    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"]
Apr 17 08:19:37.348094 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.348080    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.359262 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.359241    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"]
Apr 17 08:19:37.406529 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.406496    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-home\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.406529 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.406532    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-model-cache\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.406762 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.406565    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.406762 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.406618    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64c12487-7d51-4f9f-a774-11145c3d38d7-tls-certs\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.406762 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.406660    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcctc\" (UniqueName: \"kubernetes.io/projected/64c12487-7d51-4f9f-a774-11145c3d38d7-kube-api-access-kcctc\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.406762 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.406689    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-dshm\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.406762 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.406751    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-tls-certs\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.406956 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.406787    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-model-cache\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.406956 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.406809    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qkzg\" (UniqueName: \"kubernetes.io/projected/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-kube-api-access-5qkzg\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.406956 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.406827    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.406956 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.406848    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-home\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.406956 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.406864    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-dshm\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.507758 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.507723    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5qkzg\" (UniqueName: \"kubernetes.io/projected/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-kube-api-access-5qkzg\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.507758 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.507758    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.508051 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.507789    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-home\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.508051 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.507903    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-dshm\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.508051 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.507971    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-home\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.508051 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.507997    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-model-cache\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.508051 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.508040    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.508321 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.508070    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64c12487-7d51-4f9f-a774-11145c3d38d7-tls-certs\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.508321 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.508101    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kcctc\" (UniqueName: \"kubernetes.io/projected/64c12487-7d51-4f9f-a774-11145c3d38d7-kube-api-access-kcctc\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.508321 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.508127    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-dshm\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.508321 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.508163    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-tls-certs\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.508321 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.508199    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-model-cache\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.508321 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.508219    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-home\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.508321 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.508255    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.508696 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.508348    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-model-cache\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.508696 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.508421    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-home\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.508696 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.508490    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.508696 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.508569    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-model-cache\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.510444 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.510410    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-dshm\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.510717 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.510697    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-dshm\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.510827 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.510805    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-tls-certs\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.510917 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.510903    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64c12487-7d51-4f9f-a774-11145c3d38d7-tls-certs\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.515355 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.515331    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qkzg\" (UniqueName: \"kubernetes.io/projected/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-kube-api-access-5qkzg\") pod \"router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.515592 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.515571    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcctc\" (UniqueName: \"kubernetes.io/projected/64c12487-7d51-4f9f-a774-11145c3d38d7-kube-api-access-kcctc\") pod \"router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.626035 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.625950    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:37.659558 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.659527    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:37.768193 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.767932    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"]
Apr 17 08:19:37.770168 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:19:37.770137    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c12487_7d51_4f9f_a774_11145c3d38d7.slice/crio-fd144e7c744560b8360306c9814fd2b21d779f4395cbbafecfb2d071bdca9332 WatchSource:0}: Error finding container fd144e7c744560b8360306c9814fd2b21d779f4395cbbafecfb2d071bdca9332: Status 404 returned error can't find the container with id fd144e7c744560b8360306c9814fd2b21d779f4395cbbafecfb2d071bdca9332
Apr 17 08:19:37.802168 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:37.802143    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"]
Apr 17 08:19:37.804053 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:19:37.804026    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a9b0f3_339c_4bfa_a5a7_e556c6965da2.slice/crio-2e47abccafe2f1a62e1b9196f0bc03b86f884fe656371ca931ba5c6e0adb3f0a WatchSource:0}: Error finding container 2e47abccafe2f1a62e1b9196f0bc03b86f884fe656371ca931ba5c6e0adb3f0a: Status 404 returned error can't find the container with id 2e47abccafe2f1a62e1b9196f0bc03b86f884fe656371ca931ba5c6e0adb3f0a
Apr 17 08:19:38.447070 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:38.447018    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" event={"ID":"64c12487-7d51-4f9f-a774-11145c3d38d7","Type":"ContainerStarted","Data":"fd144e7c744560b8360306c9814fd2b21d779f4395cbbafecfb2d071bdca9332"}
Apr 17 08:19:38.448654 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:38.448611    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" event={"ID":"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2","Type":"ContainerStarted","Data":"00d1b0a9b713963c38070465e8cf2956b7f3e80b89c8dc9954a6527961d70212"}
Apr 17 08:19:38.448873 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:38.448659    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" event={"ID":"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2","Type":"ContainerStarted","Data":"2e47abccafe2f1a62e1b9196f0bc03b86f884fe656371ca931ba5c6e0adb3f0a"}
Apr 17 08:19:39.453615 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:39.453573    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" event={"ID":"64c12487-7d51-4f9f-a774-11145c3d38d7","Type":"ContainerStarted","Data":"537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541"}
Apr 17 08:19:39.454010 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:39.453732    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:40.459507 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:40.459458    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" event={"ID":"64c12487-7d51-4f9f-a774-11145c3d38d7","Type":"ContainerStarted","Data":"a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e"}
Apr 17 08:19:42.471811 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:42.471772    2571 generic.go:358] "Generic (PLEG): container finished" podID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerID="00d1b0a9b713963c38070465e8cf2956b7f3e80b89c8dc9954a6527961d70212" exitCode=0
Apr 17 08:19:42.472219 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:42.471824    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" event={"ID":"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2","Type":"ContainerDied","Data":"00d1b0a9b713963c38070465e8cf2956b7f3e80b89c8dc9954a6527961d70212"}
Apr 17 08:19:43.478217 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:43.478177    2571 generic.go:358] "Generic (PLEG): container finished" podID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerID="a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e" exitCode=0
Apr 17 08:19:43.478757 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:43.478247    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" event={"ID":"64c12487-7d51-4f9f-a774-11145c3d38d7","Type":"ContainerDied","Data":"a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e"}
Apr 17 08:19:43.480209 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:43.480179    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" event={"ID":"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2","Type":"ContainerStarted","Data":"574b9704fd3a031434a4950f781080fd542035321d9420079ccb1d34c7bb8ddc"}
Apr 17 08:19:43.517910 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:43.517869    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podStartSLOduration=6.517853216 podStartE2EDuration="6.517853216s" podCreationTimestamp="2026-04-17 08:19:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-17 08:19:43.514841567 +0000 UTC m=+1654.568444776" watchObservedRunningTime="2026-04-17 08:19:43.517853216 +0000 UTC m=+1654.571456426"
Apr 17 08:19:44.486256 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:44.486209    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" event={"ID":"64c12487-7d51-4f9f-a774-11145c3d38d7","Type":"ContainerStarted","Data":"f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b"}
Apr 17 08:19:44.511255 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:44.511194    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podStartSLOduration=6.581628631 podStartE2EDuration="7.51118067s" podCreationTimestamp="2026-04-17 08:19:37 +0000 UTC" firstStartedPulling="2026-04-17 08:19:37.772225453 +0000 UTC m=+1648.825828683" lastFinishedPulling="2026-04-17 08:19:38.701777524 +0000 UTC m=+1649.755380722" observedRunningTime="2026-04-17 08:19:44.507716973 +0000 UTC m=+1655.561320184" watchObservedRunningTime="2026-04-17 08:19:44.51118067 +0000 UTC m=+1655.564783881"
Apr 17 08:19:47.626467 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:47.626416    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:47.626467 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:47.626476    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:47.628068 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:47.628036    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8001/health\": dial tcp 10.134.0.47:8001: connect: connection refused"
Apr 17 08:19:47.659629 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:47.659590    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:47.659629 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:47.659638    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:19:47.661284 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:47.661248    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Apr 17 08:19:57.627323 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:57.627275    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8001/health\": dial tcp 10.134.0.47:8001: connect: connection refused"
Apr 17 08:19:57.639770 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:57.639744    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:19:57.659910 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:19:57.659873    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Apr 17 08:20:07.626850 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:20:07.626796    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8001/health\": dial tcp 10.134.0.47:8001: connect: connection refused"
Apr 17 08:20:07.660629 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:20:07.660586    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Apr 17 08:20:17.626590 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:20:17.626543    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8001/health\": dial tcp 10.134.0.47:8001: connect: connection refused"
Apr 17 08:20:17.659908 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:20:17.659858    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Apr 17 08:20:27.626883 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:20:27.626773    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8001/health\": dial tcp 10.134.0.47:8001: connect: connection refused"
Apr 17 08:20:27.660768 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:20:27.660721    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Apr 17 08:20:37.626690 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:20:37.626635    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8001/health\": dial tcp 10.134.0.47:8001: connect: connection refused"
Apr 17 08:20:37.660500 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:20:37.660456    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Apr 17 08:20:47.627041 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:20:47.626994    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8001/health\": dial tcp 10.134.0.47:8001: connect: connection refused"
Apr 17 08:20:47.660101 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:20:47.660062    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Apr 17 08:20:57.627128 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:20:57.627070    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8001/health\": dial tcp 10.134.0.47:8001: connect: connection refused"
Apr 17 08:20:57.660451 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:20:57.660406    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Apr 17 08:21:07.627244 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:07.627195    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8001/health\": dial tcp 10.134.0.47:8001: connect: connection refused"
Apr 17 08:21:07.660817 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:07.660782    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Apr 17 08:21:17.627321 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:17.627273    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8001/health\": dial tcp 10.134.0.47:8001: connect: connection refused"
Apr 17 08:21:17.660715 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:17.660675    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Apr 17 08:21:27.627476 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:27.627410    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8001/health\": dial tcp 10.134.0.47:8001: connect: connection refused"
Apr 17 08:21:27.659852 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:27.659817    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Apr 17 08:21:37.627290 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:37.627239    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8001/health\": dial tcp 10.134.0.47:8001: connect: connection refused"
Apr 17 08:21:37.660043 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:37.660003    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Apr 17 08:21:47.626896 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:47.626856    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8001/health\": dial tcp 10.134.0.47:8001: connect: connection refused"
Apr 17 08:21:47.660095 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:47.660047    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Apr 17 08:21:57.637736 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:57.637653    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:21:57.649617 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:57.649594    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:21:57.670299 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:57.670275    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:21:57.678030 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:21:57.678006    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:22:08.813101 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:08.813070    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"]
Apr 17 08:22:08.813672 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:08.813345    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main" containerID="cri-o://574b9704fd3a031434a4950f781080fd542035321d9420079ccb1d34c7bb8ddc" gracePeriod=30
Apr 17 08:22:08.823257 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:08.823225    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"]
Apr 17 08:22:08.823560 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:08.823530    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main" containerID="cri-o://f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b" gracePeriod=30
Apr 17 08:22:09.583867 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:09.583838    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 08:22:09.585822 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:09.585802    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 08:22:24.444813 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:24.444781    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:24.475659 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:24.475632    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:24.484179 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:24.484154    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:24.493871 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:24.493847    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:24.510060 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:24.510038    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:24.518009 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:24.517994    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:25.449691 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:25.449663    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:25.470511 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:25.470486    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:25.478309 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:25.478288    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:25.488865 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:25.488844    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:25.506720 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:25.506698    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:25.514914 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:25.514894    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:26.484328 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:26.484291    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:26.504342 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:26.504318    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:26.511371 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:26.511351    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:26.523050 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:26.523029    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:26.539789 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:26.539771    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:26.548075 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:26.548057    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:27.475481 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:27.475449    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:27.494572 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:27.494548    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:27.500759 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:27.500740    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:27.510933 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:27.510912    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:27.530017 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:27.529994    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:27.541498 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:27.541479    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:28.464244 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:28.464213    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:28.485873 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:28.485850    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:28.492529 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:28.492507    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:28.503126 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:28.503104    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:28.519404 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:28.519384    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:28.530744 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:28.530729    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:29.498216 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:29.498186    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:29.519377 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:29.519356    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:29.525884 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:29.525868    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:29.536184 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:29.536162    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:29.555042 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:29.555023    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:29.562994 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:29.562979    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:30.475329 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:30.475296    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:30.493748 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:30.493721    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:30.501191 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:30.501171    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:30.510566 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:30.510548    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:30.526194 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:30.526177    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:30.534153 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:30.534136    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:31.478483 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:31.478449    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:31.501208 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:31.501180    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:31.507564 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:31.507543    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:31.519110 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:31.519091    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:31.535196 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:31.535170    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:31.543281 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:31.543263    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:32.483527 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:32.483499    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:32.503616 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:32.503588    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:32.510490 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:32.510468    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:32.521213 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:32.521194    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:32.545548 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:32.545530    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:32.554201 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:32.554185    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:33.480684 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:33.480654    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:33.499725 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:33.499689    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:33.505701 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:33.505676    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:33.515257 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:33.515239    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:33.531331 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:33.531309    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:33.539412 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:33.539390    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:34.486337 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:34.486308    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:34.505404 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:34.505384    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:34.511523 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:34.511505    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:34.521935 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:34.521921    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:34.539983 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:34.539963    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:34.547623 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:34.547605    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:35.501494 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:35.501461    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:35.521762 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:35.521738    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:35.529097 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:35.529074    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:35.539120 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:35.539098    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:35.556156 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:35.556130    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:35.564138 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:35.564119    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:36.489965 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:36.489939    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:36.509771 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:36.509743    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:36.515876 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:36.515858    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:36.525846 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:36.525826    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:36.541992 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:36.541971    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:36.549709 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:36.549691    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:37.545913 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:37.545887    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-6866b85949-89mvx_cec87044-ce22-4b2a-af90-ffdfcb72ae20/istio-proxy/0.log"
Apr 17 08:22:37.569054 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:37.569028    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:37.575816 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:37.575798    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/llm-d-routing-sidecar/0.log"
Apr 17 08:22:37.591161 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:37.591141    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/storage-initializer/0.log"
Apr 17 08:22:37.608752 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:37.608706    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/main/0.log"
Apr 17 08:22:37.617714 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:37.617695    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm_a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/storage-initializer/0.log"
Apr 17 08:22:38.823637 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:38.823599    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="llm-d-routing-sidecar" containerID="cri-o://537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541" gracePeriod=2
Apr 17 08:22:39.068920 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.068864    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:22:39.084902 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.084882    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:39.085538 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.085517    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:22:39.157827 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.157795    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcctc\" (UniqueName: \"kubernetes.io/projected/64c12487-7d51-4f9f-a774-11145c3d38d7-kube-api-access-kcctc\") pod \"64c12487-7d51-4f9f-a774-11145c3d38d7\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") "
Apr 17 08:22:39.157827 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.157830    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qkzg\" (UniqueName: \"kubernetes.io/projected/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-kube-api-access-5qkzg\") pod \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") "
Apr 17 08:22:39.158027 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.157861    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-home\") pod \"64c12487-7d51-4f9f-a774-11145c3d38d7\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") "
Apr 17 08:22:39.158027 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.157921    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-tls-certs\") pod \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") "
Apr 17 08:22:39.158027 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.157950    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-dshm\") pod \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") "
Apr 17 08:22:39.158027 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.157983    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-kserve-provision-location\") pod \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") "
Apr 17 08:22:39.158027 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.158011    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-model-cache\") pod \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") "
Apr 17 08:22:39.158294 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.158047    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-home\") pod \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\" (UID: \"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2\") "
Apr 17 08:22:39.158294 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.158073    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64c12487-7d51-4f9f-a774-11145c3d38d7-tls-certs\") pod \"64c12487-7d51-4f9f-a774-11145c3d38d7\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") "
Apr 17 08:22:39.158294 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.158108    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-kserve-provision-location\") pod \"64c12487-7d51-4f9f-a774-11145c3d38d7\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") "
Apr 17 08:22:39.158294 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.158130    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-dshm\") pod \"64c12487-7d51-4f9f-a774-11145c3d38d7\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") "
Apr 17 08:22:39.158294 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.158158    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-model-cache\") pod \"64c12487-7d51-4f9f-a774-11145c3d38d7\" (UID: \"64c12487-7d51-4f9f-a774-11145c3d38d7\") "
Apr 17 08:22:39.158576 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.158386    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-home" (OuterVolumeSpecName: "home") pod "64c12487-7d51-4f9f-a774-11145c3d38d7" (UID: "64c12487-7d51-4f9f-a774-11145c3d38d7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:22:39.158576 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.158494    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-home" (OuterVolumeSpecName: "home") pod "a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" (UID: "a3a9b0f3-339c-4bfa-a5a7-e556c6965da2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:22:39.158674 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.158633    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-model-cache" (OuterVolumeSpecName: "model-cache") pod "a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" (UID: "a3a9b0f3-339c-4bfa-a5a7-e556c6965da2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:22:39.160607 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.160467    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-kube-api-access-5qkzg" (OuterVolumeSpecName: "kube-api-access-5qkzg") pod "a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" (UID: "a3a9b0f3-339c-4bfa-a5a7-e556c6965da2"). InnerVolumeSpecName "kube-api-access-5qkzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:22:39.160607 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.160544    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c12487-7d51-4f9f-a774-11145c3d38d7-kube-api-access-kcctc" (OuterVolumeSpecName: "kube-api-access-kcctc") pod "64c12487-7d51-4f9f-a774-11145c3d38d7" (UID: "64c12487-7d51-4f9f-a774-11145c3d38d7"). InnerVolumeSpecName "kube-api-access-kcctc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Apr 17 08:22:39.160798 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.160671    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-dshm" (OuterVolumeSpecName: "dshm") pod "a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" (UID: "a3a9b0f3-339c-4bfa-a5a7-e556c6965da2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:22:39.160939 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.160915    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" (UID: "a3a9b0f3-339c-4bfa-a5a7-e556c6965da2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 08:22:39.161016 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.160979    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64c12487-7d51-4f9f-a774-11145c3d38d7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "64c12487-7d51-4f9f-a774-11145c3d38d7" (UID: "64c12487-7d51-4f9f-a774-11145c3d38d7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Apr 17 08:22:39.161170 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.161143    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-model-cache" (OuterVolumeSpecName: "model-cache") pod "64c12487-7d51-4f9f-a774-11145c3d38d7" (UID: "64c12487-7d51-4f9f-a774-11145c3d38d7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:22:39.161286 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.161256    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-dshm" (OuterVolumeSpecName: "dshm") pod "64c12487-7d51-4f9f-a774-11145c3d38d7" (UID: "64c12487-7d51-4f9f-a774-11145c3d38d7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:22:39.165655 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.165628    2571 generic.go:358] "Generic (PLEG): container finished" podID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerID="574b9704fd3a031434a4950f781080fd542035321d9420079ccb1d34c7bb8ddc" exitCode=137
Apr 17 08:22:39.165775 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.165700    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"
Apr 17 08:22:39.165775 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.165706    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" event={"ID":"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2","Type":"ContainerDied","Data":"574b9704fd3a031434a4950f781080fd542035321d9420079ccb1d34c7bb8ddc"}
Apr 17 08:22:39.165775 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.165741    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm" event={"ID":"a3a9b0f3-339c-4bfa-a5a7-e556c6965da2","Type":"ContainerDied","Data":"2e47abccafe2f1a62e1b9196f0bc03b86f884fe656371ca931ba5c6e0adb3f0a"}
Apr 17 08:22:39.165775 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.165755    2571 scope.go:117] "RemoveContainer" containerID="574b9704fd3a031434a4950f781080fd542035321d9420079ccb1d34c7bb8ddc"
Apr 17 08:22:39.167176 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.167147    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn_64c12487-7d51-4f9f-a774-11145c3d38d7/main/0.log"
Apr 17 08:22:39.167833 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.167810    2571 generic.go:358] "Generic (PLEG): container finished" podID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerID="f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b" exitCode=137
Apr 17 08:22:39.167833 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.167833    2571 generic.go:358] "Generic (PLEG): container finished" podID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerID="537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541" exitCode=0
Apr 17 08:22:39.167953 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.167860    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" event={"ID":"64c12487-7d51-4f9f-a774-11145c3d38d7","Type":"ContainerDied","Data":"f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b"}
Apr 17 08:22:39.167953 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.167887    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" event={"ID":"64c12487-7d51-4f9f-a774-11145c3d38d7","Type":"ContainerDied","Data":"537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541"}
Apr 17 08:22:39.167953 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.167896    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"
Apr 17 08:22:39.167953 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.167902    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn" event={"ID":"64c12487-7d51-4f9f-a774-11145c3d38d7","Type":"ContainerDied","Data":"fd144e7c744560b8360306c9814fd2b21d779f4395cbbafecfb2d071bdca9332"}
Apr 17 08:22:39.186154 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.186133    2571 scope.go:117] "RemoveContainer" containerID="00d1b0a9b713963c38070465e8cf2956b7f3e80b89c8dc9954a6527961d70212"
Apr 17 08:22:39.223209 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.223170    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" (UID: "a3a9b0f3-339c-4bfa-a5a7-e556c6965da2"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:22:39.227727 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.227701    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "64c12487-7d51-4f9f-a774-11145c3d38d7" (UID: "64c12487-7d51-4f9f-a774-11145c3d38d7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Apr 17 08:22:39.255859 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.255836    2571 scope.go:117] "RemoveContainer" containerID="574b9704fd3a031434a4950f781080fd542035321d9420079ccb1d34c7bb8ddc"
Apr 17 08:22:39.256147 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:22:39.256125    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"574b9704fd3a031434a4950f781080fd542035321d9420079ccb1d34c7bb8ddc\": container with ID starting with 574b9704fd3a031434a4950f781080fd542035321d9420079ccb1d34c7bb8ddc not found: ID does not exist" containerID="574b9704fd3a031434a4950f781080fd542035321d9420079ccb1d34c7bb8ddc"
Apr 17 08:22:39.256226 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.256161    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"574b9704fd3a031434a4950f781080fd542035321d9420079ccb1d34c7bb8ddc"} err="failed to get container status \"574b9704fd3a031434a4950f781080fd542035321d9420079ccb1d34c7bb8ddc\": rpc error: code = NotFound desc = could not find container \"574b9704fd3a031434a4950f781080fd542035321d9420079ccb1d34c7bb8ddc\": container with ID starting with 574b9704fd3a031434a4950f781080fd542035321d9420079ccb1d34c7bb8ddc not found: ID does not exist"
Apr 17 08:22:39.256226 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.256190    2571 scope.go:117] "RemoveContainer" containerID="00d1b0a9b713963c38070465e8cf2956b7f3e80b89c8dc9954a6527961d70212"
Apr 17 08:22:39.256515 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:22:39.256415    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00d1b0a9b713963c38070465e8cf2956b7f3e80b89c8dc9954a6527961d70212\": container with ID starting with 00d1b0a9b713963c38070465e8cf2956b7f3e80b89c8dc9954a6527961d70212 not found: ID does not exist" containerID="00d1b0a9b713963c38070465e8cf2956b7f3e80b89c8dc9954a6527961d70212"
Apr 17 08:22:39.256515 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.256451    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d1b0a9b713963c38070465e8cf2956b7f3e80b89c8dc9954a6527961d70212"} err="failed to get container status \"00d1b0a9b713963c38070465e8cf2956b7f3e80b89c8dc9954a6527961d70212\": rpc error: code = NotFound desc = could not find container \"00d1b0a9b713963c38070465e8cf2956b7f3e80b89c8dc9954a6527961d70212\": container with ID starting with 00d1b0a9b713963c38070465e8cf2956b7f3e80b89c8dc9954a6527961d70212 not found: ID does not exist"
Apr 17 08:22:39.256515 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.256469    2571 scope.go:117] "RemoveContainer" containerID="f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b"
Apr 17 08:22:39.259799 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.259772    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-home\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:22:39.259799 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.259799    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-tls-certs\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:22:39.259923 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.259813    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-dshm\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:22:39.259923 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.259826    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-kserve-provision-location\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:22:39.259923 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.259841    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-model-cache\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:22:39.259923 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.259855    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-home\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:22:39.259923 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.259866    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64c12487-7d51-4f9f-a774-11145c3d38d7-tls-certs\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:22:39.259923 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.259879    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-kserve-provision-location\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:22:39.259923 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.259892    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-dshm\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:22:39.259923 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.259905    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64c12487-7d51-4f9f-a774-11145c3d38d7-model-cache\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:22:39.259923 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.259919    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcctc\" (UniqueName: \"kubernetes.io/projected/64c12487-7d51-4f9f-a774-11145c3d38d7-kube-api-access-kcctc\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:22:39.260222 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.259932    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qkzg\" (UniqueName: \"kubernetes.io/projected/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2-kube-api-access-5qkzg\") on node \"ip-10-0-143-45.ec2.internal\" DevicePath \"\""
Apr 17 08:22:39.276225 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.276208    2571 scope.go:117] "RemoveContainer" containerID="a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e"
Apr 17 08:22:39.336520 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.336496    2571 scope.go:117] "RemoveContainer" containerID="537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541"
Apr 17 08:22:39.344244 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.344223    2571 scope.go:117] "RemoveContainer" containerID="f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b"
Apr 17 08:22:39.344507 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:22:39.344486    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b\": container with ID starting with f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b not found: ID does not exist" containerID="f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b"
Apr 17 08:22:39.344559 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.344519    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b"} err="failed to get container status \"f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b\": rpc error: code = NotFound desc = could not find container \"f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b\": container with ID starting with f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b not found: ID does not exist"
Apr 17 08:22:39.344559 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.344539    2571 scope.go:117] "RemoveContainer" containerID="a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e"
Apr 17 08:22:39.344751 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:22:39.344737    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e\": container with ID starting with a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e not found: ID does not exist" containerID="a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e"
Apr 17 08:22:39.344792 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.344756    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e"} err="failed to get container status \"a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e\": rpc error: code = NotFound desc = could not find container \"a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e\": container with ID starting with a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e not found: ID does not exist"
Apr 17 08:22:39.344792 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.344769    2571 scope.go:117] "RemoveContainer" containerID="537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541"
Apr 17 08:22:39.345017 ip-10-0-143-45 kubenswrapper[2571]: E0417 08:22:39.344996    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541\": container with ID starting with 537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541 not found: ID does not exist" containerID="537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541"
Apr 17 08:22:39.345100 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.345021    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541"} err="failed to get container status \"537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541\": rpc error: code = NotFound desc = could not find container \"537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541\": container with ID starting with 537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541 not found: ID does not exist"
Apr 17 08:22:39.345100 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.345042    2571 scope.go:117] "RemoveContainer" containerID="f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b"
Apr 17 08:22:39.345240 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.345224    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b"} err="failed to get container status \"f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b\": rpc error: code = NotFound desc = could not find container \"f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b\": container with ID starting with f5abf912b67b5faa4b604a8c5c693c9e1fadf7abbd4eb85dc965eb969019747b not found: ID does not exist"
Apr 17 08:22:39.345281 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.345242    2571 scope.go:117] "RemoveContainer" containerID="a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e"
Apr 17 08:22:39.345555 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.345535    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e"} err="failed to get container status \"a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e\": rpc error: code = NotFound desc = could not find container \"a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e\": container with ID starting with a8575ddd67a2ae0207637b00615a4d842838fee8758184bc70275dd3dcb4635e not found: ID does not exist"
Apr 17 08:22:39.345625 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.345555    2571 scope.go:117] "RemoveContainer" containerID="537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541"
Apr 17 08:22:39.345764 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.345741    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541"} err="failed to get container status \"537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541\": rpc error: code = NotFound desc = could not find container \"537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541\": container with ID starting with 537251e6c7a128de4adf908377064b8eee3db3bd5a2ff688c8f026e611510541 not found: ID does not exist"
Apr 17 08:22:39.491527 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.491500    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"]
Apr 17 08:22:39.494868 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.494842    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-74f7f9c99b-bq5rn"]
Apr 17 08:22:39.505354 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.505330    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"]
Apr 17 08:22:39.509527 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.509506    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-769c785df7-kflqm"]
Apr 17 08:22:39.544407 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.544378    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" path="/var/lib/kubelet/pods/64c12487-7d51-4f9f-a774-11145c3d38d7/volumes"
Apr 17 08:22:39.544866 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:39.544853    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" path="/var/lib/kubelet/pods/a3a9b0f3-339c-4bfa-a5a7-e556c6965da2/volumes"
Apr 17 08:22:40.185804 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:40.185733    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-b87xk_38e5d3d2-6156-4e4a-a543-88a520539a96/authorino/0.log"
Apr 17 08:22:40.201679 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:40.201653    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-4bcjv_6e531445-d515-4ac6-8d17-bd4ea771e009/manager/0.log"
Apr 17 08:22:42.616786 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.616748    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jc8j6/must-gather-gstch"]
Apr 17 08:22:42.617282 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.617197    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main"
Apr 17 08:22:42.617282 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.617217    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main"
Apr 17 08:22:42.617282 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.617238    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="storage-initializer"
Apr 17 08:22:42.617282 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.617247    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="storage-initializer"
Apr 17 08:22:42.617282 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.617261    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="llm-d-routing-sidecar"
Apr 17 08:22:42.617282 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.617271    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="llm-d-routing-sidecar"
Apr 17 08:22:42.617606 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.617289    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main"
Apr 17 08:22:42.617606 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.617297    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main"
Apr 17 08:22:42.617606 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.617306    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="storage-initializer"
Apr 17 08:22:42.617606 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.617314    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="storage-initializer"
Apr 17 08:22:42.617606 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.617410    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="llm-d-routing-sidecar"
Apr 17 08:22:42.617606 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.617423    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="a3a9b0f3-339c-4bfa-a5a7-e556c6965da2" containerName="main"
Apr 17 08:22:42.617606 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.617443    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="64c12487-7d51-4f9f-a774-11145c3d38d7" containerName="main"
Apr 17 08:22:42.622563 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.622537    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jc8j6/must-gather-gstch"
Apr 17 08:22:42.627342 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.627320    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jc8j6\"/\"openshift-service-ca.crt\""
Apr 17 08:22:42.628399 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.628376    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jc8j6\"/\"default-dockercfg-tn4lx\""
Apr 17 08:22:42.628536 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.628414    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jc8j6\"/\"kube-root-ca.crt\""
Apr 17 08:22:42.632305 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.632281    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jc8j6/must-gather-gstch"]
Apr 17 08:22:42.687529 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.687497    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt5z8\" (UniqueName: \"kubernetes.io/projected/f490038f-2f2c-48c5-aea6-71fb3441704c-kube-api-access-nt5z8\") pod \"must-gather-gstch\" (UID: \"f490038f-2f2c-48c5-aea6-71fb3441704c\") " pod="openshift-must-gather-jc8j6/must-gather-gstch"
Apr 17 08:22:42.687693 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.687553    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f490038f-2f2c-48c5-aea6-71fb3441704c-must-gather-output\") pod \"must-gather-gstch\" (UID: \"f490038f-2f2c-48c5-aea6-71fb3441704c\") " pod="openshift-must-gather-jc8j6/must-gather-gstch"
Apr 17 08:22:42.788746 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.788716    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f490038f-2f2c-48c5-aea6-71fb3441704c-must-gather-output\") pod \"must-gather-gstch\" (UID: \"f490038f-2f2c-48c5-aea6-71fb3441704c\") " pod="openshift-must-gather-jc8j6/must-gather-gstch"
Apr 17 08:22:42.788907 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.788780    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nt5z8\" (UniqueName: \"kubernetes.io/projected/f490038f-2f2c-48c5-aea6-71fb3441704c-kube-api-access-nt5z8\") pod \"must-gather-gstch\" (UID: \"f490038f-2f2c-48c5-aea6-71fb3441704c\") " pod="openshift-must-gather-jc8j6/must-gather-gstch"
Apr 17 08:22:42.789072 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.789051    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f490038f-2f2c-48c5-aea6-71fb3441704c-must-gather-output\") pod \"must-gather-gstch\" (UID: \"f490038f-2f2c-48c5-aea6-71fb3441704c\") " pod="openshift-must-gather-jc8j6/must-gather-gstch"
Apr 17 08:22:42.796664 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.796631    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt5z8\" (UniqueName: \"kubernetes.io/projected/f490038f-2f2c-48c5-aea6-71fb3441704c-kube-api-access-nt5z8\") pod \"must-gather-gstch\" (UID: \"f490038f-2f2c-48c5-aea6-71fb3441704c\") " pod="openshift-must-gather-jc8j6/must-gather-gstch"
Apr 17 08:22:42.933417 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:42.933317    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jc8j6/must-gather-gstch"
Apr 17 08:22:43.051962 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:43.051789    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jc8j6/must-gather-gstch"]
Apr 17 08:22:43.054497 ip-10-0-143-45 kubenswrapper[2571]: W0417 08:22:43.054470    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf490038f_2f2c_48c5_aea6_71fb3441704c.slice/crio-aa615b315a53d6f8a57b0cbb1a943ad48af6c9625363226b2b4f432ad2f2660d WatchSource:0}: Error finding container aa615b315a53d6f8a57b0cbb1a943ad48af6c9625363226b2b4f432ad2f2660d: Status 404 returned error can't find the container with id aa615b315a53d6f8a57b0cbb1a943ad48af6c9625363226b2b4f432ad2f2660d
Apr 17 08:22:43.056132 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:43.056114    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Apr 17 08:22:43.185125 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:43.185036    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jc8j6/must-gather-gstch" event={"ID":"f490038f-2f2c-48c5-aea6-71fb3441704c","Type":"ContainerStarted","Data":"aa615b315a53d6f8a57b0cbb1a943ad48af6c9625363226b2b4f432ad2f2660d"}
Apr 17 08:22:44.192867 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:44.192822    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jc8j6/must-gather-gstch" event={"ID":"f490038f-2f2c-48c5-aea6-71fb3441704c","Type":"ContainerStarted","Data":"6d00e838524223c7f32e1472d5c154fccb69c4289f050f29c4e035803090c62b"}
Apr 17 08:22:44.192867 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:44.192872    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jc8j6/must-gather-gstch" event={"ID":"f490038f-2f2c-48c5-aea6-71fb3441704c","Type":"ContainerStarted","Data":"1dbf9a71f5198c25b1ad2bf4877e6b53aebddc1ffef85b6979cc9c913fae36d5"}
Apr 17 08:22:44.208160 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:44.208106    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-jc8j6/must-gather-gstch" podStartSLOduration=1.486328775 podStartE2EDuration="2.208090714s" podCreationTimestamp="2026-04-17 08:22:42 +0000 UTC" firstStartedPulling="2026-04-17 08:22:43.056241437 +0000 UTC m=+1834.109844626" lastFinishedPulling="2026-04-17 08:22:43.778003373 +0000 UTC m=+1834.831606565" observedRunningTime="2026-04-17 08:22:44.207677746 +0000 UTC m=+1835.261280969" watchObservedRunningTime="2026-04-17 08:22:44.208090714 +0000 UTC m=+1835.261693924"
Apr 17 08:22:45.266901 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:45.266846    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bgjfg_f1e8aadb-7290-45b3-8467-a2a13b104ce7/global-pull-secret-syncer/0.log"
Apr 17 08:22:45.378516 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:45.378485    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v2g9s_99ac4cf1-ceb9-44bd-9ab5-ad7588da1738/konnectivity-agent/0.log"
Apr 17 08:22:45.439383 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:45.439353    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-45.ec2.internal_bd84846842e0dd9650476cd580a9ac6c/haproxy/0.log"
Apr 17 08:22:49.420548 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:49.420504    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-b87xk_38e5d3d2-6156-4e4a-a543-88a520539a96/authorino/0.log"
Apr 17 08:22:49.447738 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:49.447708    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-4bcjv_6e531445-d515-4ac6-8d17-bd4ea771e009/manager/0.log"
Apr 17 08:22:50.864062 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:50.864024    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b7k7k_763faa91-078d-456f-9daf-77ee5acc9aca/node-exporter/0.log"
Apr 17 08:22:50.883453 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:50.883401    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b7k7k_763faa91-078d-456f-9daf-77ee5acc9aca/kube-rbac-proxy/0.log"
Apr 17 08:22:50.903167 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:50.903140    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b7k7k_763faa91-078d-456f-9daf-77ee5acc9aca/init-textfile/0.log"
Apr 17 08:22:51.056118 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:51.056090    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9d44df66c-258jh_9d171f5a-aa91-4614-9e29-c3e5b59fdba3/kube-rbac-proxy-main/0.log"
Apr 17 08:22:51.081335 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:51.081298    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9d44df66c-258jh_9d171f5a-aa91-4614-9e29-c3e5b59fdba3/kube-rbac-proxy-self/0.log"
Apr 17 08:22:51.106519 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:51.106476    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9d44df66c-258jh_9d171f5a-aa91-4614-9e29-c3e5b59fdba3/openshift-state-metrics/0.log"
Apr 17 08:22:51.367104 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:51.367050    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-57cf98b594-2zvfs_49619515-087b-4a5a-aeb4-b95a5802cde4/prometheus-operator-admission-webhook/0.log"
Apr 17 08:22:51.482996 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:51.482942    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56684c54b9-49k8p_8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23/thanos-query/0.log"
Apr 17 08:22:51.507029 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:51.506976    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56684c54b9-49k8p_8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23/kube-rbac-proxy-web/0.log"
Apr 17 08:22:51.536359 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:51.536083    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56684c54b9-49k8p_8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23/kube-rbac-proxy/0.log"
Apr 17 08:22:51.581116 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:51.581080    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56684c54b9-49k8p_8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23/prom-label-proxy/0.log"
Apr 17 08:22:51.603565 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:51.603538    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56684c54b9-49k8p_8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23/kube-rbac-proxy-rules/0.log"
Apr 17 08:22:51.634496 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:51.634370    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56684c54b9-49k8p_8d6e4e5e-1d41-4f7c-b780-9a2bd9a98a23/kube-rbac-proxy-metrics/0.log"
Apr 17 08:22:53.904086 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:53.904047    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"]
Apr 17 08:22:53.909498 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:53.909471    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:53.917013 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:53.916986    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"]
Apr 17 08:22:53.990469 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:53.990405    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3ba4985e-8356-4958-8952-ffd5ca985c21-podres\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:53.990793 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:53.990771    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3ba4985e-8356-4958-8952-ffd5ca985c21-proc\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:53.990981 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:53.990956    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ba4985e-8356-4958-8952-ffd5ca985c21-sys\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:53.991082 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:53.991027    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4t5h\" (UniqueName: \"kubernetes.io/projected/3ba4985e-8356-4958-8952-ffd5ca985c21-kube-api-access-n4t5h\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:53.991082 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:53.991068    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ba4985e-8356-4958-8952-ffd5ca985c21-lib-modules\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:54.091823 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.091789    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ba4985e-8356-4958-8952-ffd5ca985c21-sys\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:54.091977 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.091835    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n4t5h\" (UniqueName: \"kubernetes.io/projected/3ba4985e-8356-4958-8952-ffd5ca985c21-kube-api-access-n4t5h\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:54.091977 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.091912    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ba4985e-8356-4958-8952-ffd5ca985c21-sys\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:54.091977 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.091955    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ba4985e-8356-4958-8952-ffd5ca985c21-lib-modules\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:54.092110 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.092001    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3ba4985e-8356-4958-8952-ffd5ca985c21-podres\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:54.092110 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.092032    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3ba4985e-8356-4958-8952-ffd5ca985c21-proc\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:54.092216 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.092138    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3ba4985e-8356-4958-8952-ffd5ca985c21-proc\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:54.092216 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.092148    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3ba4985e-8356-4958-8952-ffd5ca985c21-podres\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:54.092216 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.092149    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ba4985e-8356-4958-8952-ffd5ca985c21-lib-modules\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:54.099728 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.099701    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4t5h\" (UniqueName: \"kubernetes.io/projected/3ba4985e-8356-4958-8952-ffd5ca985c21-kube-api-access-n4t5h\") pod \"perf-node-gather-daemonset-nfvgn\" (UID: \"3ba4985e-8356-4958-8952-ffd5ca985c21\") " pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:54.221721 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.221641    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:54.377155 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.372329    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"]
Apr 17 08:22:54.877890 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.877860    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jjcmh_b99d5d29-1f8b-46e2-b121-000bc5b77dfc/dns/0.log"
Apr 17 08:22:54.896295 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:54.896269    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jjcmh_b99d5d29-1f8b-46e2-b121-000bc5b77dfc/kube-rbac-proxy/0.log"
Apr 17 08:22:55.036748 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:55.036716    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vcnrb_accd983c-5d20-4f58-9453-9dd0a312b13b/dns-node-resolver/0.log"
Apr 17 08:22:55.246507 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:55.246395    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn" event={"ID":"3ba4985e-8356-4958-8952-ffd5ca985c21","Type":"ContainerStarted","Data":"1a015521adafbedd1ea2898fce99fef189689b2ce53886028ac93b39459f9d26"}
Apr 17 08:22:55.246742 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:55.246717    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn" event={"ID":"3ba4985e-8356-4958-8952-ffd5ca985c21","Type":"ContainerStarted","Data":"b1d68841b505af452521183feda8ca77c0bc599bea0c149c64e447e13b8fe65d"}
Apr 17 08:22:55.246815 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:55.246752    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:22:55.265290 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:55.265238    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn" podStartSLOduration=2.265223877 podStartE2EDuration="2.265223877s" podCreationTimestamp="2026-04-17 08:22:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-17 08:22:55.261746804 +0000 UTC m=+1846.315350014" watchObservedRunningTime="2026-04-17 08:22:55.265223877 +0000 UTC m=+1846.318827089"
Apr 17 08:22:55.492585 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:55.492551    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-27pc9_07b9a2c8-ac6f-44a3-96e9-995ed5bf5d32/node-ca/0.log"
Apr 17 08:22:56.867558 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:56.867527    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qqmvg_68a69e91-436d-4788-b2de-32c453ccd36a/serve-healthcheck-canary/0.log"
Apr 17 08:22:57.318820 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:57.318790    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-74g7z_4acfa4c5-aaf8-4d9b-bf5d-7b0640375711/kube-rbac-proxy/0.log"
Apr 17 08:22:57.336685 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:57.336656    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-74g7z_4acfa4c5-aaf8-4d9b-bf5d-7b0640375711/exporter/0.log"
Apr 17 08:22:57.358447 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:22:57.358410    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-74g7z_4acfa4c5-aaf8-4d9b-bf5d-7b0640375711/extractor/0.log"
Apr 17 08:23:00.009992 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:00.009961    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c76446df-qdq6q_9feb5804-7f75-4b2e-90f2-fb1c62bd5396/manager/0.log"
Apr 17 08:23:00.575407 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:00.575381    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-86f7b4b499-6rv84_4355290c-c266-4053-9da1-71ad9182681a/server/0.log"
Apr 17 08:23:00.815116 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:00.815085    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-696fc77849-xc7vc_a22c9ec0-07ac-4327-934a-12d476ca5771/manager/0.log"
Apr 17 08:23:00.832161 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:00.832098    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-gx66p_639bb21e-fc99-4584-9644-b218441f2001/s3-init/0.log"
Apr 17 08:23:01.261085 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:01.261007    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jc8j6/perf-node-gather-daemonset-nfvgn"
Apr 17 08:23:06.561894 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:06.561860    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8jsdp_161f0c24-f3ae-495e-a9db-213c81687526/kube-multus-additional-cni-plugins/0.log"
Apr 17 08:23:06.580561 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:06.580535    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8jsdp_161f0c24-f3ae-495e-a9db-213c81687526/egress-router-binary-copy/0.log"
Apr 17 08:23:06.598219 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:06.598195    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8jsdp_161f0c24-f3ae-495e-a9db-213c81687526/cni-plugins/0.log"
Apr 17 08:23:06.617937 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:06.617861    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8jsdp_161f0c24-f3ae-495e-a9db-213c81687526/bond-cni-plugin/0.log"
Apr 17 08:23:06.638981 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:06.638955    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8jsdp_161f0c24-f3ae-495e-a9db-213c81687526/routeoverride-cni/0.log"
Apr 17 08:23:06.659721 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:06.659699    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8jsdp_161f0c24-f3ae-495e-a9db-213c81687526/whereabouts-cni-bincopy/0.log"
Apr 17 08:23:06.679565 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:06.679539    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8jsdp_161f0c24-f3ae-495e-a9db-213c81687526/whereabouts-cni/0.log"
Apr 17 08:23:07.046208 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:07.046177    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-v5cdv_268375f5-3f20-4e84-9af0-286587a445c7/kube-multus/0.log"
Apr 17 08:23:07.152128 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:07.152100    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wxkt9_6f91ec65-f7dd-4657-838b-318538310ead/network-metrics-daemon/0.log"
Apr 17 08:23:07.171686 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:07.171660    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wxkt9_6f91ec65-f7dd-4657-838b-318538310ead/kube-rbac-proxy/0.log"
Apr 17 08:23:08.508844 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:08.508810    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-controller/0.log"
Apr 17 08:23:08.528629 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:08.528603    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/0.log"
Apr 17 08:23:08.537711 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:08.537690    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovn-acl-logging/1.log"
Apr 17 08:23:08.559172 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:08.559150    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/kube-rbac-proxy-node/0.log"
Apr 17 08:23:08.580731 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:08.580708    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/kube-rbac-proxy-ovn-metrics/0.log"
Apr 17 08:23:08.600944 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:08.600915    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/northd/0.log"
Apr 17 08:23:08.624200 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:08.624173    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/nbdb/0.log"
Apr 17 08:23:08.648281 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:08.648257    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/sbdb/0.log"
Apr 17 08:23:08.759418 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:08.759342    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krj9p_e65a3e99-bd0c-4035-b41f-db6915c67e9a/ovnkube-controller/0.log"
Apr 17 08:23:09.762984 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:09.762954    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-r78gk_bde75c6d-e1a4-4848-bef6-94161be7ed61/network-check-target-container/0.log"
Apr 17 08:23:10.810537 ip-10-0-143-45 kubenswrapper[2571]: I0417 08:23:10.810503    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-8qgq7_ff540440-d950-4564-be29-fd2a0bdc83d8/iptables-alerter/0.log"