Jul 06 20:48:40.394051 ip-10-0-131-165 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 20:48:40.873851 ip-10-0-131-165 systemd[1]: Started Kubernetes Kubelet.
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.799053    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803592    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803607    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803614    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803617    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803620    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803627    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803638    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 20:48:40.903238 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803641    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803647    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803651    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803658    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803661    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803664    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803667    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803672    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803674    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803677    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803682    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803684    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803687    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803690    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803694    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803697    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803700    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803706    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803709    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 20:48:40.906662 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803712    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803714    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803718    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803721    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803723    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803731    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803734    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803739    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803742    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803745    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803747    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803753    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803775    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803782    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803788    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803791    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803795    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803800    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803803    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803805    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 20:48:40.909194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803810    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803813    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803816    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803821    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803823    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803826    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803828    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803833    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803836    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803838    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803841    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803846    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803849    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803851    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803857    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803859    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803867    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803870    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803874    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 20:48:40.910978 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803879    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803882    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803885    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803887    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803892    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803895    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803898    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803904    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803907    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803910    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803918    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803920    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803923    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803926    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803929    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803932    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803938    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803941    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803944    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 20:48:40.913100 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803949    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803954    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803959    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803962    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803965    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803970    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803973    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803976    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803982    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803985    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803988    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803991    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803996    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.803998    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.804001    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806475    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806495    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806500    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806506    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806510    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 20:48:40.928290 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806514    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806521    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806535    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806542    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806547    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806551    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806571    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806621    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806625    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806628    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806631    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806634    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806638    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806640    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806643    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806646    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806649    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806652    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806654    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 20:48:40.931140 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806657    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806663    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806666    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806670    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806673    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806676    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806679    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806682    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806685    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806688    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806691    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806694    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806696    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806699    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806704    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806707    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806710    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806713    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806717    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806721    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 20:48:40.935209 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806724    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806727    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806730    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806733    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806736    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806741    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806744    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806747    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806750    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806753    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806768    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806773    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806776    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806779    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806783    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806786    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806789    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806796    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806799    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 20:48:40.953607 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806802    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806806    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806812    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806816    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806819    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806822    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806825    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806828    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806841    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806844    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806849    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806852    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806855    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806857    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806860    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806863    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806865    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806868    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806871    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806873    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 20:48:41.104266 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806876    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806880    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806882    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806887    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806890    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806893    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806896    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806899    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806901    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806904    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806907    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806909    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806912    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806914    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806920    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.806923    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807525    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807533    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807537    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807540    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 20:48:41.105352 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807543    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807546    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807549    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807552    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807554    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807557    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807560    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807562    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807565    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807567    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807570    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807572    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807575    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807577    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807580    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807584    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807586    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807589    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807592    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 20:48:41.106437 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807594    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807598    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807601    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807604    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807607    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807609    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807612    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807616    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807620    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807622    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807625    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807627    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807630    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807632    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807635    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807637    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807640    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807642    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807645    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807647    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 20:48:41.107237 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807650    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807654    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807656    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807659    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807661    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807664    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807666    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807669    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807674    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807678    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807681    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807684    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807687    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807690    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807692    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807695    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807697    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807702    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 20:48:41.108051 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807705    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807709    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807712    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807715    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807718    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807721    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807723    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807726    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807729    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807732    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807735    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807738    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807740    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807743    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807745    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807748    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807750    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807752    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807755    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807777    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 20:48:41.109048 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807780    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807784    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807788    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807791    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807793    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807796    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807799    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807801    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807804    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807806    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807809    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807811    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807814    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807817    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807820    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807822    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807825    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.807827    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807897    2566 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807904    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807910    2566 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 20:48:41.109635 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807915    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807919    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807922    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807926    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807931    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807935    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807940    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807945    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807949    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807952    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807955    2566 flags.go:64] FLAG: --cgroup-root=""
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807958    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807961    2566 flags.go:64] FLAG: --client-ca-file=""
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807966    2566 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807969    2566 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807973    2566 flags.go:64] FLAG: --cluster-domain=""
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807976    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807979    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807982    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807985    2566 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807989    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807992    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807995    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 20:48:41.110278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.807998    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808003    2566 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808006    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808009    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808012    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808015    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808019    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808022    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808025    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808028    2566 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808031    2566 flags.go:64] FLAG: --enable-server="true"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808034    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808038    2566 flags.go:64] FLAG: --event-burst="100"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808041    2566 flags.go:64] FLAG: --event-qps="50"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808044    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808046    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808049    2566 flags.go:64] FLAG: --eviction-hard=""
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808053    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808056    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808059    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808061    2566 flags.go:64] FLAG: --eviction-soft=""
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808065    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808069    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808073    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808075    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 20:48:41.113025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808078    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808081    2566 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808085    2566 flags.go:64] FLAG: --feature-gates=""
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808093    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808096    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808099    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808102    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808105    2566 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808110    2566 flags.go:64] FLAG: --help="false"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808113    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808116    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808119    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808122    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808125    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808128    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808131    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808134    2566 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808137    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808140    2566 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808143    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808146    2566 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808149    2566 flags.go:64] FLAG: --kube-reserved=""
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808152    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808154    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 20:48:41.113844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808157    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808160    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808163    2566 flags.go:64] FLAG: --lock-file=""
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808166    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808168    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808173    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808179    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808182    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808185    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808187    2566 flags.go:64] FLAG: --logging-format="text"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808190    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808193    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808196    2566 flags.go:64] FLAG: --manifest-url=""
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808199    2566 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808203    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808206    2566 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808211    2566 flags.go:64] FLAG: --max-pods="110"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808214    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808217    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808220    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808223    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808226    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808229    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808232    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808239    2566 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 20:48:41.114870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808242    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808245    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808248    2566 flags.go:64] FLAG: --pod-cidr=""
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808250    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808256    2566 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808259    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808262    2566 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808265    2566 flags.go:64] FLAG: --port="10250"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808268    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808271    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0731fbd66d6451628"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808274    2566 flags.go:64] FLAG: --qos-reserved=""
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808277    2566 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808281    2566 flags.go:64] FLAG: --register-node="true"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808284    2566 flags.go:64] FLAG: --register-with-taints=""
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808288    2566 flags.go:64] FLAG: --registry-burst="10"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808291    2566 flags.go:64] FLAG: --registry-qps="5"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808294    2566 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808297    2566 flags.go:64] FLAG: --reserved-memory=""
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808300    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808303    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808306    2566 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808309    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808312    2566 flags.go:64] FLAG: --runonce="false"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808315    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808318    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 20:48:41.115577 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808322    2566 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808324    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808327    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808330    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808333    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808336    2566 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808339    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808342    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808345    2566 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808348    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808351    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808353    2566 flags.go:64] FLAG: --system-cgroups=""
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808356    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808361    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808364    2566 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808366    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808371    2566 flags.go:64] FLAG: --tls-min-version=""
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808373    2566 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808376    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808380    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808383    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808386    2566 flags.go:64] FLAG: --v="2"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808390    2566 flags.go:64] FLAG: --version="false"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808395    2566 flags.go:64] FLAG: --vmodule=""
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808401    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 20:48:41.117055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.808404    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808530    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808535    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808538    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808541    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808544    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808549    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808551    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808554    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808557    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808559    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808562    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808564    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808567    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808570    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808574    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808577    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808579    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808582    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 20:48:41.120419 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808584    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808587    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808589    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808592    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808594    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808597    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808599    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808603    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808606    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808609    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808611    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808614    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808616    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808619    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808622    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808624    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808627    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808629    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808632    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808636    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 20:48:41.121217 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808639    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808641    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808644    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808647    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808649    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808652    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808655    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808657    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808660    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808662    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808665    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808668    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808670    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808672    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808675    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808677    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808680    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808682    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808685    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808689    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 20:48:41.121858 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808691    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808694    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808696    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808699    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808701    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808705    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808707    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808710    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808712    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808715    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808718    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808723    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808726    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808729    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808732    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808735    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808738    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808741    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808743    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 20:48:41.122402 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808746    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808749    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808751    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808754    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808775    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808778    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808781    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808783    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808786    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808789    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808792    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808794    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808798    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808801    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808803    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808806    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808808    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808811    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808813    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808817    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 20:48:41.287277 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808820    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.808822    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.809514    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.816067    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.816084    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.816104    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.816110    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816166    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816170    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816174    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816177    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816180    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816183    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816185    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816188    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 20:48:41.288218 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816191    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816194    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816198    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816200    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816203    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816205    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816208    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816211    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816215    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816219    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816222    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816225    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816228    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816232    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816235    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816240    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816243    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816247    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816251    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 20:48:41.289056 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816254    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816257    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816260    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816263    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816266    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816268    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816271    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816274    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816276    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816279    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816281    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816284    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816286    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816289    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816292    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816295    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816299    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816302    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816306    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816309    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 20:48:41.289688 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816312    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816314    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816317    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816320    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816323    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816325    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816328    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816330    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816334    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816337    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816339    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816342    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816345    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816347    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816350    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816353    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816355    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816358    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816361    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 20:48:41.290499 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816363    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816366    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816368    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816371    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816374    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816376    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816379    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816381    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816384    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816386    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816389    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816393    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816396    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816398    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816401    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816403    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816406    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816409    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816411    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816414    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 20:48:41.291085 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816417    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816420    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816423    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816426    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816428    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816431    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816434    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816436    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816439    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816442    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816444    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816447    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816449    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.816454    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816563    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 20:48:41.291650 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816567    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816570    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816573    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816576    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816579    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816582    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816585    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816588    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816592    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816594    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816597    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816600    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816603    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816605    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816608    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816610    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816613    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816615    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816618    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816622    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 20:48:41.292079 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816624    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816627    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816629    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816632    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816635    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816637    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816640    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816643    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816645    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816648    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816653    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816656    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816658    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816661    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816663    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816666    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816668    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816670    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816673    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816675    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 20:48:41.292638 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816679    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816682    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816684    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816687    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816689    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816692    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816695    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816698    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816700    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816702    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816705    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816709    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816711    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816714    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816717    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816719    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816722    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816724    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816727    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816730    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 20:48:41.293182 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816732    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816735    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816738    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816742    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816745    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816747    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816751    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816754    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816773    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816776    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816779    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816782    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816789    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816791    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816794    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816797    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816799    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816802    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816804    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 20:48:41.293732 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816807    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816809    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816812    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816815    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816819    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816821    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816824    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816826    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816829    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816832    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816834    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816837    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816840    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816844    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816847    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816851    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816853    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816856    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 20:48:41.294258 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:40.816858    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 20:48:41.294770 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.816864    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 20:48:41.294770 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.816976    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 20:48:41.294770 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.818964    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 20:48:41.294770 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.820248    2566 server.go:1019] "Starting client certificate rotation"
Jul 06 20:48:41.294770 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.820349    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 20:48:41.294770 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.821112    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 20:48:41.294770 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.822992    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 20:48:41.294770 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.826162    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 20:48:41.294770 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.838871    2566 log.go:25] "Validated CRI v1 runtime API"
Jul 06 20:48:41.294770 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.840963    2566 log.go:25] "Validated CRI v1 image API"
Jul 06 20:48:41.294770 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.842929    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 20:48:41.294770 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.848210    2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bb3259e4-ed95-44e6-bfe1-c3053cc83c33:/dev/nvme0n1p3 ea2fc2c0-6389-4633-a78c-e0f071290a36:/dev/nvme0n1p4]
Jul 06 20:48:41.295126 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.848228    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 20:48:41.295126 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.852043    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.855893    2566 manager.go:216] Machine: {Timestamp:2026-07-06 20:48:40.854879696 +0000 UTC m=+0.330816451 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098673 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23b17960b95ba90d41f5e00b114aef SystemUUID:ec23b179-60b9-5ba9-0d41-f5e00b114aef BootID:bea137bb-43ca-454a-852f-6315c5d36e99 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d6:e1:9b:e5:0b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d6:e1:9b:e5:0b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:e2:53:9e:67:78 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.855997    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.856087    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.857096    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.857123    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-165.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.857754    2566 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.857780    2566 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.857787    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.858564    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.859787    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.862901    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.863014    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.863040    2566 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.863050    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.863065    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.863074    2566 kubelet.go:398] "Adding apiserver pod source"
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.863082    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 20:48:41.295575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.863396    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.863410    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.865572    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.867707    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.867749    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.868796    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.868813    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.868820    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.868827    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.868838    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.868844    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.868854    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.868862    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.868871    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.868876    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.868889    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.868900    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.869776    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.869786    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.872899    2566 server.go:1268] "Started kubelet"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.872987    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.872994    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.873064    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.874499    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:40.874702    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:40.874827    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.874870    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-165.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.875325    2566 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:40.879618    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.881161    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.881166    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:40.881262    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.881732    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.882046    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.882169    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.882348    2566 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.882362    2566 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.882599    2566 factory.go:55] Registering systemd factory
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.882617    2566 factory.go:223] Registration of the systemd container factory successfully
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.882935    2566 factory.go:153] Registering CRI-O factory
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.882952    2566 factory.go:223] Registration of the crio container factory successfully
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.883029    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.883051    2566 factory.go:103] Registering Raw factory
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.883067    2566 manager.go:1196] Started watching for new ooms in manager
Jul 06 20:48:41.296161 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.883542    2566 manager.go:349] Starting recovery of all containers
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:40.882648    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-165.ec2.internal.18bfce6761637be8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-165.ec2.internal,UID:ip-10-0-131-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-165.ec2.internal,},FirstTimestamp:2026-07-06 20:48:40.87286884 +0000 UTC m=+0.348805595,LastTimestamp:2026-07-06 20:48:40.87286884 +0000 UTC m=+0.348805595,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-165.ec2.internal,}"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:40.886980    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:40.887078    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.893953    2566 manager.go:354] Recovery completed
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.898049    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.899947    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.899991    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.900005    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.901510    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.901520    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.901538    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.901587    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.902356    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.902385    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.902404    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.902818    2566 policy_none.go:49] "None policy: Start"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.902841    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.902855    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:40.902802    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-165.ec2.internal.18bfce676301129e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-165.ec2.internal,UID:ip-10-0-131-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-165.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-165.ec2.internal,},FirstTimestamp:2026-07-06 20:48:40.89997379 +0000 UTC m=+0.375910548,LastTimestamp:2026-07-06 20:48:40.89997379 +0000 UTC m=+0.375910548,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-165.ec2.internal,}"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.903902    2566 policy_none.go:47] "Start"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.906627    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cmpp4"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.911578    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cmpp4"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:40.913055    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-165.ec2.internal.18bfce676301707e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-165.ec2.internal,UID:ip-10-0-131-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-165.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-165.ec2.internal,},FirstTimestamp:2026-07-06 20:48:40.899997822 +0000 UTC m=+0.375934581,LastTimestamp:2026-07-06 20:48:40.899997822 +0000 UTC m=+0.375934581,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-165.ec2.internal,}"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.976360    2566 manager.go:337] "Starting Device Plugin manager"
Jul 06 20:48:41.365551 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:40.976391    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.976401    2566 server.go:87] "Starting device plugin registration server"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.976671    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.976681    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.976823    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.977223    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.977235    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:40.977382    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:40.977454    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.984599    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.985940    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.985975    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.985993    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:40.986034    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:40.987986    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.076827    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.077895    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.077927    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.077941    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.077969    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.086142    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal"]
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.086222    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.087737    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:41.087754    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-165.ec2.internal\": node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.089727    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.089754    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.089781    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.091229    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.091338    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.091374    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.091989    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.092014    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.092019    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.092029    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.092041    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.092050    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.093173    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.093199    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.093889    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.093912    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.093922    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:41.117881    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:41.367024 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:41.136482    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-165.ec2.internal\" not found" node="ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.369006 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:41.145298    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-165.ec2.internal\" not found" node="ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.369006 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:41.218521    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:41.369006 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.283964    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fce54f2a107c2f172eea3a387f3af721-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal\" (UID: \"fce54f2a107c2f172eea3a387f3af721\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.369006 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.284000    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fce54f2a107c2f172eea3a387f3af721-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal\" (UID: \"fce54f2a107c2f172eea3a387f3af721\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.369006 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.284027    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e45831538ee9d5022543246cbe3ad035-config\") pod \"kube-apiserver-proxy-ip-10-0-131-165.ec2.internal\" (UID: \"e45831538ee9d5022543246cbe3ad035\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.369006 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:41.319028    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:41.435476 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.386659    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fce54f2a107c2f172eea3a387f3af721-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal\" (UID: \"fce54f2a107c2f172eea3a387f3af721\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.435476 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.386688    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fce54f2a107c2f172eea3a387f3af721-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal\" (UID: \"fce54f2a107c2f172eea3a387f3af721\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.435476 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.386706    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e45831538ee9d5022543246cbe3ad035-config\") pod \"kube-apiserver-proxy-ip-10-0-131-165.ec2.internal\" (UID: \"e45831538ee9d5022543246cbe3ad035\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.435476 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.386779    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fce54f2a107c2f172eea3a387f3af721-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal\" (UID: \"fce54f2a107c2f172eea3a387f3af721\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.435476 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.386786    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fce54f2a107c2f172eea3a387f3af721-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal\" (UID: \"fce54f2a107c2f172eea3a387f3af721\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.435476 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.386867    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e45831538ee9d5022543246cbe3ad035-config\") pod \"kube-apiserver-proxy-ip-10-0-131-165.ec2.internal\" (UID: \"e45831538ee9d5022543246cbe3ad035\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.435476 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:41.419808    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:41.527664 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.441971    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.527664 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.448117    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:41.527664 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:41.520581    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:41.695795 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:41.621151    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:41.847870 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:41.721645    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:41.847870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.820222    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 20:48:41.847870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:41.820356    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 20:48:41.847870 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:41.822368    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:42.995997 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:42.923330    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 20:43:41 +0000 UTC" deadline="2028-02-23 04:58:12.911359162 +0000 UTC"
Jul 06 20:48:42.995997 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:42.923367    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14312h9m29.98799675s"
Jul 06 20:48:42.995997 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:42.932496    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:43.143158 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:43.033411    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:43.143158 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.083224    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 20:48:43.143158 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:43.086956    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45831538ee9d5022543246cbe3ad035.slice/crio-c8f2eb37b812704e51b5569d3812e41b558596a2f4f82fa65cb81714df9d350c WatchSource:0}: Error finding container c8f2eb37b812704e51b5569d3812e41b558596a2f4f82fa65cb81714df9d350c: Status 404 returned error can't find the container with id c8f2eb37b812704e51b5569d3812e41b558596a2f4f82fa65cb81714df9d350c
Jul 06 20:48:43.143158 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.088025    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:48:43.143158 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.089640    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:48:43.143158 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.089676    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:48:43.143158 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.089689    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:48:43.143158 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.093012    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:48:43.143158 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.110742    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 20:48:43.143158 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:43.133777    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:43.270930 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:43.233889    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:43.400365 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:43.334476    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:43.500256 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:43.435128    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-165.ec2.internal\" not found"
Jul 06 20:48:43.500256 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.438241    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 20:48:43.500256 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.491952    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:43.579730 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.505385    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 20:48:43.579730 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.510385    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal"
Jul 06 20:48:43.579730 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.516840    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 20:48:43.667235 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.643273    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.873611    2566 apiserver.go:52] "Watching apiserver"
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.881162    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.881530    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-mrll5","openshift-image-registry/node-ca-qxhpq","openshift-network-diagnostics/network-check-target-x2kkv","openshift-network-operator/iptables-alerter-kbwmh","openshift-ovn-kubernetes/ovnkube-node-psw72","kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w","openshift-cluster-node-tuning-operator/tuned-rhjwc","openshift-dns/node-resolver-ptn9v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal","openshift-multus/multus-additional-cni-plugins-hjc94","openshift-multus/multus-xh69c","openshift-multus/network-metrics-daemon-w4sfv","kube-system/global-pull-secret-syncer-7lk82"]
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.883300    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.885230    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:43.885309    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.886337    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:43.886393    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.887842    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kbwmh"
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.889760    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.891204    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.891264    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.891310    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.891440    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pqdzp\""
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.891545    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:43.891626    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.891896    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 20:48:43.896027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.895426    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ptn9v"
Jul 06 20:48:43.897733 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.896057    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 20:48:43.899074 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.898044    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mrll5"
Jul 06 20:48:43.899074 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.898756    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hq98n\""
Jul 06 20:48:43.899074 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.899047    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 20:48:43.901884 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.900068    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qxhpq"
Jul 06 20:48:43.902466 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.902425    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:43.904049 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.904030    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:43.905197 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905174    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-var-lib-openvswitch\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.905319 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905207    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-ovnkube-config\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.905319 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905225    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-ovn-node-metrics-cert\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.905319 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905250    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7dpd\" (UniqueName: \"kubernetes.io/projected/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-kube-api-access-v7dpd\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.905319 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905301    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d9a37c14-16e3-47f7-ae66-5a1d840daa48-iptables-alerter-script\") pod \"iptables-alerter-kbwmh\" (UID: \"d9a37c14-16e3-47f7-ae66-5a1d840daa48\") " pod="openshift-network-operator/iptables-alerter-kbwmh"
Jul 06 20:48:43.905622 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905340    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-etc-openvswitch\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.905622 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905376    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-ovnkube-script-lib\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.905622 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905408    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pws8j\" (UniqueName: \"kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j\") pod \"network-check-target-x2kkv\" (UID: \"748cbc30-cb4a-4a02-bda9-58b147b12ce9\") " pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:43.905622 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905439    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:43.905622 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905479    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-etc-selinux\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:43.905622 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905504    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jd67\" (UniqueName: \"kubernetes.io/projected/02732679-8a59-416d-ac6d-6de4e7e5ab39-kube-api-access-9jd67\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:43.905622 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905528    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1236b937-e2b7-41ac-a612-142a60a86806-hosts-file\") pod \"node-resolver-ptn9v\" (UID: \"1236b937-e2b7-41ac-a612-142a60a86806\") " pod="openshift-dns/node-resolver-ptn9v"
Jul 06 20:48:43.905622 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905550    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1236b937-e2b7-41ac-a612-142a60a86806-tmp-dir\") pod \"node-resolver-ptn9v\" (UID: \"1236b937-e2b7-41ac-a612-142a60a86806\") " pod="openshift-dns/node-resolver-ptn9v"
Jul 06 20:48:43.905622 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905572    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-run-openvswitch\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.905622 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905594    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-node-log\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.905622 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905615    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04009265-aa63-49ba-a715-0a555264ccd7-dbus\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905636    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-device-dir\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905642    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xh69c"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905658    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-run-netns\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905681    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-env-overrides\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905726    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-slash\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905761    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-cni-bin\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905785    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-socket-dir\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905807    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-registration-dir\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905832    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-sys-fs\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905855    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47862\" (UniqueName: \"kubernetes.io/projected/997ba244-1db8-4c94-bb4e-eda91480d511-kube-api-access-47862\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905878    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wngf5\" (UniqueName: \"kubernetes.io/projected/d9a37c14-16e3-47f7-ae66-5a1d840daa48-kube-api-access-wngf5\") pod \"iptables-alerter-kbwmh\" (UID: \"d9a37c14-16e3-47f7-ae66-5a1d840daa48\") " pod="openshift-network-operator/iptables-alerter-kbwmh"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905914    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-systemd-units\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905946    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-run-systemd\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905971    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.905995    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04009265-aa63-49ba-a715-0a555264ccd7-kubelet-config\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.906017    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9a37c14-16e3-47f7-ae66-5a1d840daa48-host-slash\") pod \"iptables-alerter-kbwmh\" (UID: \"d9a37c14-16e3-47f7-ae66-5a1d840daa48\") " pod="openshift-network-operator/iptables-alerter-kbwmh"
Jul 06 20:48:43.906213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.906047    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsbf4\" (UniqueName: \"kubernetes.io/projected/1236b937-e2b7-41ac-a612-142a60a86806-kube-api-access-gsbf4\") pod \"node-resolver-ptn9v\" (UID: \"1236b937-e2b7-41ac-a612-142a60a86806\") " pod="openshift-dns/node-resolver-ptn9v"
Jul 06 20:48:43.906866 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.906076    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-kubelet\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.906866 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.906098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-run-ovn\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.906866 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.906120    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-log-socket\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.906866 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.906144    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-cni-netd\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.906866 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.906174    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:43.906866 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.906201    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:43.906866 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.906235    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:43.912444 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.912229    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 20:48:43.928825 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.928781    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 20:48:43.928986 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.928825    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 20:48:43.929075 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.929055    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 20:48:43.929164 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.929148    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 20:48:43.929252 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.929238    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 20:48:43.929390 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.929376    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 20:48:43.929474 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.929418    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 20:48:43.929526 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.929506    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 20:48:43.933421 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.933345    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 20:48:43.934415 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.933552    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 20:48:43.934415 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.933695    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 20:48:43.934415 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.933836    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 20:48:43.934415 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934019    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 20:48:43.934415 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934201    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 20:48:43.934415 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934294    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 20:48:43.934415 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934330    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 20:48:43.934415 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934357    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 20:48:43.934836 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934420    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xlrgk\""
Jul 06 20:48:43.934836 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934511    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 20:48:43.934836 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934560    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pnmdp\""
Jul 06 20:48:43.934836 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934596    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-phll7\""
Jul 06 20:48:43.934836 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934634    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vmbml\""
Jul 06 20:48:43.934836 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934678    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4j5j7\""
Jul 06 20:48:43.934836 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934739    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 20:48:43.934836 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934514    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 20:48:43.934836 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934802    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 20:48:43.934836 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.934836    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5cvm7\""
Jul 06 20:48:43.946678 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.946620    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9c8zm\""
Jul 06 20:48:43.953924 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.953902    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 20:48:43.993264 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:43.993235    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 20:48:44.001541 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.001486    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal" event={"ID":"fce54f2a107c2f172eea3a387f3af721","Type":"ContainerStarted","Data":"0e698fc722b41e353be7f7e261f55b5d99000992d3968169a52be629b28ef966"}
Jul 06 20:48:44.003174 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.003145    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal" event={"ID":"e45831538ee9d5022543246cbe3ad035","Type":"ContainerStarted","Data":"c8f2eb37b812704e51b5569d3812e41b558596a2f4f82fa65cb81714df9d350c"}
Jul 06 20:48:44.007409 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007374    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:44.007535 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007413    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-etc-selinux\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.007535 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007439    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jd67\" (UniqueName: \"kubernetes.io/projected/02732679-8a59-416d-ac6d-6de4e7e5ab39-kube-api-access-9jd67\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.007535 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007487    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-run-openvswitch\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.007535 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007511    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-node-log\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.007730 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007538    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e267afa3-e061-4f08-bc2c-4c90e6b26753-cni-binary-copy\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.007730 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.007551    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:48:44.007730 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007577    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-run-netns\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.007730 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007582    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-etc-selinux\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.007730 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.007620    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret podName:04009265-aa63-49ba-a715-0a555264ccd7 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:44.507599403 +0000 UTC m=+2.973840636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret") pod "global-pull-secret-syncer-7lk82" (UID: "04009265-aa63-49ba-a715-0a555264ccd7") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:48:44.007730 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007621    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-node-log\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.007730 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007618    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-run-openvswitch\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.007730 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007641    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04009265-aa63-49ba-a715-0a555264ccd7-dbus\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:44.007730 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007669    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-env-overrides\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.007730 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007691    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-sysconfig\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.007730 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007712    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-sysctl-d\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007735    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-run-k8s-cni-cncf-io\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007772    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt5n7\" (UniqueName: \"kubernetes.io/projected/8f813d5c-13a4-485c-b5ab-5f26fd9f84fe-kube-api-access-jt5n7\") pod \"node-ca-qxhpq\" (UID: \"8f813d5c-13a4-485c-b5ab-5f26fd9f84fe\") " pod="openshift-image-registry/node-ca-qxhpq"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007797    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-slash\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007816    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04009265-aa63-49ba-a715-0a555264ccd7-dbus\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007822    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-cni-bin\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007856    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/089e53a7-3267-465e-8abb-52b283f3d756-cnibin\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007869    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-cni-bin\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007883    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/089e53a7-3267-465e-8abb-52b283f3d756-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007893    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-slash\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007907    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-modprobe-d\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007934    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-etc-kubernetes\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007956    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-sysctl-conf\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.007981    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-socket-dir\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008005    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-registration-dir\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008060    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-registration-dir\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008087    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-sys-fs\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.008155 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008100    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-socket-dir\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008126    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47862\" (UniqueName: \"kubernetes.io/projected/997ba244-1db8-4c94-bb4e-eda91480d511-kube-api-access-47862\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008151    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wngf5\" (UniqueName: \"kubernetes.io/projected/d9a37c14-16e3-47f7-ae66-5a1d840daa48-kube-api-access-wngf5\") pod \"iptables-alerter-kbwmh\" (UID: \"d9a37c14-16e3-47f7-ae66-5a1d840daa48\") " pod="openshift-network-operator/iptables-alerter-kbwmh"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008157    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-sys-fs\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008176    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-systemd-units\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008211    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-var-lib-cni-bin\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008236    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04009265-aa63-49ba-a715-0a555264ccd7-kubelet-config\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008262    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsbf4\" (UniqueName: \"kubernetes.io/projected/1236b937-e2b7-41ac-a612-142a60a86806-kube-api-access-gsbf4\") pod \"node-resolver-ptn9v\" (UID: \"1236b937-e2b7-41ac-a612-142a60a86806\") " pod="openshift-dns/node-resolver-ptn9v"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008287    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-run-ovn\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008296    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-env-overrides\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008312    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008341    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-ovn-node-metrics-cert\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008347    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-systemd-units\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008386    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-run-multus-certs\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008414    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-run-ovn\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008427    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008478    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.008934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008473    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-ovnkube-config\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008428    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04009265-aa63-49ba-a715-0a555264ccd7-kubelet-config\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008497    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008514    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7dpd\" (UniqueName: \"kubernetes.io/projected/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-kube-api-access-v7dpd\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008552    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-system-cni-dir\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008575    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8f813d5c-13a4-485c-b5ab-5f26fd9f84fe-serviceca\") pod \"node-ca-qxhpq\" (UID: \"8f813d5c-13a4-485c-b5ab-5f26fd9f84fe\") " pod="openshift-image-registry/node-ca-qxhpq"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008600    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sq2t\" (UniqueName: \"kubernetes.io/projected/089e53a7-3267-465e-8abb-52b283f3d756-kube-api-access-6sq2t\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008631    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-run\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.008821    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009064    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-ovnkube-config\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009139    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-var-lib-kubelet\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009191    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5002d46b-a551-424b-bd08-f61d84e3c45b-konnectivity-ca\") pod \"konnectivity-agent-mrll5\" (UID: \"5002d46b-a551-424b-bd08-f61d84e3c45b\") " pod="kube-system/konnectivity-agent-mrll5"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009216    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/089e53a7-3267-465e-8abb-52b283f3d756-os-release\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009238    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-kubernetes\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009259    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-host\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009287    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-multus-cni-dir\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009312    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl867\" (UniqueName: \"kubernetes.io/projected/e267afa3-e061-4f08-bc2c-4c90e6b26753-kube-api-access-rl867\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.009504 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009334    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-systemd\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009360    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1236b937-e2b7-41ac-a612-142a60a86806-hosts-file\") pod \"node-resolver-ptn9v\" (UID: \"1236b937-e2b7-41ac-a612-142a60a86806\") " pod="openshift-dns/node-resolver-ptn9v"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009384    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1236b937-e2b7-41ac-a612-142a60a86806-tmp-dir\") pod \"node-resolver-ptn9v\" (UID: \"1236b937-e2b7-41ac-a612-142a60a86806\") " pod="openshift-dns/node-resolver-ptn9v"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/089e53a7-3267-465e-8abb-52b283f3d756-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009433    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-device-dir\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009472    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-run-netns\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009479    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1236b937-e2b7-41ac-a612-142a60a86806-hosts-file\") pod \"node-resolver-ptn9v\" (UID: \"1236b937-e2b7-41ac-a612-142a60a86806\") " pod="openshift-dns/node-resolver-ptn9v"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009497    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/089e53a7-3267-465e-8abb-52b283f3d756-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009521    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-multus-conf-dir\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009544    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/02732679-8a59-416d-ac6d-6de4e7e5ab39-device-dir\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009544    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/089e53a7-3267-465e-8abb-52b283f3d756-cni-binary-copy\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009588    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-lib-modules\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009611    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-var-lib-kubelet\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009622    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-run-netns\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009658    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-cnibin\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009684    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e267afa3-e061-4f08-bc2c-4c90e6b26753-multus-daemon-config\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009694    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1236b937-e2b7-41ac-a612-142a60a86806-tmp-dir\") pod \"node-resolver-ptn9v\" (UID: \"1236b937-e2b7-41ac-a612-142a60a86806\") " pod="openshift-dns/node-resolver-ptn9v"
Jul 06 20:48:44.010228 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009712    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-cni-netd\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009742    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-run-systemd\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009766    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009790    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/071fb91e-f1b8-476f-9b44-0297a6661a85-tmp\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009802    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-cni-netd\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009823    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-var-lib-cni-multus\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009838    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-run-systemd\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009848    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5002d46b-a551-424b-bd08-f61d84e3c45b-agent-certs\") pod \"konnectivity-agent-mrll5\" (UID: \"5002d46b-a551-424b-bd08-f61d84e3c45b\") " pod="kube-system/konnectivity-agent-mrll5"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009873    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9a37c14-16e3-47f7-ae66-5a1d840daa48-host-slash\") pod \"iptables-alerter-kbwmh\" (UID: \"d9a37c14-16e3-47f7-ae66-5a1d840daa48\") " pod="openshift-network-operator/iptables-alerter-kbwmh"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009876    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009898    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-kubelet\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009921    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9a37c14-16e3-47f7-ae66-5a1d840daa48-host-slash\") pod \"iptables-alerter-kbwmh\" (UID: \"d9a37c14-16e3-47f7-ae66-5a1d840daa48\") " pod="openshift-network-operator/iptables-alerter-kbwmh"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009952    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-log-socket\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.009967    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-host-kubelet\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010014    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-os-release\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010039    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-multus-socket-dir-parent\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010090    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:44.010944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-var-lib-openvswitch\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010151    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-sys\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010196    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-tuned\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010220    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plp55\" (UniqueName: \"kubernetes.io/projected/071fb91e-f1b8-476f-9b44-0297a6661a85-kube-api-access-plp55\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010243    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-hostroot\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010268    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d9a37c14-16e3-47f7-ae66-5a1d840daa48-iptables-alerter-script\") pod \"iptables-alerter-kbwmh\" (UID: \"d9a37c14-16e3-47f7-ae66-5a1d840daa48\") " pod="openshift-network-operator/iptables-alerter-kbwmh"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010294    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-etc-openvswitch\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010319    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-ovnkube-script-lib\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010393    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/089e53a7-3267-465e-8abb-52b283f3d756-system-cni-dir\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010417    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f813d5c-13a4-485c-b5ab-5f26fd9f84fe-host\") pod \"node-ca-qxhpq\" (UID: \"8f813d5c-13a4-485c-b5ab-5f26fd9f84fe\") " pod="openshift-image-registry/node-ca-qxhpq"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010443    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pws8j\" (UniqueName: \"kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j\") pod \"network-check-target-x2kkv\" (UID: \"748cbc30-cb4a-4a02-bda9-58b147b12ce9\") " pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010712    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-etc-openvswitch\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010768    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-log-socket\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.010839    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.010891    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs podName:997ba244-1db8-4c94-bb4e-eda91480d511 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:44.51087168 +0000 UTC m=+2.977112920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs") pod "network-metrics-daemon-w4sfv" (UID: "997ba244-1db8-4c94-bb4e-eda91480d511") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010943    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d9a37c14-16e3-47f7-ae66-5a1d840daa48-iptables-alerter-script\") pod \"iptables-alerter-kbwmh\" (UID: \"d9a37c14-16e3-47f7-ae66-5a1d840daa48\") " pod="openshift-network-operator/iptables-alerter-kbwmh"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010928    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-var-lib-openvswitch\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.011671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.010991    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-ovnkube-script-lib\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.012967 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.012947    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-ovn-node-metrics-cert\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.016379 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.016358    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-klnk2"
Jul 06 20:48:44.026402 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.026378    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-klnk2"
Jul 06 20:48:44.032469 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.032426    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47862\" (UniqueName: \"kubernetes.io/projected/997ba244-1db8-4c94-bb4e-eda91480d511-kube-api-access-47862\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:44.036051 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.036027    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:48:44.036153 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.036057    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:48:44.036153 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.036072    2566 projected.go:196] Error preparing data for projected volume kube-api-access-pws8j for pod openshift-network-diagnostics/network-check-target-x2kkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:48:44.036153 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.036144    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j podName:748cbc30-cb4a-4a02-bda9-58b147b12ce9 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:44.536125064 +0000 UTC m=+3.002366299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pws8j" (UniqueName: "kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j") pod "network-check-target-x2kkv" (UID: "748cbc30-cb4a-4a02-bda9-58b147b12ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:48:44.037700 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.037679    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jd67\" (UniqueName: \"kubernetes.io/projected/02732679-8a59-416d-ac6d-6de4e7e5ab39-kube-api-access-9jd67\") pod \"aws-ebs-csi-driver-node-t4g9w\" (UID: \"02732679-8a59-416d-ac6d-6de4e7e5ab39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.037826 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.037808    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsbf4\" (UniqueName: \"kubernetes.io/projected/1236b937-e2b7-41ac-a612-142a60a86806-kube-api-access-gsbf4\") pod \"node-resolver-ptn9v\" (UID: \"1236b937-e2b7-41ac-a612-142a60a86806\") " pod="openshift-dns/node-resolver-ptn9v"
Jul 06 20:48:44.037932 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.037912    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7dpd\" (UniqueName: \"kubernetes.io/projected/e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7-kube-api-access-v7dpd\") pod \"ovnkube-node-psw72\" (UID: \"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.038244 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.038227    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wngf5\" (UniqueName: \"kubernetes.io/projected/d9a37c14-16e3-47f7-ae66-5a1d840daa48-kube-api-access-wngf5\") pod \"iptables-alerter-kbwmh\" (UID: \"d9a37c14-16e3-47f7-ae66-5a1d840daa48\") " pod="openshift-network-operator/iptables-alerter-kbwmh"
Jul 06 20:48:44.111489 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111436    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-var-lib-cni-bin\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.111669 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111503    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-run-multus-certs\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.111669 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111563    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-var-lib-cni-bin\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.111669 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111573    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-run-multus-certs\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.111669 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111625    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-system-cni-dir\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.111889 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111672    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8f813d5c-13a4-485c-b5ab-5f26fd9f84fe-serviceca\") pod \"node-ca-qxhpq\" (UID: \"8f813d5c-13a4-485c-b5ab-5f26fd9f84fe\") " pod="openshift-image-registry/node-ca-qxhpq"
Jul 06 20:48:44.111889 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111705    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sq2t\" (UniqueName: \"kubernetes.io/projected/089e53a7-3267-465e-8abb-52b283f3d756-kube-api-access-6sq2t\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.111889 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111734    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-run\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.111889 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111751    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-system-cni-dir\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.111889 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111761    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-var-lib-kubelet\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.111889 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111788    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5002d46b-a551-424b-bd08-f61d84e3c45b-konnectivity-ca\") pod \"konnectivity-agent-mrll5\" (UID: \"5002d46b-a551-424b-bd08-f61d84e3c45b\") " pod="kube-system/konnectivity-agent-mrll5"
Jul 06 20:48:44.111889 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111814    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/089e53a7-3267-465e-8abb-52b283f3d756-os-release\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.111889 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111838    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-kubernetes\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.111889 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111864    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-host\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.111889 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111887    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-multus-cni-dir\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111913    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl867\" (UniqueName: \"kubernetes.io/projected/e267afa3-e061-4f08-bc2c-4c90e6b26753-kube-api-access-rl867\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111937    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-systemd\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.111966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/089e53a7-3267-465e-8abb-52b283f3d756-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112012    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/089e53a7-3267-465e-8abb-52b283f3d756-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112039    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-multus-conf-dir\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112073    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/089e53a7-3267-465e-8abb-52b283f3d756-cni-binary-copy\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112098    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-lib-modules\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-var-lib-kubelet\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-cnibin\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112171    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e267afa3-e061-4f08-bc2c-4c90e6b26753-multus-daemon-config\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112202    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/071fb91e-f1b8-476f-9b44-0297a6661a85-tmp\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112228    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-var-lib-cni-multus\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112254    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5002d46b-a551-424b-bd08-f61d84e3c45b-agent-certs\") pod \"konnectivity-agent-mrll5\" (UID: \"5002d46b-a551-424b-bd08-f61d84e3c45b\") " pod="kube-system/konnectivity-agent-mrll5"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112291    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-os-release\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112317    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-multus-socket-dir-parent\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112328    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8f813d5c-13a4-485c-b5ab-5f26fd9f84fe-serviceca\") pod \"node-ca-qxhpq\" (UID: \"8f813d5c-13a4-485c-b5ab-5f26fd9f84fe\") " pod="openshift-image-registry/node-ca-qxhpq"
Jul 06 20:48:44.112362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112360    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-sys\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112389    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-tuned\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112414    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plp55\" (UniqueName: \"kubernetes.io/projected/071fb91e-f1b8-476f-9b44-0297a6661a85-kube-api-access-plp55\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112440    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-hostroot\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112489    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/089e53a7-3267-465e-8abb-52b283f3d756-system-cni-dir\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112521    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f813d5c-13a4-485c-b5ab-5f26fd9f84fe-host\") pod \"node-ca-qxhpq\" (UID: \"8f813d5c-13a4-485c-b5ab-5f26fd9f84fe\") " pod="openshift-image-registry/node-ca-qxhpq"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112590    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e267afa3-e061-4f08-bc2c-4c90e6b26753-cni-binary-copy\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112619    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-run-netns\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112652    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-sysconfig\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112687    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-sysctl-d\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112714    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-run-k8s-cni-cncf-io\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112743    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt5n7\" (UniqueName: \"kubernetes.io/projected/8f813d5c-13a4-485c-b5ab-5f26fd9f84fe-kube-api-access-jt5n7\") pod \"node-ca-qxhpq\" (UID: \"8f813d5c-13a4-485c-b5ab-5f26fd9f84fe\") " pod="openshift-image-registry/node-ca-qxhpq"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112793    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/089e53a7-3267-465e-8abb-52b283f3d756-cnibin\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112822    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/089e53a7-3267-465e-8abb-52b283f3d756-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112853    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-modprobe-d\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112879    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-etc-kubernetes\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112886    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/089e53a7-3267-465e-8abb-52b283f3d756-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.113200 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-sysctl-conf\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112946    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-run\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.112984    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-var-lib-kubelet\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113117    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-sysctl-conf\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113168    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-multus-conf-dir\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113358    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5002d46b-a551-424b-bd08-f61d84e3c45b-konnectivity-ca\") pod \"konnectivity-agent-mrll5\" (UID: \"5002d46b-a551-424b-bd08-f61d84e3c45b\") " pod="kube-system/konnectivity-agent-mrll5"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113421    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/089e53a7-3267-465e-8abb-52b283f3d756-os-release\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113520    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-kubernetes\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113556    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-systemd\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113568    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-multus-cni-dir\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113630    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/089e53a7-3267-465e-8abb-52b283f3d756-cnibin\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113672    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/089e53a7-3267-465e-8abb-52b283f3d756-cni-binary-copy\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113674    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-run-k8s-cni-cncf-io\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113698    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-sysctl-d\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113752    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/089e53a7-3267-465e-8abb-52b283f3d756-system-cni-dir\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113782    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-host\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113796    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f813d5c-13a4-485c-b5ab-5f26fd9f84fe-host\") pod \"node-ca-qxhpq\" (UID: \"8f813d5c-13a4-485c-b5ab-5f26fd9f84fe\") " pod="openshift-image-registry/node-ca-qxhpq"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113804    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-modprobe-d\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.114033 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113832    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-var-lib-cni-multus\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113867    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-sysconfig\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113913    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-var-lib-kubelet\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113926    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-cnibin\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113930    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-lib-modules\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113949    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-etc-kubernetes\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.113993    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-multus-socket-dir-parent\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.114040    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/089e53a7-3267-465e-8abb-52b283f3d756-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.114124    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-hostroot\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.114163    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/071fb91e-f1b8-476f-9b44-0297a6661a85-sys\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.114371    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e267afa3-e061-4f08-bc2c-4c90e6b26753-cni-binary-copy\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.114383    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e267afa3-e061-4f08-bc2c-4c90e6b26753-multus-daemon-config\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.114436    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-host-run-netns\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.114546    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e267afa3-e061-4f08-bc2c-4c90e6b26753-os-release\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.114977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.114615    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/089e53a7-3267-465e-8abb-52b283f3d756-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.116707 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.116664    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/071fb91e-f1b8-476f-9b44-0297a6661a85-etc-tuned\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.116843 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.116823    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/071fb91e-f1b8-476f-9b44-0297a6661a85-tmp\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.117016 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.116997    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5002d46b-a551-424b-bd08-f61d84e3c45b-agent-certs\") pod \"konnectivity-agent-mrll5\" (UID: \"5002d46b-a551-424b-bd08-f61d84e3c45b\") " pod="kube-system/konnectivity-agent-mrll5"
Jul 06 20:48:44.127254 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.127230    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl867\" (UniqueName: \"kubernetes.io/projected/e267afa3-e061-4f08-bc2c-4c90e6b26753-kube-api-access-rl867\") pod \"multus-xh69c\" (UID: \"e267afa3-e061-4f08-bc2c-4c90e6b26753\") " pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.127585 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.127566    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sq2t\" (UniqueName: \"kubernetes.io/projected/089e53a7-3267-465e-8abb-52b283f3d756-kube-api-access-6sq2t\") pod \"multus-additional-cni-plugins-hjc94\" (UID: \"089e53a7-3267-465e-8abb-52b283f3d756\") " pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.127997 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.127976    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt5n7\" (UniqueName: \"kubernetes.io/projected/8f813d5c-13a4-485c-b5ab-5f26fd9f84fe-kube-api-access-jt5n7\") pod \"node-ca-qxhpq\" (UID: \"8f813d5c-13a4-485c-b5ab-5f26fd9f84fe\") " pod="openshift-image-registry/node-ca-qxhpq"
Jul 06 20:48:44.128777 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.128752    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plp55\" (UniqueName: \"kubernetes.io/projected/071fb91e-f1b8-476f-9b44-0297a6661a85-kube-api-access-plp55\") pod \"tuned-rhjwc\" (UID: \"071fb91e-f1b8-476f-9b44-0297a6661a85\") " pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.201320 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.201243    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w"
Jul 06 20:48:44.212584 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.212563    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kbwmh"
Jul 06 20:48:44.224700 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.224669    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:48:44.238403 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.238383    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ptn9v"
Jul 06 20:48:44.240107 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.240089    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mrll5"
Jul 06 20:48:44.247167 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.247149    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qxhpq"
Jul 06 20:48:44.254752 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.254726    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rhjwc"
Jul 06 20:48:44.262285 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.262265    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hjc94"
Jul 06 20:48:44.268926 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.268903    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xh69c"
Jul 06 20:48:44.515388 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.515293    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:44.515388 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.515344    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:44.515616 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.515447    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:48:44.515616 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.515447    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:48:44.515616 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.515528    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret podName:04009265-aa63-49ba-a715-0a555264ccd7 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:45.515513099 +0000 UTC m=+3.981754330 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret") pod "global-pull-secret-syncer-7lk82" (UID: "04009265-aa63-49ba-a715-0a555264ccd7") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:48:44.515616 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.515543    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs podName:997ba244-1db8-4c94-bb4e-eda91480d511 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:45.515537096 +0000 UTC m=+3.981778328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs") pod "network-metrics-daemon-w4sfv" (UID: "997ba244-1db8-4c94-bb4e-eda91480d511") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:48:44.615913 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.615880    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pws8j\" (UniqueName: \"kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j\") pod \"network-check-target-x2kkv\" (UID: \"748cbc30-cb4a-4a02-bda9-58b147b12ce9\") " pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:44.616097 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.616041    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:48:44.616097 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.616057    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:48:44.616097 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.616065    2566 projected.go:196] Error preparing data for projected volume kube-api-access-pws8j for pod openshift-network-diagnostics/network-check-target-x2kkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:48:44.616249 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.616130    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j podName:748cbc30-cb4a-4a02-bda9-58b147b12ce9 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:45.616110317 +0000 UTC m=+4.082351556 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pws8j" (UniqueName: "kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j") pod "network-check-target-x2kkv" (UID: "748cbc30-cb4a-4a02-bda9-58b147b12ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:48:44.996635 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:44.996561    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:44.996770 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:44.996676    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:48:45.029816 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:45.029763    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 20:43:44 +0000 UTC" deadline="2028-03-25 04:04:14.34034517 +0000 UTC"
Jul 06 20:48:45.029816 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:45.029800    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15055h15m29.310549759s"
Jul 06 20:48:45.138881 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:45.138855    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1236b937_e2b7_41ac_a612_142a60a86806.slice/crio-5db4896d69ba2d84455065d9a64fd6c2ae4b51e76bcc78dac5442a469bd6e64b WatchSource:0}: Error finding container 5db4896d69ba2d84455065d9a64fd6c2ae4b51e76bcc78dac5442a469bd6e64b: Status 404 returned error can't find the container with id 5db4896d69ba2d84455065d9a64fd6c2ae4b51e76bcc78dac5442a469bd6e64b
Jul 06 20:48:45.140412 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:45.140383    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02732679_8a59_416d_ac6d_6de4e7e5ab39.slice/crio-fec588fdd19b70fce790c4abfc58b1c31fbcd35507b0a9103c977228ede13b35 WatchSource:0}: Error finding container fec588fdd19b70fce790c4abfc58b1c31fbcd35507b0a9103c977228ede13b35: Status 404 returned error can't find the container with id fec588fdd19b70fce790c4abfc58b1c31fbcd35507b0a9103c977228ede13b35
Jul 06 20:48:45.142568 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:45.142545    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode267afa3_e061_4f08_bc2c_4c90e6b26753.slice/crio-02f2333e6216646837f93a1e34d028cb6cfe4552ceb2f730a0b819a2056efadb WatchSource:0}: Error finding container 02f2333e6216646837f93a1e34d028cb6cfe4552ceb2f730a0b819a2056efadb: Status 404 returned error can't find the container with id 02f2333e6216646837f93a1e34d028cb6cfe4552ceb2f730a0b819a2056efadb
Jul 06 20:48:45.143716 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:45.143636    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod071fb91e_f1b8_476f_9b44_0297a6661a85.slice/crio-bc3c4750b8ce2ee46792feb6a83b1883688f80d3a245838a3cf1adf8179c151f WatchSource:0}: Error finding container bc3c4750b8ce2ee46792feb6a83b1883688f80d3a245838a3cf1adf8179c151f: Status 404 returned error can't find the container with id bc3c4750b8ce2ee46792feb6a83b1883688f80d3a245838a3cf1adf8179c151f
Jul 06 20:48:45.146597 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:45.146576    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode15f043c_3e63_4c20_a0c7_a2fabbc1b9d7.slice/crio-9f4a3f90b6528225a1d7ca0d1929e75cb337c8f65d9adf419d3c7ef4a5a7fc5b WatchSource:0}: Error finding container 9f4a3f90b6528225a1d7ca0d1929e75cb337c8f65d9adf419d3c7ef4a5a7fc5b: Status 404 returned error can't find the container with id 9f4a3f90b6528225a1d7ca0d1929e75cb337c8f65d9adf419d3c7ef4a5a7fc5b
Jul 06 20:48:45.147792 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:45.147561    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a37c14_16e3_47f7_ae66_5a1d840daa48.slice/crio-b0b4e62468adceb8479d6df81a3ef56cd35196bf75e57adfe54f5ac6410617fb WatchSource:0}: Error finding container b0b4e62468adceb8479d6df81a3ef56cd35196bf75e57adfe54f5ac6410617fb: Status 404 returned error can't find the container with id b0b4e62468adceb8479d6df81a3ef56cd35196bf75e57adfe54f5ac6410617fb
Jul 06 20:48:45.149380 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:45.149242    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f813d5c_13a4_485c_b5ab_5f26fd9f84fe.slice/crio-18a7eea78e67d839a279b619181e3d0a1589ed0d784f1e28ed7612df6995f3ab WatchSource:0}: Error finding container 18a7eea78e67d839a279b619181e3d0a1589ed0d784f1e28ed7612df6995f3ab: Status 404 returned error can't find the container with id 18a7eea78e67d839a279b619181e3d0a1589ed0d784f1e28ed7612df6995f3ab
Jul 06 20:48:45.151137 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:45.150101    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5002d46b_a551_424b_bd08_f61d84e3c45b.slice/crio-4042360701da23e7210911f24ec63fe3920db1d41b51b9873345baf59e2c5720 WatchSource:0}: Error finding container 4042360701da23e7210911f24ec63fe3920db1d41b51b9873345baf59e2c5720: Status 404 returned error can't find the container with id 4042360701da23e7210911f24ec63fe3920db1d41b51b9873345baf59e2c5720
Jul 06 20:48:45.152619 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:48:45.152592    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod089e53a7_3267_465e_8abb_52b283f3d756.slice/crio-05edccefc0da449fcde8a043bfa35f36b8c84514ccc6767d8a3d60e3faf48334 WatchSource:0}: Error finding container 05edccefc0da449fcde8a043bfa35f36b8c84514ccc6767d8a3d60e3faf48334: Status 404 returned error can't find the container with id 05edccefc0da449fcde8a043bfa35f36b8c84514ccc6767d8a3d60e3faf48334
Jul 06 20:48:45.523573 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:45.523337    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:45.523743 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:45.523580    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:48:45.523743 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:45.523617    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:45.523743 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:45.523678    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs podName:997ba244-1db8-4c94-bb4e-eda91480d511 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:47.523657927 +0000 UTC m=+5.989899170 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs") pod "network-metrics-daemon-w4sfv" (UID: "997ba244-1db8-4c94-bb4e-eda91480d511") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:48:45.523743 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:45.523733    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:48:45.523897 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:45.523785    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret podName:04009265-aa63-49ba-a715-0a555264ccd7 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:47.523770373 +0000 UTC m=+5.990011607 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret") pod "global-pull-secret-syncer-7lk82" (UID: "04009265-aa63-49ba-a715-0a555264ccd7") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:48:45.625400 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:45.624763    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pws8j\" (UniqueName: \"kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j\") pod \"network-check-target-x2kkv\" (UID: \"748cbc30-cb4a-4a02-bda9-58b147b12ce9\") " pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:45.625400 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:45.624956    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:48:45.625400 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:45.624981    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:48:45.625400 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:45.624993    2566 projected.go:196] Error preparing data for projected volume kube-api-access-pws8j for pod openshift-network-diagnostics/network-check-target-x2kkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:48:45.625400 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:45.625051    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j podName:748cbc30-cb4a-4a02-bda9-58b147b12ce9 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:47.625033131 +0000 UTC m=+6.091274378 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pws8j" (UniqueName: "kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j") pod "network-check-target-x2kkv" (UID: "748cbc30-cb4a-4a02-bda9-58b147b12ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:48:45.998941 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:45.998910    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:45.999114 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:45.999043    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:48:45.999533 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:45.999515    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:45.999630 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:45.999611    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:48:46.022907 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.022871    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ptn9v" event={"ID":"1236b937-e2b7-41ac-a612-142a60a86806","Type":"ContainerStarted","Data":"5db4896d69ba2d84455065d9a64fd6c2ae4b51e76bcc78dac5442a469bd6e64b"}
Jul 06 20:48:46.030084 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.029988    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 20:43:44 +0000 UTC" deadline="2028-01-18 19:57:06.318932005 +0000 UTC"
Jul 06 20:48:46.030084 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.030044    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13463h8m20.288891764s"
Jul 06 20:48:46.048158 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.048095    2566 generic.go:356] "Generic (PLEG): container finished" podID="fce54f2a107c2f172eea3a387f3af721" containerID="59dfedc1b2ff25cd50f7f4562c8ada932873b149139751e2ef418e912b06094a" exitCode=0
Jul 06 20:48:46.048328 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.048193    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal" event={"ID":"fce54f2a107c2f172eea3a387f3af721","Type":"ContainerDied","Data":"59dfedc1b2ff25cd50f7f4562c8ada932873b149139751e2ef418e912b06094a"}
Jul 06 20:48:46.072974 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.072942    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qxhpq" event={"ID":"8f813d5c-13a4-485c-b5ab-5f26fd9f84fe","Type":"ContainerStarted","Data":"18a7eea78e67d839a279b619181e3d0a1589ed0d784f1e28ed7612df6995f3ab"}
Jul 06 20:48:46.092268 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.092230    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mrll5" event={"ID":"5002d46b-a551-424b-bd08-f61d84e3c45b","Type":"ContainerStarted","Data":"4042360701da23e7210911f24ec63fe3920db1d41b51b9873345baf59e2c5720"}
Jul 06 20:48:46.104592 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.104555    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kbwmh" event={"ID":"d9a37c14-16e3-47f7-ae66-5a1d840daa48","Type":"ContainerStarted","Data":"b0b4e62468adceb8479d6df81a3ef56cd35196bf75e57adfe54f5ac6410617fb"}
Jul 06 20:48:46.121467 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.121414    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rhjwc" event={"ID":"071fb91e-f1b8-476f-9b44-0297a6661a85","Type":"ContainerStarted","Data":"bc3c4750b8ce2ee46792feb6a83b1883688f80d3a245838a3cf1adf8179c151f"}
Jul 06 20:48:46.147136 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.147106    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psw72" event={"ID":"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7","Type":"ContainerStarted","Data":"9f4a3f90b6528225a1d7ca0d1929e75cb337c8f65d9adf419d3c7ef4a5a7fc5b"}
Jul 06 20:48:46.157419 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.157361    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xh69c" event={"ID":"e267afa3-e061-4f08-bc2c-4c90e6b26753","Type":"ContainerStarted","Data":"02f2333e6216646837f93a1e34d028cb6cfe4552ceb2f730a0b819a2056efadb"}
Jul 06 20:48:46.189834 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.189789    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w" event={"ID":"02732679-8a59-416d-ac6d-6de4e7e5ab39","Type":"ContainerStarted","Data":"fec588fdd19b70fce790c4abfc58b1c31fbcd35507b0a9103c977228ede13b35"}
Jul 06 20:48:46.200293 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.200256    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal" event={"ID":"e45831538ee9d5022543246cbe3ad035","Type":"ContainerStarted","Data":"d88b79f9d83473df1839551bad5ae18ff09752d03a7ffd6d7793d1950ba7c445"}
Jul 06 20:48:46.209022 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.208943    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hjc94" event={"ID":"089e53a7-3267-465e-8abb-52b283f3d756","Type":"ContainerStarted","Data":"05edccefc0da449fcde8a043bfa35f36b8c84514ccc6767d8a3d60e3faf48334"}
Jul 06 20:48:46.996835 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:46.996530    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:46.996835 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:46.996706    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:48:47.240240 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:47.240033    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal" event={"ID":"fce54f2a107c2f172eea3a387f3af721","Type":"ContainerStarted","Data":"bf28500799fa1537d09392bd2352264e8b80bb0217aaffeb3cf0f3bb4532f4d1"}
Jul 06 20:48:47.256710 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:47.256607    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-165.ec2.internal" podStartSLOduration=4.2565830479999995 podStartE2EDuration="4.256583048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:48:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:48:46.218227181 +0000 UTC m=+4.684468436" watchObservedRunningTime="2026-07-06 20:48:47.256583048 +0000 UTC m=+5.722824304"
Jul 06 20:48:47.543206 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:47.542128    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:47.543206 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:47.542192    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:47.543206 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:47.542381    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:48:47.543206 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:47.542443    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret podName:04009265-aa63-49ba-a715-0a555264ccd7 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:51.542425224 +0000 UTC m=+10.008666467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret") pod "global-pull-secret-syncer-7lk82" (UID: "04009265-aa63-49ba-a715-0a555264ccd7") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:48:47.543206 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:47.542854    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:48:47.543206 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:47.542904    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs podName:997ba244-1db8-4c94-bb4e-eda91480d511 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:51.54288878 +0000 UTC m=+10.009130019 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs") pod "network-metrics-daemon-w4sfv" (UID: "997ba244-1db8-4c94-bb4e-eda91480d511") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:48:47.643975 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:47.643192    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pws8j\" (UniqueName: \"kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j\") pod \"network-check-target-x2kkv\" (UID: \"748cbc30-cb4a-4a02-bda9-58b147b12ce9\") " pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:47.643975 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:47.643381    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:48:47.643975 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:47.643406    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:48:47.643975 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:47.643419    2566 projected.go:196] Error preparing data for projected volume kube-api-access-pws8j for pod openshift-network-diagnostics/network-check-target-x2kkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:48:47.643975 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:47.643492    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j podName:748cbc30-cb4a-4a02-bda9-58b147b12ce9 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:51.643472622 +0000 UTC m=+10.109713864 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pws8j" (UniqueName: "kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j") pod "network-check-target-x2kkv" (UID: "748cbc30-cb4a-4a02-bda9-58b147b12ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:48:47.996767 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:47.996669    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:47.996915 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:47.996802    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:48:47.999271 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:47.999125    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:47.999271 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:47.999232    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:48:48.996065 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:48.996031    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:48.996580 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:48.996181    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:48:49.997184 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:49.996561    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:49.997184 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:49.996704    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:48:49.997184 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:49.997042    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:49.997184 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:49.997142    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:48:50.996515 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:50.996355    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:50.996726 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:50.996519    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:48:51.578959 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:51.578902    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:51.578959 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:51.578957    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:51.579536 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:51.579102    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:48:51.579536 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:51.579178    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret podName:04009265-aa63-49ba-a715-0a555264ccd7 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:59.579159383 +0000 UTC m=+18.045400702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret") pod "global-pull-secret-syncer-7lk82" (UID: "04009265-aa63-49ba-a715-0a555264ccd7") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:48:51.579536 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:51.579104    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:48:51.579536 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:51.579515    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs podName:997ba244-1db8-4c94-bb4e-eda91480d511 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:59.579498334 +0000 UTC m=+18.045739592 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs") pod "network-metrics-daemon-w4sfv" (UID: "997ba244-1db8-4c94-bb4e-eda91480d511") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:48:51.679741 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:51.679676    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pws8j\" (UniqueName: \"kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j\") pod \"network-check-target-x2kkv\" (UID: \"748cbc30-cb4a-4a02-bda9-58b147b12ce9\") " pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:51.679958 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:51.679837    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:48:51.679958 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:51.679857    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:48:51.679958 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:51.679870    2566 projected.go:196] Error preparing data for projected volume kube-api-access-pws8j for pod openshift-network-diagnostics/network-check-target-x2kkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:48:51.679958 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:51.679935    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j podName:748cbc30-cb4a-4a02-bda9-58b147b12ce9 nodeName:}" failed. No retries permitted until 2026-07-06 20:48:59.679915284 +0000 UTC m=+18.146156520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pws8j" (UniqueName: "kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j") pod "network-check-target-x2kkv" (UID: "748cbc30-cb4a-4a02-bda9-58b147b12ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:48:52.001108 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:52.001071    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:52.001426 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:52.001397    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:48:52.001562 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:52.001495    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:52.001718 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:52.001700    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:48:52.997317 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:52.996829    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:52.997317 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:52.996973    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:48:53.999078 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:53.998591    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:53.999078 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:53.998717    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:48:53.999078 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:53.999060    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:53.999639 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:53.999155    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:48:54.996333 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:54.996298    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:54.996519 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:54.996474    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:48:55.996468 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:55.996415    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:55.996929 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:55.996419    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:55.996929 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:55.996634    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:48:55.996929 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:55.996653    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:48:56.996520 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:56.996483    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:56.996941 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:56.996619    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:48:57.996276 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:57.996237    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:57.996276 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:57.996283    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:57.996546 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:57.996380    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:48:57.996874 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:57.996572    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:48:58.996770 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:58.996738    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:58.997178 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:58.996866    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:48:59.638979 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:59.638947    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:59.639177 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:59.639027    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:48:59.639177 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:59.639129    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:48:59.639285 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:59.639192    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs podName:997ba244-1db8-4c94-bb4e-eda91480d511 nodeName:}" failed. No retries permitted until 2026-07-06 20:49:15.639172922 +0000 UTC m=+34.105414170 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs") pod "network-metrics-daemon-w4sfv" (UID: "997ba244-1db8-4c94-bb4e-eda91480d511") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:48:59.639285 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:59.639131    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:48:59.639285 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:59.639270    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret podName:04009265-aa63-49ba-a715-0a555264ccd7 nodeName:}" failed. No retries permitted until 2026-07-06 20:49:15.639253315 +0000 UTC m=+34.105494554 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret") pod "global-pull-secret-syncer-7lk82" (UID: "04009265-aa63-49ba-a715-0a555264ccd7") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:48:59.740067 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:59.740031    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pws8j\" (UniqueName: \"kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j\") pod \"network-check-target-x2kkv\" (UID: \"748cbc30-cb4a-4a02-bda9-58b147b12ce9\") " pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:59.740239 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:59.740209    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:48:59.740313 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:59.740240    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:48:59.740313 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:59.740253    2566 projected.go:196] Error preparing data for projected volume kube-api-access-pws8j for pod openshift-network-diagnostics/network-check-target-x2kkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:48:59.740313 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:59.740306    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j podName:748cbc30-cb4a-4a02-bda9-58b147b12ce9 nodeName:}" failed. No retries permitted until 2026-07-06 20:49:15.740291703 +0000 UTC m=+34.206532936 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pws8j" (UniqueName: "kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j") pod "network-check-target-x2kkv" (UID: "748cbc30-cb4a-4a02-bda9-58b147b12ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:48:59.996468 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:59.996431    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:48:59.996620 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:48:59.996431    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:48:59.996620 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:59.996548    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:48:59.996738 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:48:59.996639    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:49:00.996886 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:00.996842    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:49:00.997334 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:00.996973    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:49:01.996810 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:01.996778    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:01.996981 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:01.996904    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:49:01.996981 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:01.996959    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:49:01.997421 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:01.997039    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:49:02.995919 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:02.995873    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:49:02.996097 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:02.996018    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:49:03.996663 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:03.996624    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:03.997252 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:03.996764    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:49:03.997252 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:03.996823    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:49:03.997252 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:03.996920    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:49:04.283309 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:04.282428    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rhjwc" event={"ID":"071fb91e-f1b8-476f-9b44-0297a6661a85","Type":"ContainerStarted","Data":"05c1281591763528219479527b7e4aefbf5b4cd91f8f4ffeb23c2699006b1533"}
Jul 06 20:49:04.304431 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:04.303420    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rhjwc" podStartSLOduration=3.337375416 podStartE2EDuration="22.303399779s" totalImagesPullingTime="18.966024363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:49:04.30261303 +0000 UTC m=+22.768854287" watchObservedRunningTime="2026-07-06 20:49:04.303399779 +0000 UTC m=+22.769641034"
Jul 06 20:49:04.304431 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:04.303664    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-165.ec2.internal" podStartSLOduration=21.303653167 podStartE2EDuration="21.303653167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:48:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:48:47.25738861 +0000 UTC m=+5.723629867" watchObservedRunningTime="2026-07-06 20:49:04.303653167 +0000 UTC m=+22.769894422"
Jul 06 20:49:04.996614 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:04.996342    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:49:04.996794 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:04.996762    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:49:05.285265 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.285230    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qxhpq" event={"ID":"8f813d5c-13a4-485c-b5ab-5f26fd9f84fe","Type":"ContainerStarted","Data":"80d9dfd88d70c43dc28577d882a824fca238d1421939d112ce299d5916befbac"}
Jul 06 20:49:05.286587 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.286557    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mrll5" event={"ID":"5002d46b-a551-424b-bd08-f61d84e3c45b","Type":"ContainerStarted","Data":"40c621e476b3fe940d2b762d440070ab164508f9d5946dd4256e3a654ad8a9f4"}
Jul 06 20:49:05.292142 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.292113    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psw72" event={"ID":"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7","Type":"ContainerStarted","Data":"12633ff1385fdbb2da4205257dba5a7ec1c3a1f9fb3da3bc2a559bbc906bcbba"}
Jul 06 20:49:05.292268 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.292208    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psw72" event={"ID":"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7","Type":"ContainerStarted","Data":"cbaf7013eb03f2436c89aa185b31090736c4cb7c0c8f7a9368b7db10893a9dd3"}
Jul 06 20:49:05.292268 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.292241    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psw72" event={"ID":"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7","Type":"ContainerStarted","Data":"a53d1468893196584e2da98d37744598ff2b78ce6f76784c3339b544354a4ab2"}
Jul 06 20:49:05.292361 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.292276    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psw72" event={"ID":"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7","Type":"ContainerStarted","Data":"b0c833d8ed98b76b13c944ddafcde83b0d666d9e92fcf9c07f909f15729c0b4f"}
Jul 06 20:49:05.292361 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.292303    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psw72" event={"ID":"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7","Type":"ContainerStarted","Data":"fb41fe5838909af3754f057165d3e76f3f9b290e0fc5ebebf1f670093f0aaedf"}
Jul 06 20:49:05.292361 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.292345    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psw72" event={"ID":"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7","Type":"ContainerStarted","Data":"57ef85864e9a19ebe628d0e2b212b53d8205ef190e7aadcd3d214c207285f140"}
Jul 06 20:49:05.293566 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.293544    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xh69c" event={"ID":"e267afa3-e061-4f08-bc2c-4c90e6b26753","Type":"ContainerStarted","Data":"29b9d65a0ecdadcbd9d643492bf63b8ca40aa5d744599c2b2eb92837822d918e"}
Jul 06 20:49:05.294972 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.294950    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w" event={"ID":"02732679-8a59-416d-ac6d-6de4e7e5ab39","Type":"ContainerStarted","Data":"40ef07e863023a01e52b604530083333c6803b3ac3256f7d784d151048b37607"}
Jul 06 20:49:05.296287 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.296266    2566 generic.go:356] "Generic (PLEG): container finished" podID="089e53a7-3267-465e-8abb-52b283f3d756" containerID="924189d360e32b430d6cab1f7adb2fed3e325b96b34ba08cd74c2c37d36ae18b" exitCode=0
Jul 06 20:49:05.296357 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.296332    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hjc94" event={"ID":"089e53a7-3267-465e-8abb-52b283f3d756","Type":"ContainerDied","Data":"924189d360e32b430d6cab1f7adb2fed3e325b96b34ba08cd74c2c37d36ae18b"}
Jul 06 20:49:05.297735 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.297715    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ptn9v" event={"ID":"1236b937-e2b7-41ac-a612-142a60a86806","Type":"ContainerStarted","Data":"7eff9bb84291162f9da2564b906280d7317cfe6c04c5fd39bdd1724b88661ad3"}
Jul 06 20:49:05.318696 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.318648    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qxhpq" podStartSLOduration=4.388669469 podStartE2EDuration="23.318632522s" totalImagesPullingTime="18.929963053s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:49:05.304322004 +0000 UTC m=+23.770563259" watchObservedRunningTime="2026-07-06 20:49:05.318632522 +0000 UTC m=+23.784873775"
Jul 06 20:49:05.341840 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.341795    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mrll5" podStartSLOduration=4.419240093 podStartE2EDuration="23.341779131s" totalImagesPullingTime="18.922539038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:49:05.31871662 +0000 UTC m=+23.784957874" watchObservedRunningTime="2026-07-06 20:49:05.341779131 +0000 UTC m=+23.808020364"
Jul 06 20:49:05.358384 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.358334    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xh69c" podStartSLOduration=4.328423069 podStartE2EDuration="23.358318897s" totalImagesPullingTime="19.029895828s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:49:05.357951734 +0000 UTC m=+23.824192987" watchObservedRunningTime="2026-07-06 20:49:05.358318897 +0000 UTC m=+23.824560152"
Jul 06 20:49:05.374198 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.374119    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ptn9v" podStartSLOduration=4.4331501509999995 podStartE2EDuration="23.374103909s" totalImagesPullingTime="18.940953758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:49:05.373680195 +0000 UTC m=+23.839921446" watchObservedRunningTime="2026-07-06 20:49:05.374103909 +0000 UTC m=+23.840345162"
Jul 06 20:49:05.861211 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.861176    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 20:49:05.996622 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.996591    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:05.996802 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.996594    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:49:05.996802 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:05.996725    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:49:05.997197 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:05.996797    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:49:05.997605 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.997511    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T20:49:05.861199731Z","UUID":"27591d25-9c78-40bb-ab5f-5c7d17e550f1","Handler":null,"Name":"","Endpoint":""}
Jul 06 20:49:05.999422 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.999402    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 20:49:05.999525 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:05.999443    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 20:49:06.301424 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:06.301340    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kbwmh" event={"ID":"d9a37c14-16e3-47f7-ae66-5a1d840daa48","Type":"ContainerStarted","Data":"587c6955cd7a5d12e009932c255a7770cedab06faade85bf0c02621a8debe6e3"}
Jul 06 20:49:06.303388 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:06.303324    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w" event={"ID":"02732679-8a59-416d-ac6d-6de4e7e5ab39","Type":"ContainerStarted","Data":"49623fd0eb31ab44174611a4b8571b9952f8aed61b8e28187d19684191abe43c"}
Jul 06 20:49:06.316961 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:06.316919    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kbwmh" podStartSLOduration=5.377749938 podStartE2EDuration="24.316904733s" totalImagesPullingTime="18.939154795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:49:06.316646575 +0000 UTC m=+24.782887826" watchObservedRunningTime="2026-07-06 20:49:06.316904733 +0000 UTC m=+24.783145994"
Jul 06 20:49:06.996174 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:06.996151    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:49:06.996305 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:06.996265    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:49:07.309334 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:07.309146    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psw72" event={"ID":"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7","Type":"ContainerStarted","Data":"f5502c88a837f92302a74559dfaeeafefa5b2ac23838b3ba2e562176c38be108"}
Jul 06 20:49:07.996814 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:07.996778    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:07.997046 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:07.996788    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:49:07.997046 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:07.996969    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:49:07.997046 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:07.996977    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:49:08.314502 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:08.314396    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w" event={"ID":"02732679-8a59-416d-ac6d-6de4e7e5ab39","Type":"ContainerStarted","Data":"23aef858b0281c46907f9182f85c8da0fd7628097a8fbe9c875f8689fcd579d6"}
Jul 06 20:49:08.332812 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:08.332762    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4g9w" podStartSLOduration=4.5470976929999996 podStartE2EDuration="26.332741576s" totalImagesPullingTime="21.785643883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:48:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:49:08.332336745 +0000 UTC m=+26.798577999" watchObservedRunningTime="2026-07-06 20:49:08.332741576 +0000 UTC m=+26.798982831"
Jul 06 20:49:08.995882 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:08.995846    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:49:08.996155 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:08.995978    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:49:09.054468 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:09.054416    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mrll5"
Jul 06 20:49:09.054994 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:09.054978    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mrll5"
Jul 06 20:49:09.320835 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:09.320654    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psw72" event={"ID":"e15f043c-3e63-4c20-a0c7-a2fabbc1b9d7","Type":"ContainerStarted","Data":"75fe6fc31245489c92ddbd8941e0a4a62029f338405c5a071e37965539e212de"}
Jul 06 20:49:09.349631 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:09.349585    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-psw72" podStartSLOduration=8.343442912 podStartE2EDuration="27.34956854s" totalImagesPullingTime="19.006125628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:49:09.348384235 +0000 UTC m=+27.814625489" watchObservedRunningTime="2026-07-06 20:49:09.34956854 +0000 UTC m=+27.815809794"
Jul 06 20:49:09.995987 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:09.995946    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:49:09.996207 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:09.996001    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:09.996207 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:09.996093    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:49:09.996207 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:09.996126    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:49:10.323406 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:10.323321    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:49:10.323406 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:10.323397    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:49:10.323883 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:10.323426    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:49:10.340647 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:10.340618    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:49:10.340805 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:10.340772    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:49:10.996159 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:10.996130    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:49:10.996355 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:10.996264    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:49:11.171237 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:11.171145    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7lk82"]
Jul 06 20:49:11.171424 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:11.171273    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:49:11.171424 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:11.171385    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:49:11.173826 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:11.173798    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x2kkv"]
Jul 06 20:49:11.173947 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:11.173907    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:11.174022 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:11.174001    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:49:11.176928 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:11.176903    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w4sfv"]
Jul 06 20:49:11.324977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:11.324892    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:49:11.325472 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:11.325030    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:49:12.036615 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:12.033501    2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod089e53a7_3267_465e_8abb_52b283f3d756.slice/crio-conmon-04af248fdb3f4b77c9afb94fb82b56ee8333359b130cc8ff33d297c08ee95759.scope\": RecentStats: unable to find data in memory cache]"
Jul 06 20:49:12.328444 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:12.328406    2566 generic.go:356] "Generic (PLEG): container finished" podID="089e53a7-3267-465e-8abb-52b283f3d756" containerID="04af248fdb3f4b77c9afb94fb82b56ee8333359b130cc8ff33d297c08ee95759" exitCode=0
Jul 06 20:49:12.328851 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:12.328505    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hjc94" event={"ID":"089e53a7-3267-465e-8abb-52b283f3d756","Type":"ContainerDied","Data":"04af248fdb3f4b77c9afb94fb82b56ee8333359b130cc8ff33d297c08ee95759"}
Jul 06 20:49:12.996160 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:12.996125    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:49:12.996160 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:12.996146    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:12.996520 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:12.996147    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:49:12.996520 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:12.996262    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:49:12.996520 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:12.996344    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:49:12.996520 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:12.996418    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:49:13.333210 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:13.333027    2566 generic.go:356] "Generic (PLEG): container finished" podID="089e53a7-3267-465e-8abb-52b283f3d756" containerID="4078b9961b0c5b641cab121e117039fface561a525b149797842cded1ea28e09" exitCode=0
Jul 06 20:49:13.333210 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:13.333107    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hjc94" event={"ID":"089e53a7-3267-465e-8abb-52b283f3d756","Type":"ContainerDied","Data":"4078b9961b0c5b641cab121e117039fface561a525b149797842cded1ea28e09"}
Jul 06 20:49:13.990030 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:13.990001    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mrll5"
Jul 06 20:49:13.990175 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:13.990163    2566 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 06 20:49:13.990631 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:13.990608    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mrll5"
Jul 06 20:49:14.341008 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:14.340978    2566 generic.go:356] "Generic (PLEG): container finished" podID="089e53a7-3267-465e-8abb-52b283f3d756" containerID="6250923e239d00a766e2584eed782595ddc4a09bcdf9eac9c15e432d9381b033" exitCode=0
Jul 06 20:49:14.341530 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:14.341054    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hjc94" event={"ID":"089e53a7-3267-465e-8abb-52b283f3d756","Type":"ContainerDied","Data":"6250923e239d00a766e2584eed782595ddc4a09bcdf9eac9c15e432d9381b033"}
Jul 06 20:49:14.996554 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:14.996517    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:49:14.996775 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:14.996530    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:49:14.996775 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:14.996644    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4sfv" podUID="997ba244-1db8-4c94-bb4e-eda91480d511"
Jul 06 20:49:14.996775 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:14.996530    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:14.996964 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:14.996737    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7lk82" podUID="04009265-aa63-49ba-a715-0a555264ccd7"
Jul 06 20:49:14.996964 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:14.996834    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2kkv" podUID="748cbc30-cb4a-4a02-bda9-58b147b12ce9"
Jul 06 20:49:15.658284 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:15.658248    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:49:15.658791 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:15.658313    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:49:15.658791 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:15.658420    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:49:15.658791 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:15.658450    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:49:15.658791 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:15.658521    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs podName:997ba244-1db8-4c94-bb4e-eda91480d511 nodeName:}" failed. No retries permitted until 2026-07-06 20:49:47.658501234 +0000 UTC m=+66.124742471 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs") pod "network-metrics-daemon-w4sfv" (UID: "997ba244-1db8-4c94-bb4e-eda91480d511") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:49:15.658791 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:15.658542    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret podName:04009265-aa63-49ba-a715-0a555264ccd7 nodeName:}" failed. No retries permitted until 2026-07-06 20:49:47.658533271 +0000 UTC m=+66.124774509 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret") pod "global-pull-secret-syncer-7lk82" (UID: "04009265-aa63-49ba-a715-0a555264ccd7") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:49:15.758969 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:15.758932    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pws8j\" (UniqueName: \"kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j\") pod \"network-check-target-x2kkv\" (UID: \"748cbc30-cb4a-4a02-bda9-58b147b12ce9\") " pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:15.759145 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:15.759079    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:49:15.759145 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:15.759100    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:49:15.759145 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:15.759110    2566 projected.go:196] Error preparing data for projected volume kube-api-access-pws8j for pod openshift-network-diagnostics/network-check-target-x2kkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:49:15.759314 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:15.759168    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j podName:748cbc30-cb4a-4a02-bda9-58b147b12ce9 nodeName:}" failed. No retries permitted until 2026-07-06 20:49:47.759151339 +0000 UTC m=+66.225392573 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pws8j" (UniqueName: "kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j") pod "network-check-target-x2kkv" (UID: "748cbc30-cb4a-4a02-bda9-58b147b12ce9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:49:16.861181 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.861151    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-165.ec2.internal" event="NodeReady"
Jul 06 20:49:16.861603 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.861279    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 20:49:16.896627 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.896542    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68d4f94c5d-xzkmd"]
Jul 06 20:49:16.899198 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.899174    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:16.901499 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.901440    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 20:49:16.901618 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.901504    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 20:49:16.901711 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.901686    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ldn5x\""
Jul 06 20:49:16.901808 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.901750    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 20:49:16.911178 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.908979    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 20:49:16.911178 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.910041    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6mllp"]
Jul 06 20:49:16.915118 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.915097    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68d4f94c5d-xzkmd"]
Jul 06 20:49:16.915242 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.915136    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fswwk"]
Jul 06 20:49:16.915242 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.915210    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:16.917051 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.917029    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fswwk"
Jul 06 20:49:16.917440 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.917419    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 20:49:16.917565 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.917473    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cpffn\""
Jul 06 20:49:16.917873 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.917776    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 20:49:16.919530 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.919308    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 20:49:16.919530 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.919371    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zzls9\""
Jul 06 20:49:16.919642 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.919532    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 20:49:16.919688 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.919675    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 20:49:16.922143 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.922120    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6mllp"]
Jul 06 20:49:16.928907 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.928887    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fswwk"]
Jul 06 20:49:16.996303 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.996268    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:49:16.996303 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.996299    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:49:16.996552 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.996299    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:16.999043 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.998994    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 20:49:16.999043 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.999024    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 20:49:16.999043 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.999040    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 20:49:16.999252 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.999043    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hgfwv\""
Jul 06 20:49:16.999252 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.999107    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mf9wp\""
Jul 06 20:49:16.999252 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:16.998997    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 20:49:17.070333 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070296    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1cd6b87c-70df-459e-9f27-d458d34d1bed-image-registry-private-configuration\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.070333 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070341    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.070609 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070367    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1cd6b87c-70df-459e-9f27-d458d34d1bed-installation-pull-secrets\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.070609 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070481    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8plq\" (UniqueName: \"kubernetes.io/projected/a36da1e0-2692-4633-b274-158c32c67c8a-kube-api-access-j8plq\") pod \"ingress-canary-fswwk\" (UID: \"a36da1e0-2692-4633-b274-158c32c67c8a\") " pod="openshift-ingress-canary/ingress-canary-fswwk"
Jul 06 20:49:17.070609 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070524    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1cd6b87c-70df-459e-9f27-d458d34d1bed-trusted-ca\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.070609 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070556    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-bound-sa-token\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.070609 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070594    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4133d58d-63a9-46cd-b3d5-eddf0406fe31-tmp-dir\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:17.070809 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070664    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dxh4\" (UniqueName: \"kubernetes.io/projected/4133d58d-63a9-46cd-b3d5-eddf0406fe31-kube-api-access-6dxh4\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:17.070809 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070729    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1cd6b87c-70df-459e-9f27-d458d34d1bed-ca-trust-extracted\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.070809 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070767    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwczg\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-kube-api-access-fwczg\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.070933 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070813    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a36da1e0-2692-4633-b274-158c32c67c8a-cert\") pod \"ingress-canary-fswwk\" (UID: \"a36da1e0-2692-4633-b274-158c32c67c8a\") " pod="openshift-ingress-canary/ingress-canary-fswwk"
Jul 06 20:49:17.070933 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070839    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-certificates\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.070933 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070878    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4133d58d-63a9-46cd-b3d5-eddf0406fe31-config-volume\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:17.070933 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.070903    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:17.171873 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.171784    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4133d58d-63a9-46cd-b3d5-eddf0406fe31-tmp-dir\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:17.171873 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.171838    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dxh4\" (UniqueName: \"kubernetes.io/projected/4133d58d-63a9-46cd-b3d5-eddf0406fe31-kube-api-access-6dxh4\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:17.171873 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.171866    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1cd6b87c-70df-459e-9f27-d458d34d1bed-ca-trust-extracted\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.172094 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.171893    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwczg\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-kube-api-access-fwczg\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.172094 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.171932    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a36da1e0-2692-4633-b274-158c32c67c8a-cert\") pod \"ingress-canary-fswwk\" (UID: \"a36da1e0-2692-4633-b274-158c32c67c8a\") " pod="openshift-ingress-canary/ingress-canary-fswwk"
Jul 06 20:49:17.172094 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.171956    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-certificates\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.172094 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.172007    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4133d58d-63a9-46cd-b3d5-eddf0406fe31-config-volume\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:17.172094 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.172029    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:17.172094 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.172059    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1cd6b87c-70df-459e-9f27-d458d34d1bed-image-registry-private-configuration\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.172094 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.172085    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.172368 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.172109    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1cd6b87c-70df-459e-9f27-d458d34d1bed-installation-pull-secrets\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.172368 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.172142    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8plq\" (UniqueName: \"kubernetes.io/projected/a36da1e0-2692-4633-b274-158c32c67c8a-kube-api-access-j8plq\") pod \"ingress-canary-fswwk\" (UID: \"a36da1e0-2692-4633-b274-158c32c67c8a\") " pod="openshift-ingress-canary/ingress-canary-fswwk"
Jul 06 20:49:17.172368 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.172168    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1cd6b87c-70df-459e-9f27-d458d34d1bed-trusted-ca\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.172368 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.172188    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4133d58d-63a9-46cd-b3d5-eddf0406fe31-tmp-dir\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:17.172368 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.172193    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-bound-sa-token\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.172368 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.172299    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1cd6b87c-70df-459e-9f27-d458d34d1bed-ca-trust-extracted\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.173100 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:17.172787    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:49:17.173100 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.172795    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-certificates\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.173100 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:17.172851    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls podName:4133d58d-63a9-46cd-b3d5-eddf0406fe31 nodeName:}" failed. No retries permitted until 2026-07-06 20:49:17.672832626 +0000 UTC m=+36.139073874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls") pod "dns-default-6mllp" (UID: "4133d58d-63a9-46cd-b3d5-eddf0406fe31") : secret "dns-default-metrics-tls" not found
Jul 06 20:49:17.173100 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.172858    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4133d58d-63a9-46cd-b3d5-eddf0406fe31-config-volume\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:17.173100 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:17.172871    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:49:17.173100 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:17.172886    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d4f94c5d-xzkmd: secret "image-registry-tls" not found
Jul 06 20:49:17.173100 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:17.172956    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls podName:1cd6b87c-70df-459e-9f27-d458d34d1bed nodeName:}" failed. No retries permitted until 2026-07-06 20:49:17.6729379 +0000 UTC m=+36.139179133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls") pod "image-registry-68d4f94c5d-xzkmd" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed") : secret "image-registry-tls" not found
Jul 06 20:49:17.173505 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.173442    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1cd6b87c-70df-459e-9f27-d458d34d1bed-trusted-ca\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.176786 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.176765    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1cd6b87c-70df-459e-9f27-d458d34d1bed-installation-pull-secrets\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.176909 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.176879    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1cd6b87c-70df-459e-9f27-d458d34d1bed-image-registry-private-configuration\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.176968 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.176950    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a36da1e0-2692-4633-b274-158c32c67c8a-cert\") pod \"ingress-canary-fswwk\" (UID: \"a36da1e0-2692-4633-b274-158c32c67c8a\") " pod="openshift-ingress-canary/ingress-canary-fswwk"
Jul 06 20:49:17.184490 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.184445    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dxh4\" (UniqueName: \"kubernetes.io/projected/4133d58d-63a9-46cd-b3d5-eddf0406fe31-kube-api-access-6dxh4\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:17.184709 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.184613    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-bound-sa-token\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.184709 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.184671    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwczg\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-kube-api-access-fwczg\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.186424 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.186404    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8plq\" (UniqueName: \"kubernetes.io/projected/a36da1e0-2692-4633-b274-158c32c67c8a-kube-api-access-j8plq\") pod \"ingress-canary-fswwk\" (UID: \"a36da1e0-2692-4633-b274-158c32c67c8a\") " pod="openshift-ingress-canary/ingress-canary-fswwk"
Jul 06 20:49:17.234499 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.234447    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fswwk"
Jul 06 20:49:17.676348 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.676306    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:17.676639 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:17.676354    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:17.676639 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:17.676490    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:49:17.676639 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:17.676570    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls podName:4133d58d-63a9-46cd-b3d5-eddf0406fe31 nodeName:}" failed. No retries permitted until 2026-07-06 20:49:18.676549411 +0000 UTC m=+37.142790645 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls") pod "dns-default-6mllp" (UID: "4133d58d-63a9-46cd-b3d5-eddf0406fe31") : secret "dns-default-metrics-tls" not found
Jul 06 20:49:17.676639 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:17.676491    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:49:17.676639 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:17.676604    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d4f94c5d-xzkmd: secret "image-registry-tls" not found
Jul 06 20:49:17.676841 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:17.676696    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls podName:1cd6b87c-70df-459e-9f27-d458d34d1bed nodeName:}" failed. No retries permitted until 2026-07-06 20:49:18.676677443 +0000 UTC m=+37.142918675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls") pod "image-registry-68d4f94c5d-xzkmd" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed") : secret "image-registry-tls" not found
Jul 06 20:49:18.466278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:18.466084    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fswwk"]
Jul 06 20:49:18.519938 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:49:18.519907    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda36da1e0_2692_4633_b274_158c32c67c8a.slice/crio-0eb07a04d92a699326f57bef62097b2aadfd828e779625e7fce131d8536c3fe8 WatchSource:0}: Error finding container 0eb07a04d92a699326f57bef62097b2aadfd828e779625e7fce131d8536c3fe8: Status 404 returned error can't find the container with id 0eb07a04d92a699326f57bef62097b2aadfd828e779625e7fce131d8536c3fe8
Jul 06 20:49:18.684772 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:18.684724    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:18.684772 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:18.684774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:18.685029 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:18.684871    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:49:18.685029 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:18.684914    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:49:18.685029 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:18.684926    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d4f94c5d-xzkmd: secret "image-registry-tls" not found
Jul 06 20:49:18.685029 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:18.684946    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls podName:4133d58d-63a9-46cd-b3d5-eddf0406fe31 nodeName:}" failed. No retries permitted until 2026-07-06 20:49:20.684930426 +0000 UTC m=+39.151171661 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls") pod "dns-default-6mllp" (UID: "4133d58d-63a9-46cd-b3d5-eddf0406fe31") : secret "dns-default-metrics-tls" not found
Jul 06 20:49:18.685029 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:18.684966    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls podName:1cd6b87c-70df-459e-9f27-d458d34d1bed nodeName:}" failed. No retries permitted until 2026-07-06 20:49:20.684955454 +0000 UTC m=+39.151196687 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls") pod "image-registry-68d4f94c5d-xzkmd" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed") : secret "image-registry-tls" not found
Jul 06 20:49:18.896172 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:18.896078    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrll5_5002d46b-a551-424b-bd08-f61d84e3c45b/konnectivity-agent/0.log" limit=1024
Jul 06 20:49:19.007449 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:19.007393    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrll5_5002d46b-a551-424b-bd08-f61d84e3c45b/konnectivity-agent/0.log" limit=1024
Jul 06 20:49:19.109090 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:19.109056    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrll5_5002d46b-a551-424b-bd08-f61d84e3c45b/konnectivity-agent/0.log" limit=1024
Jul 06 20:49:19.210913 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:19.210816    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrll5_5002d46b-a551-424b-bd08-f61d84e3c45b/konnectivity-agent/0.log" limit=1024
Jul 06 20:49:19.309839 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:19.309805    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrll5_5002d46b-a551-424b-bd08-f61d84e3c45b/konnectivity-agent/0.log" limit=1024
Jul 06 20:49:19.357278 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:19.357246    2566 generic.go:356] "Generic (PLEG): container finished" podID="089e53a7-3267-465e-8abb-52b283f3d756" containerID="a7ed204c37803e3d211173e763b605320f0e7f19e77a90205cd3efaf7771f0d9" exitCode=0
Jul 06 20:49:19.357448 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:19.357331    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hjc94" event={"ID":"089e53a7-3267-465e-8abb-52b283f3d756","Type":"ContainerDied","Data":"a7ed204c37803e3d211173e763b605320f0e7f19e77a90205cd3efaf7771f0d9"}
Jul 06 20:49:19.358757 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:19.358723    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fswwk" event={"ID":"a36da1e0-2692-4633-b274-158c32c67c8a","Type":"ContainerStarted","Data":"0eb07a04d92a699326f57bef62097b2aadfd828e779625e7fce131d8536c3fe8"}
Jul 06 20:49:20.364917 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:20.364888    2566 generic.go:356] "Generic (PLEG): container finished" podID="089e53a7-3267-465e-8abb-52b283f3d756" containerID="239d7e4e1ca2fa56dbf7f85151229ecb0c200fec75489c62724672905214adac" exitCode=0
Jul 06 20:49:20.365446 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:20.364949    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hjc94" event={"ID":"089e53a7-3267-465e-8abb-52b283f3d756","Type":"ContainerDied","Data":"239d7e4e1ca2fa56dbf7f85151229ecb0c200fec75489c62724672905214adac"}
Jul 06 20:49:20.710164 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:20.710079    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:20.710326 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:20.710213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:20.710326 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:20.710243    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:49:20.710326 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:20.710260    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d4f94c5d-xzkmd: secret "image-registry-tls" not found
Jul 06 20:49:20.710326 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:20.710308    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:49:20.710326 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:20.710324    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls podName:1cd6b87c-70df-459e-9f27-d458d34d1bed nodeName:}" failed. No retries permitted until 2026-07-06 20:49:24.7103052 +0000 UTC m=+43.176546452 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls") pod "image-registry-68d4f94c5d-xzkmd" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed") : secret "image-registry-tls" not found
Jul 06 20:49:20.710577 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:20.710351    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls podName:4133d58d-63a9-46cd-b3d5-eddf0406fe31 nodeName:}" failed. No retries permitted until 2026-07-06 20:49:24.710341488 +0000 UTC m=+43.176582734 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls") pod "dns-default-6mllp" (UID: "4133d58d-63a9-46cd-b3d5-eddf0406fe31") : secret "dns-default-metrics-tls" not found
Jul 06 20:49:21.371215 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:21.370953    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hjc94" event={"ID":"089e53a7-3267-465e-8abb-52b283f3d756","Type":"ContainerStarted","Data":"71df09465dec8af8c383cc004cacb48e0b3766ee36c2e63b970cad94072b4651"}
Jul 06 20:49:21.372524 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:21.372490    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fswwk" event={"ID":"a36da1e0-2692-4633-b274-158c32c67c8a","Type":"ContainerStarted","Data":"d94a2c4ff10656d4a45b1293fd160de108c73ecf0e2e3cb416b28183b9ebb871"}
Jul 06 20:49:21.399292 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:21.399243    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hjc94" podStartSLOduration=8.510177492 podStartE2EDuration="39.399225681s" totalImagesPullingTime="30.889048189s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:48:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:49:21.398072747 +0000 UTC m=+39.864314001" watchObservedRunningTime="2026-07-06 20:49:21.399225681 +0000 UTC m=+39.865466935"
Jul 06 20:49:21.414603 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:21.414550    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fswwk" podStartSLOduration=2.822578162 podStartE2EDuration="5.414534554s" totalImagesPullingTime="2.591956392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:49:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:49:21.413970278 +0000 UTC m=+39.880211533" watchObservedRunningTime="2026-07-06 20:49:21.414534554 +0000 UTC m=+39.880775808"
Jul 06 20:49:21.631592 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:21.631488    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrll5_5002d46b-a551-424b-bd08-f61d84e3c45b/konnectivity-agent/0.log" limit=1024
Jul 06 20:49:21.732974 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:21.732939    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrll5_5002d46b-a551-424b-bd08-f61d84e3c45b/konnectivity-agent/0.log" limit=1024
Jul 06 20:49:22.415781 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:22.415748    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrll5_5002d46b-a551-424b-bd08-f61d84e3c45b/konnectivity-agent/0.log" limit=1024
Jul 06 20:49:22.701229 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:22.701119    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrll5_5002d46b-a551-424b-bd08-f61d84e3c45b/konnectivity-agent/0.log" limit=1024
Jul 06 20:49:22.805013 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:22.804977    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrll5_5002d46b-a551-424b-bd08-f61d84e3c45b/konnectivity-agent/0.log" limit=1024
Jul 06 20:49:24.742148 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:24.742111    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:24.742148 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:24.742153    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:24.742599 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:24.742252    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:49:24.742599 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:24.742257    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:49:24.742599 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:24.742322    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls podName:4133d58d-63a9-46cd-b3d5-eddf0406fe31 nodeName:}" failed. No retries permitted until 2026-07-06 20:49:32.742305456 +0000 UTC m=+51.208546688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls") pod "dns-default-6mllp" (UID: "4133d58d-63a9-46cd-b3d5-eddf0406fe31") : secret "dns-default-metrics-tls" not found
Jul 06 20:49:24.742599 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:24.742264    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d4f94c5d-xzkmd: secret "image-registry-tls" not found
Jul 06 20:49:24.742599 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:24.742403    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls podName:1cd6b87c-70df-459e-9f27-d458d34d1bed nodeName:}" failed. No retries permitted until 2026-07-06 20:49:32.742391268 +0000 UTC m=+51.208632501 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls") pod "image-registry-68d4f94c5d-xzkmd" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed") : secret "image-registry-tls" not found
Jul 06 20:49:32.799444 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:32.799403    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:32.799888 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:32.799473    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:32.799888 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:32.799555    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:49:32.799888 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:32.799566    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:49:32.799888 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:32.799585    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d4f94c5d-xzkmd: secret "image-registry-tls" not found
Jul 06 20:49:32.799888 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:32.799625    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls podName:4133d58d-63a9-46cd-b3d5-eddf0406fe31 nodeName:}" failed. No retries permitted until 2026-07-06 20:49:48.799609577 +0000 UTC m=+67.265850808 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls") pod "dns-default-6mllp" (UID: "4133d58d-63a9-46cd-b3d5-eddf0406fe31") : secret "dns-default-metrics-tls" not found
Jul 06 20:49:32.799888 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:32.799639    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls podName:1cd6b87c-70df-459e-9f27-d458d34d1bed nodeName:}" failed. No retries permitted until 2026-07-06 20:49:48.799633358 +0000 UTC m=+67.265874590 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls") pod "image-registry-68d4f94c5d-xzkmd" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed") : secret "image-registry-tls" not found
Jul 06 20:49:42.339232 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:42.339203    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psw72"
Jul 06 20:49:47.712918 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:47.712828    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:49:47.712918 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:47.712879    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:49:47.715181 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:47.715157    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 20:49:47.715286 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:47.715270    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 20:49:47.723507 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:47.723486    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 20:49:47.723589 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:47.723559    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs podName:997ba244-1db8-4c94-bb4e-eda91480d511 nodeName:}" failed. No retries permitted until 2026-07-06 20:50:51.723542192 +0000 UTC m=+130.189783424 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs") pod "network-metrics-daemon-w4sfv" (UID: "997ba244-1db8-4c94-bb4e-eda91480d511") : secret "metrics-daemon-secret" not found
Jul 06 20:49:47.727137 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:47.727109    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04009265-aa63-49ba-a715-0a555264ccd7-original-pull-secret\") pod \"global-pull-secret-syncer-7lk82\" (UID: \"04009265-aa63-49ba-a715-0a555264ccd7\") " pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:49:47.813156 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:47.813123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pws8j\" (UniqueName: \"kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j\") pod \"network-check-target-x2kkv\" (UID: \"748cbc30-cb4a-4a02-bda9-58b147b12ce9\") " pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:47.816144 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:47.816125    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 20:49:47.826611 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:47.826593    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 20:49:47.836657 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:47.836626    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pws8j\" (UniqueName: \"kubernetes.io/projected/748cbc30-cb4a-4a02-bda9-58b147b12ce9-kube-api-access-pws8j\") pod \"network-check-target-x2kkv\" (UID: \"748cbc30-cb4a-4a02-bda9-58b147b12ce9\") " pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:47.907583 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:47.907545    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7lk82"
Jul 06 20:49:47.922425 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:47.922402    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hgfwv\""
Jul 06 20:49:47.930760 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:47.930729    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:48.058350 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:48.058320    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7lk82"]
Jul 06 20:49:48.061565 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:49:48.061533    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04009265_aa63_49ba_a715_0a555264ccd7.slice/crio-f10605061d3f43332538f09aa514da1aa8ffc7476c770b99a5cbcc2fe3d259cd WatchSource:0}: Error finding container f10605061d3f43332538f09aa514da1aa8ffc7476c770b99a5cbcc2fe3d259cd: Status 404 returned error can't find the container with id f10605061d3f43332538f09aa514da1aa8ffc7476c770b99a5cbcc2fe3d259cd
Jul 06 20:49:48.074703 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:48.074678    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x2kkv"]
Jul 06 20:49:48.079194 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:49:48.079163    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod748cbc30_cb4a_4a02_bda9_58b147b12ce9.slice/crio-d9aba6a541181284efd3ab2950f5e467cc977fd83231c9b52ba72cf161b08f6c WatchSource:0}: Error finding container d9aba6a541181284efd3ab2950f5e467cc977fd83231c9b52ba72cf161b08f6c: Status 404 returned error can't find the container with id d9aba6a541181284efd3ab2950f5e467cc977fd83231c9b52ba72cf161b08f6c
Jul 06 20:49:48.435490 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:48.435432    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x2kkv" event={"ID":"748cbc30-cb4a-4a02-bda9-58b147b12ce9","Type":"ContainerStarted","Data":"d9aba6a541181284efd3ab2950f5e467cc977fd83231c9b52ba72cf161b08f6c"}
Jul 06 20:49:48.436774 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:48.436743    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7lk82" event={"ID":"04009265-aa63-49ba-a715-0a555264ccd7","Type":"ContainerStarted","Data":"f10605061d3f43332538f09aa514da1aa8ffc7476c770b99a5cbcc2fe3d259cd"}
Jul 06 20:49:48.822542 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:48.822502    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:49:48.822983 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:48.822557    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:49:48.822983 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:48.822666    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:49:48.822983 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:48.822685    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:49:48.822983 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:48.822698    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d4f94c5d-xzkmd: secret "image-registry-tls" not found
Jul 06 20:49:48.822983 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:48.822735    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls podName:4133d58d-63a9-46cd-b3d5-eddf0406fe31 nodeName:}" failed. No retries permitted until 2026-07-06 20:50:20.822715999 +0000 UTC m=+99.288957230 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls") pod "dns-default-6mllp" (UID: "4133d58d-63a9-46cd-b3d5-eddf0406fe31") : secret "dns-default-metrics-tls" not found
Jul 06 20:49:48.822983 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:49:48.822750    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls podName:1cd6b87c-70df-459e-9f27-d458d34d1bed nodeName:}" failed. No retries permitted until 2026-07-06 20:50:20.822743828 +0000 UTC m=+99.288985060 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls") pod "image-registry-68d4f94c5d-xzkmd" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed") : secret "image-registry-tls" not found
Jul 06 20:49:53.450120 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:53.450078    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x2kkv" event={"ID":"748cbc30-cb4a-4a02-bda9-58b147b12ce9","Type":"ContainerStarted","Data":"b6c79e5a229f2942abdcc6f8757d93b71a3402ef51e1f3262072546fe5bd22e3"}
Jul 06 20:49:53.450627 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:53.450141    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:49:53.451532 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:53.451507    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7lk82" event={"ID":"04009265-aa63-49ba-a715-0a555264ccd7","Type":"ContainerStarted","Data":"31c73855bf4e2a3fa3e625574af6dc0ad6553e1b1b2125ec77cc3b8c1fdfedd3"}
Jul 06 20:49:53.468625 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:49:53.468573    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-x2kkv" podStartSLOduration=67.076297718 podStartE2EDuration="1m11.468558118s" totalImagesPullingTime="4.3922604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:49:53.467246801 +0000 UTC m=+71.933488058" watchObservedRunningTime="2026-07-06 20:49:53.468558118 +0000 UTC m=+71.934799363"
Jul 06 20:50:05.015601 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.015434    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7lk82" podStartSLOduration=78.601679843 podStartE2EDuration="1m23.015418455s" totalImagesPullingTime="4.413738612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:49:53.490380454 +0000 UTC m=+71.956621710" watchObservedRunningTime="2026-07-06 20:50:05.015418455 +0000 UTC m=+83.481659709"
Jul 06 20:50:05.016015 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.016000    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5655df4f5-g8dcn"]
Jul 06 20:50:05.018362 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.018348    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.024499 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.024104    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 06 20:50:05.024499 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.024129    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 20:50:05.024499 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.024114    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hqr5q\""
Jul 06 20:50:05.024649 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.024593    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 20:50:05.024899 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.024883    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 06 20:50:05.024953 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.024919    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 06 20:50:05.024953 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.024941    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 06 20:50:05.034354 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.034332    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5655df4f5-g8dcn"]
Jul 06 20:50:05.140527 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.140489    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btqwz\" (UniqueName: \"kubernetes.io/projected/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-kube-api-access-btqwz\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.140714 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.140562    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.140714 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.140631    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-default-certificate\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.140714 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.140665    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-stats-auth\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.140714 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.140703    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.241110 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.241057    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btqwz\" (UniqueName: \"kubernetes.io/projected/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-kube-api-access-btqwz\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.241110 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.241125    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.241375 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.241159    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-default-certificate\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.241375 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.241175    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-stats-auth\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.241375 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.241206    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.241375 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:05.241272    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 20:50:05.241375 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:05.241332    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle podName:d1e1eb8b-dd63-41a9-9842-be5520a5cb73 nodeName:}" failed. No retries permitted until 2026-07-06 20:50:05.741317096 +0000 UTC m=+84.207558347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle") pod "router-default-5655df4f5-g8dcn" (UID: "d1e1eb8b-dd63-41a9-9842-be5520a5cb73") : configmap references non-existent config key: service-ca.crt
Jul 06 20:50:05.241375 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:05.241349    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs podName:d1e1eb8b-dd63-41a9-9842-be5520a5cb73 nodeName:}" failed. No retries permitted until 2026-07-06 20:50:05.741341707 +0000 UTC m=+84.207582939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs") pod "router-default-5655df4f5-g8dcn" (UID: "d1e1eb8b-dd63-41a9-9842-be5520a5cb73") : secret "router-metrics-certs-default" not found
Jul 06 20:50:05.243623 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.243598    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-default-certificate\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.243710 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.243649    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-stats-auth\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.250162 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.250141    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btqwz\" (UniqueName: \"kubernetes.io/projected/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-kube-api-access-btqwz\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.745387 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.745334    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.745566 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:05.745413    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:05.745566 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:05.745519    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle podName:d1e1eb8b-dd63-41a9-9842-be5520a5cb73 nodeName:}" failed. No retries permitted until 2026-07-06 20:50:06.745502458 +0000 UTC m=+85.211743689 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle") pod "router-default-5655df4f5-g8dcn" (UID: "d1e1eb8b-dd63-41a9-9842-be5520a5cb73") : configmap references non-existent config key: service-ca.crt
Jul 06 20:50:05.745566 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:05.745531    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 20:50:05.745703 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:05.745587    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs podName:d1e1eb8b-dd63-41a9-9842-be5520a5cb73 nodeName:}" failed. No retries permitted until 2026-07-06 20:50:06.745571191 +0000 UTC m=+85.211812426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs") pod "router-default-5655df4f5-g8dcn" (UID: "d1e1eb8b-dd63-41a9-9842-be5520a5cb73") : secret "router-metrics-certs-default" not found
Jul 06 20:50:06.754644 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:06.754603    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:06.755028 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:06.754665    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:06.755028 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:06.754769    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 20:50:06.755028 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:06.754811    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle podName:d1e1eb8b-dd63-41a9-9842-be5520a5cb73 nodeName:}" failed. No retries permitted until 2026-07-06 20:50:08.754784827 +0000 UTC m=+87.221026082 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle") pod "router-default-5655df4f5-g8dcn" (UID: "d1e1eb8b-dd63-41a9-9842-be5520a5cb73") : configmap references non-existent config key: service-ca.crt
Jul 06 20:50:06.755028 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:06.754841    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs podName:d1e1eb8b-dd63-41a9-9842-be5520a5cb73 nodeName:}" failed. No retries permitted until 2026-07-06 20:50:08.75483029 +0000 UTC m=+87.221071522 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs") pod "router-default-5655df4f5-g8dcn" (UID: "d1e1eb8b-dd63-41a9-9842-be5520a5cb73") : secret "router-metrics-certs-default" not found
Jul 06 20:50:08.069901 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.069870    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jnd7m"]
Jul 06 20:50:08.071908 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.071891    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.074657 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.074636    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 20:50:08.074865 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.074849    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 20:50:08.075522 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.075510    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 20:50:08.075574 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.075514    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 20:50:08.075574 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.075555    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mz8wb\""
Jul 06 20:50:08.084188 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.084168    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jnd7m"]
Jul 06 20:50:08.165478 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.165428    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aba8779f-d992-4819-bbac-b519fe6e7e0d-crio-socket\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.165660 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.165490    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aba8779f-d992-4819-bbac-b519fe6e7e0d-data-volume\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.165660 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.165591    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.165660 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.165640    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nrm8\" (UniqueName: \"kubernetes.io/projected/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-api-access-4nrm8\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.165828 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.165676    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.266681 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.266640    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aba8779f-d992-4819-bbac-b519fe6e7e0d-crio-socket\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.266864 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.266698    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aba8779f-d992-4819-bbac-b519fe6e7e0d-data-volume\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.266864 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.266733    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.266864 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.266757    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nrm8\" (UniqueName: \"kubernetes.io/projected/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-api-access-4nrm8\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.266864 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.266769    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aba8779f-d992-4819-bbac-b519fe6e7e0d-crio-socket\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.266864 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.266790    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.267075 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:08.266876    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 20:50:08.267075 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:08.266966    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm podName:aba8779f-d992-4819-bbac-b519fe6e7e0d nodeName:}" failed. No retries permitted until 2026-07-06 20:50:08.766942093 +0000 UTC m=+87.233183343 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jnd7m" (UID: "aba8779f-d992-4819-bbac-b519fe6e7e0d") : configmap "kube-rbac-proxy" not found
Jul 06 20:50:08.267075 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:08.266878    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 20:50:08.267075 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:08.267033    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls podName:aba8779f-d992-4819-bbac-b519fe6e7e0d nodeName:}" failed. No retries permitted until 2026-07-06 20:50:08.76701786 +0000 UTC m=+87.233259096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jnd7m" (UID: "aba8779f-d992-4819-bbac-b519fe6e7e0d") : secret "insights-runtime-extractor-tls" not found
Jul 06 20:50:08.267204 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.267159    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aba8779f-d992-4819-bbac-b519fe6e7e0d-data-volume\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.275321 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.275296    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nrm8\" (UniqueName: \"kubernetes.io/projected/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-api-access-4nrm8\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.771413 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.771362    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.771413 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.771418    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:08.771712 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.771451    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:08.771712 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:08.771537    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 20:50:08.771712 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:08.771556    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 20:50:08.771712 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:08.771603    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:08.771712 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:08.771613    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm podName:aba8779f-d992-4819-bbac-b519fe6e7e0d nodeName:}" failed. No retries permitted until 2026-07-06 20:50:09.771593374 +0000 UTC m=+88.237834608 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jnd7m" (UID: "aba8779f-d992-4819-bbac-b519fe6e7e0d") : configmap "kube-rbac-proxy" not found
Jul 06 20:50:08.771712 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:08.771670    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 20:50:08.771712 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:08.771673    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls podName:aba8779f-d992-4819-bbac-b519fe6e7e0d nodeName:}" failed. No retries permitted until 2026-07-06 20:50:09.771655853 +0000 UTC m=+88.237897085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jnd7m" (UID: "aba8779f-d992-4819-bbac-b519fe6e7e0d") : secret "insights-runtime-extractor-tls" not found
Jul 06 20:50:08.771712 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:08.771702    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle podName:d1e1eb8b-dd63-41a9-9842-be5520a5cb73 nodeName:}" failed. No retries permitted until 2026-07-06 20:50:12.771692951 +0000 UTC m=+91.237934184 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle") pod "router-default-5655df4f5-g8dcn" (UID: "d1e1eb8b-dd63-41a9-9842-be5520a5cb73") : configmap references non-existent config key: service-ca.crt
Jul 06 20:50:08.771712 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:08.771717    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs podName:d1e1eb8b-dd63-41a9-9842-be5520a5cb73 nodeName:}" failed. No retries permitted until 2026-07-06 20:50:12.771711868 +0000 UTC m=+91.237953100 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs") pod "router-default-5655df4f5-g8dcn" (UID: "d1e1eb8b-dd63-41a9-9842-be5520a5cb73") : secret "router-metrics-certs-default" not found
Jul 06 20:50:09.779441 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:09.779399    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:09.779441 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:09.779448    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:09.779901 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:09.779545    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 20:50:09.779901 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:09.779570    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 20:50:09.779901 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:09.779615    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm podName:aba8779f-d992-4819-bbac-b519fe6e7e0d nodeName:}" failed. No retries permitted until 2026-07-06 20:50:11.779600674 +0000 UTC m=+90.245841906 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jnd7m" (UID: "aba8779f-d992-4819-bbac-b519fe6e7e0d") : configmap "kube-rbac-proxy" not found
Jul 06 20:50:09.779901 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:09.779629    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls podName:aba8779f-d992-4819-bbac-b519fe6e7e0d nodeName:}" failed. No retries permitted until 2026-07-06 20:50:11.779623351 +0000 UTC m=+90.245864583 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jnd7m" (UID: "aba8779f-d992-4819-bbac-b519fe6e7e0d") : secret "insights-runtime-extractor-tls" not found
Jul 06 20:50:10.053972 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:10.053898    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ptn9v_1236b937-e2b7-41ac-a612-142a60a86806/dns-node-resolver/0.log"
Jul 06 20:50:10.859654 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:10.859626    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qxhpq_8f813d5c-13a4-485c-b5ab-5f26fd9f84fe/node-ca/0.log"
Jul 06 20:50:11.796236 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:11.796197    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:11.796236 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:11.796244    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:11.796492 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:11.796323    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 20:50:11.796492 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:11.796356    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 20:50:11.796492 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:11.796396    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm podName:aba8779f-d992-4819-bbac-b519fe6e7e0d nodeName:}" failed. No retries permitted until 2026-07-06 20:50:15.79637883 +0000 UTC m=+94.262620062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jnd7m" (UID: "aba8779f-d992-4819-bbac-b519fe6e7e0d") : configmap "kube-rbac-proxy" not found
Jul 06 20:50:11.796492 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:11.796411    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls podName:aba8779f-d992-4819-bbac-b519fe6e7e0d nodeName:}" failed. No retries permitted until 2026-07-06 20:50:15.796405706 +0000 UTC m=+94.262646938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jnd7m" (UID: "aba8779f-d992-4819-bbac-b519fe6e7e0d") : secret "insights-runtime-extractor-tls" not found
Jul 06 20:50:11.860133 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:11.860082    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fswwk_a36da1e0-2692-4633-b274-158c32c67c8a/serve-healthcheck-canary/0.log"
Jul 06 20:50:12.804411 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:12.804358    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:12.804840 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:12.804494    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:12.804840 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:12.804534    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 20:50:12.804840 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:12.804611    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs podName:d1e1eb8b-dd63-41a9-9842-be5520a5cb73 nodeName:}" failed. No retries permitted until 2026-07-06 20:50:20.80459577 +0000 UTC m=+99.270837006 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs") pod "router-default-5655df4f5-g8dcn" (UID: "d1e1eb8b-dd63-41a9-9842-be5520a5cb73") : secret "router-metrics-certs-default" not found
Jul 06 20:50:12.804840 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:12.804644    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle podName:d1e1eb8b-dd63-41a9-9842-be5520a5cb73 nodeName:}" failed. No retries permitted until 2026-07-06 20:50:20.804625487 +0000 UTC m=+99.270866722 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle") pod "router-default-5655df4f5-g8dcn" (UID: "d1e1eb8b-dd63-41a9-9842-be5520a5cb73") : configmap references non-existent config key: service-ca.crt
Jul 06 20:50:15.831599 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:15.831558    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:15.831599 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:15.831602    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:15.832029 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:15.831686    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 20:50:15.832029 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:15.831762    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm podName:aba8779f-d992-4819-bbac-b519fe6e7e0d nodeName:}" failed. No retries permitted until 2026-07-06 20:50:23.831742761 +0000 UTC m=+102.297983996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jnd7m" (UID: "aba8779f-d992-4819-bbac-b519fe6e7e0d") : configmap "kube-rbac-proxy" not found
Jul 06 20:50:15.832029 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:15.831824    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 20:50:15.832029 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:15.831906    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls podName:aba8779f-d992-4819-bbac-b519fe6e7e0d nodeName:}" failed. No retries permitted until 2026-07-06 20:50:23.831892004 +0000 UTC m=+102.298133237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jnd7m" (UID: "aba8779f-d992-4819-bbac-b519fe6e7e0d") : secret "insights-runtime-extractor-tls" not found
Jul 06 20:50:20.873773 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:20.873740    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:20.874190 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:20.873782    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:50:20.874190 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:20.873835    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:20.874190 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:20.873868    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:50:20.874582 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:20.874557    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-service-ca-bundle\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:20.876259 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:20.876239    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4133d58d-63a9-46cd-b3d5-eddf0406fe31-metrics-tls\") pod \"dns-default-6mllp\" (UID: \"4133d58d-63a9-46cd-b3d5-eddf0406fe31\") " pod="openshift-dns/dns-default-6mllp"
Jul 06 20:50:20.876259 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:20.876253    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e1eb8b-dd63-41a9-9842-be5520a5cb73-metrics-certs\") pod \"router-default-5655df4f5-g8dcn\" (UID: \"d1e1eb8b-dd63-41a9-9842-be5520a5cb73\") " pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:20.876342 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:20.876252    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls\") pod \"image-registry-68d4f94c5d-xzkmd\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") " pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:50:20.927503 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:20.927429    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:21.051187 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.051154    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5655df4f5-g8dcn"]
Jul 06 20:50:21.054210 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:50:21.054179    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1e1eb8b_dd63_41a9_9842_be5520a5cb73.slice/crio-5f5b9ac58916beff29395a38bc10883cd0616d222149da9888b3f4d4e8e257a1 WatchSource:0}: Error finding container 5f5b9ac58916beff29395a38bc10883cd0616d222149da9888b3f4d4e8e257a1: Status 404 returned error can't find the container with id 5f5b9ac58916beff29395a38bc10883cd0616d222149da9888b3f4d4e8e257a1
Jul 06 20:50:21.114956 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.114927    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ldn5x\""
Jul 06 20:50:21.123231 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.123201    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:50:21.132736 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.132713    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cpffn\""
Jul 06 20:50:21.140836 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.140812    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6mllp"
Jul 06 20:50:21.268206 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.267914    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68d4f94c5d-xzkmd"]
Jul 06 20:50:21.278080 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.278042    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6mllp"]
Jul 06 20:50:21.281092 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:50:21.281067    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4133d58d_63a9_46cd_b3d5_eddf0406fe31.slice/crio-0ef615b68ad57fbe8984bab1f310e3b65ef74429543dceca68d95a89ccf2b772 WatchSource:0}: Error finding container 0ef615b68ad57fbe8984bab1f310e3b65ef74429543dceca68d95a89ccf2b772: Status 404 returned error can't find the container with id 0ef615b68ad57fbe8984bab1f310e3b65ef74429543dceca68d95a89ccf2b772
Jul 06 20:50:21.519201 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.519154    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd" event={"ID":"1cd6b87c-70df-459e-9f27-d458d34d1bed","Type":"ContainerStarted","Data":"c95703ad99fd1b58aadad28f21e4157d5c376b84acbb8ca509a6419fa255785a"}
Jul 06 20:50:21.519404 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.519229    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:50:21.519404 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.519253    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd" event={"ID":"1cd6b87c-70df-459e-9f27-d458d34d1bed","Type":"ContainerStarted","Data":"8ed6296e79c2a320a1342d1446d5378bda82f9e9eb5b62b8ac04e14336013386"}
Jul 06 20:50:21.520737 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.520710    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5655df4f5-g8dcn" event={"ID":"d1e1eb8b-dd63-41a9-9842-be5520a5cb73","Type":"ContainerStarted","Data":"c6fe02547e27542c293269417f1adb7713ff6451081337c2023e76001abb7986"}
Jul 06 20:50:21.520866 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.520758    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5655df4f5-g8dcn" event={"ID":"d1e1eb8b-dd63-41a9-9842-be5520a5cb73","Type":"ContainerStarted","Data":"5f5b9ac58916beff29395a38bc10883cd0616d222149da9888b3f4d4e8e257a1"}
Jul 06 20:50:21.521716 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.521697    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6mllp" event={"ID":"4133d58d-63a9-46cd-b3d5-eddf0406fe31","Type":"ContainerStarted","Data":"0ef615b68ad57fbe8984bab1f310e3b65ef74429543dceca68d95a89ccf2b772"}
Jul 06 20:50:21.542378 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.542311    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd" podStartSLOduration=76.542292319 podStartE2EDuration="1m16.542292319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:49:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:21.541792042 +0000 UTC m=+100.008033291" watchObservedRunningTime="2026-07-06 20:50:21.542292319 +0000 UTC m=+100.008533574"
Jul 06 20:50:21.569208 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.569148    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5655df4f5-g8dcn" podStartSLOduration=17.569130981 podStartE2EDuration="17.569130981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:50:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:21.568586088 +0000 UTC m=+100.034827341" watchObservedRunningTime="2026-07-06 20:50:21.569130981 +0000 UTC m=+100.035372214"
Jul 06 20:50:21.928229 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.928138    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:21.930930 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:21.930906    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:22.525088 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:22.525060    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:22.526511 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:22.526440    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5655df4f5-g8dcn"
Jul 06 20:50:23.534225 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:23.534189    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6mllp" event={"ID":"4133d58d-63a9-46cd-b3d5-eddf0406fe31","Type":"ContainerStarted","Data":"4733be64887fbc6198707e66c37caa7085addfedde301dc4442a1f184ce5184a"}
Jul 06 20:50:23.534666 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:23.534275    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6mllp" event={"ID":"4133d58d-63a9-46cd-b3d5-eddf0406fe31","Type":"ContainerStarted","Data":"69357a305cd4495041a00e9f9f5aaff185d048fc9ebce303e464b234f0178621"}
Jul 06 20:50:23.534666 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:23.534306    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6mllp"
Jul 06 20:50:23.558276 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:23.558228    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6mllp" podStartSLOduration=66.111276021 podStartE2EDuration="1m7.558211904s" totalImagesPullingTime="1.446935883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:49:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:23.557396154 +0000 UTC m=+102.023637408" watchObservedRunningTime="2026-07-06 20:50:23.558211904 +0000 UTC m=+102.024453195"
Jul 06 20:50:23.897980 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:23.897887    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:23.897980 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:23.897930    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:23.898523 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:23.898496    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aba8779f-d992-4819-bbac-b519fe6e7e0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:23.900286 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:23.900269    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aba8779f-d992-4819-bbac-b519fe6e7e0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jnd7m\" (UID: \"aba8779f-d992-4819-bbac-b519fe6e7e0d\") " pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:23.981729 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:23.981693    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jnd7m"
Jul 06 20:50:24.132134 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:24.132106    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jnd7m"]
Jul 06 20:50:24.135119 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:50:24.135093    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaba8779f_d992_4819_bbac_b519fe6e7e0d.slice/crio-acdb6be9537bc682a460ca9e4a423ee969514510971a60a8140808cc2c79a3b6 WatchSource:0}: Error finding container acdb6be9537bc682a460ca9e4a423ee969514510971a60a8140808cc2c79a3b6: Status 404 returned error can't find the container with id acdb6be9537bc682a460ca9e4a423ee969514510971a60a8140808cc2c79a3b6
Jul 06 20:50:24.457055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:24.457025    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-x2kkv"
Jul 06 20:50:24.538883 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:24.538838    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jnd7m" event={"ID":"aba8779f-d992-4819-bbac-b519fe6e7e0d","Type":"ContainerStarted","Data":"acdb6be9537bc682a460ca9e4a423ee969514510971a60a8140808cc2c79a3b6"}
Jul 06 20:50:28.557020 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:28.556983    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jnd7m" event={"ID":"aba8779f-d992-4819-bbac-b519fe6e7e0d","Type":"ContainerStarted","Data":"2dc20a782d09237f220647c1810a22c8ab44ed16aa57f680bab376d881ded734"}
Jul 06 20:50:29.562496 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:29.562441    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jnd7m" event={"ID":"aba8779f-d992-4819-bbac-b519fe6e7e0d","Type":"ContainerStarted","Data":"9989e027b9e6725f620bbc0957ec55c9344e76a3abb0bf3828de8162408e48cc"}
Jul 06 20:50:31.570145 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:31.570107    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jnd7m" event={"ID":"aba8779f-d992-4819-bbac-b519fe6e7e0d","Type":"ContainerStarted","Data":"b6d5296410a3e4d22250484b15f3cf659f219fa9bb606f62df845d862e1c413e"}
Jul 06 20:50:32.415356 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.415303    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jnd7m" podStartSLOduration=17.281778381 podStartE2EDuration="24.415281157s" totalImagesPullingTime="7.133502776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:50:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:31.600601511 +0000 UTC m=+110.066842761" watchObservedRunningTime="2026-07-06 20:50:32.415281157 +0000 UTC m=+110.881522412"
Jul 06 20:50:32.415741 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.415720    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8"]
Jul 06 20:50:32.417606 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.417586    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8"
Jul 06 20:50:32.420787 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.420762    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-wrmqt\""
Jul 06 20:50:32.420891 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.420791    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 20:50:32.421157 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.421140    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 20:50:32.421157 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.421146    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 20:50:32.421279 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.421245    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 06 20:50:32.431497 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.431475    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8"]
Jul 06 20:50:32.464160 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.464124    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvvnr\" (UniqueName: \"kubernetes.io/projected/e2f105ff-1fe0-4f5f-835c-7ffcdb862c22-kube-api-access-hvvnr\") pod \"managed-serviceaccount-addon-agent-55fbd94458-cvqm8\" (UID: \"e2f105ff-1fe0-4f5f-835c-7ffcdb862c22\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8"
Jul 06 20:50:32.464160 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.464163    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2f105ff-1fe0-4f5f-835c-7ffcdb862c22-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55fbd94458-cvqm8\" (UID: \"e2f105ff-1fe0-4f5f-835c-7ffcdb862c22\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8"
Jul 06 20:50:32.546926 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.546891    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-j67bg"]
Jul 06 20:50:32.552848 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.552829    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-j67bg"
Jul 06 20:50:32.564958 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.564934    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfmbv\" (UniqueName: \"kubernetes.io/projected/68669423-993b-4d84-a433-ba9a3ac918ec-kube-api-access-lfmbv\") pod \"network-check-source-989f8b8dc-j67bg\" (UID: \"68669423-993b-4d84-a433-ba9a3ac918ec\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-j67bg"
Jul 06 20:50:32.565077 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.564972    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvvnr\" (UniqueName: \"kubernetes.io/projected/e2f105ff-1fe0-4f5f-835c-7ffcdb862c22-kube-api-access-hvvnr\") pod \"managed-serviceaccount-addon-agent-55fbd94458-cvqm8\" (UID: \"e2f105ff-1fe0-4f5f-835c-7ffcdb862c22\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8"
Jul 06 20:50:32.565077 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.564991    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2f105ff-1fe0-4f5f-835c-7ffcdb862c22-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55fbd94458-cvqm8\" (UID: \"e2f105ff-1fe0-4f5f-835c-7ffcdb862c22\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8"
Jul 06 20:50:32.567407 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.567379    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2f105ff-1fe0-4f5f-835c-7ffcdb862c22-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55fbd94458-cvqm8\" (UID: \"e2f105ff-1fe0-4f5f-835c-7ffcdb862c22\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8"
Jul 06 20:50:32.571174 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.571152    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-txfkp\""
Jul 06 20:50:32.589493 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.588735    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvvnr\" (UniqueName: \"kubernetes.io/projected/e2f105ff-1fe0-4f5f-835c-7ffcdb862c22-kube-api-access-hvvnr\") pod \"managed-serviceaccount-addon-agent-55fbd94458-cvqm8\" (UID: \"e2f105ff-1fe0-4f5f-835c-7ffcdb862c22\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8"
Jul 06 20:50:32.599220 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.599183    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-j67bg"]
Jul 06 20:50:32.666059 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.665970    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfmbv\" (UniqueName: \"kubernetes.io/projected/68669423-993b-4d84-a433-ba9a3ac918ec-kube-api-access-lfmbv\") pod \"network-check-source-989f8b8dc-j67bg\" (UID: \"68669423-993b-4d84-a433-ba9a3ac918ec\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-j67bg"
Jul 06 20:50:32.691199 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.691160    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfmbv\" (UniqueName: \"kubernetes.io/projected/68669423-993b-4d84-a433-ba9a3ac918ec-kube-api-access-lfmbv\") pod \"network-check-source-989f8b8dc-j67bg\" (UID: \"68669423-993b-4d84-a433-ba9a3ac918ec\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-j67bg"
Jul 06 20:50:32.732249 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.732212    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8"
Jul 06 20:50:32.861753 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.861723    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-j67bg"
Jul 06 20:50:32.867109 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.867081    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8"]
Jul 06 20:50:32.869448 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:50:32.869422    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2f105ff_1fe0_4f5f_835c_7ffcdb862c22.slice/crio-6dd73266526f18a83e73231b630ecbb62c1befe0211e99f098396ba1e65ddf93 WatchSource:0}: Error finding container 6dd73266526f18a83e73231b630ecbb62c1befe0211e99f098396ba1e65ddf93: Status 404 returned error can't find the container with id 6dd73266526f18a83e73231b630ecbb62c1befe0211e99f098396ba1e65ddf93
Jul 06 20:50:32.997270 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:32.997235    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-j67bg"]
Jul 06 20:50:33.000170 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:50:33.000144    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68669423_993b_4d84_a433_ba9a3ac918ec.slice/crio-dac96598d261889534a2b5dab5633142d162f686dffc4c38a4cf5ad476cfa1b9 WatchSource:0}: Error finding container dac96598d261889534a2b5dab5633142d162f686dffc4c38a4cf5ad476cfa1b9: Status 404 returned error can't find the container with id dac96598d261889534a2b5dab5633142d162f686dffc4c38a4cf5ad476cfa1b9
Jul 06 20:50:33.541713 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:33.541678    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6mllp"
Jul 06 20:50:33.582837 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:33.582799    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-j67bg" event={"ID":"68669423-993b-4d84-a433-ba9a3ac918ec","Type":"ContainerStarted","Data":"0d28864e46b95468cafdc341b8eaff4fb6a3f478bdd45f4caab88c55d6981b7d"}
Jul 06 20:50:33.583293 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:33.582863    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-j67bg" event={"ID":"68669423-993b-4d84-a433-ba9a3ac918ec","Type":"ContainerStarted","Data":"dac96598d261889534a2b5dab5633142d162f686dffc4c38a4cf5ad476cfa1b9"}
Jul 06 20:50:33.585503 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:33.585471    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8" event={"ID":"e2f105ff-1fe0-4f5f-835c-7ffcdb862c22","Type":"ContainerStarted","Data":"6dd73266526f18a83e73231b630ecbb62c1befe0211e99f098396ba1e65ddf93"}
Jul 06 20:50:33.627640 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:33.627535    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-j67bg" podStartSLOduration=1.627511504 podStartE2EDuration="1.627511504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:33.625839771 +0000 UTC m=+112.092081027" watchObservedRunningTime="2026-07-06 20:50:33.627511504 +0000 UTC m=+112.093752759"
Jul 06 20:50:35.592847 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:35.592810    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8" event={"ID":"e2f105ff-1fe0-4f5f-835c-7ffcdb862c22","Type":"ContainerStarted","Data":"a2d379994be41ef66f970db9163fa282009cf0bda958cc77f9ce6a6b13d0d531"}
Jul 06 20:50:35.620642 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:35.620583    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fbd94458-cvqm8" podStartSLOduration=1.342856665 podStartE2EDuration="3.620566933s" totalImagesPullingTime="2.277710268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:35.620449162 +0000 UTC m=+114.086690416" watchObservedRunningTime="2026-07-06 20:50:35.620566933 +0000 UTC m=+114.086808186"
Jul 06 20:50:40.057470 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:40.057427    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbp6v"]
Jul 06 20:50:40.059707 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:40.059690    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbp6v"
Jul 06 20:50:40.062248 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:40.062219    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 06 20:50:40.062978 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:40.062954    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 06 20:50:40.063349 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:40.063330    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-q5tqg\""
Jul 06 20:50:40.075871 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:40.075849    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbp6v"]
Jul 06 20:50:40.119774 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:40.119736    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6dtw\" (UniqueName: \"kubernetes.io/projected/b6935f66-ae75-4b3a-8b05-904a42a0dacb-kube-api-access-c6dtw\") pod \"migrator-d57d58cc9-pbp6v\" (UID: \"b6935f66-ae75-4b3a-8b05-904a42a0dacb\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbp6v"
Jul 06 20:50:40.220873 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:40.220836    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6dtw\" (UniqueName: \"kubernetes.io/projected/b6935f66-ae75-4b3a-8b05-904a42a0dacb-kube-api-access-c6dtw\") pod \"migrator-d57d58cc9-pbp6v\" (UID: \"b6935f66-ae75-4b3a-8b05-904a42a0dacb\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbp6v"
Jul 06 20:50:40.229602 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:40.229568    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6dtw\" (UniqueName: \"kubernetes.io/projected/b6935f66-ae75-4b3a-8b05-904a42a0dacb-kube-api-access-c6dtw\") pod \"migrator-d57d58cc9-pbp6v\" (UID: \"b6935f66-ae75-4b3a-8b05-904a42a0dacb\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbp6v"
Jul 06 20:50:40.368902 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:40.368790    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbp6v"
Jul 06 20:50:40.514192 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:40.514162    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbp6v"]
Jul 06 20:50:40.517269 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:50:40.517248    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6935f66_ae75_4b3a_8b05_904a42a0dacb.slice/crio-d7d6e2061df0ac30de33517a20d7d4dcf397c26e6fe86503edf8602072e92f97 WatchSource:0}: Error finding container d7d6e2061df0ac30de33517a20d7d4dcf397c26e6fe86503edf8602072e92f97: Status 404 returned error can't find the container with id d7d6e2061df0ac30de33517a20d7d4dcf397c26e6fe86503edf8602072e92f97
Jul 06 20:50:40.609545 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:40.609512    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbp6v" event={"ID":"b6935f66-ae75-4b3a-8b05-904a42a0dacb","Type":"ContainerStarted","Data":"d7d6e2061df0ac30de33517a20d7d4dcf397c26e6fe86503edf8602072e92f97"}
Jul 06 20:50:41.129030 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:41.128988    2566 patch_prober.go:28] interesting pod/image-registry-68d4f94c5d-xzkmd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 06 20:50:41.129480 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:41.129060    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd" podUID="1cd6b87c-70df-459e-9f27-d458d34d1bed" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 20:50:42.529093 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:42.529066    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:50:42.617647 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:42.617600    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbp6v" event={"ID":"b6935f66-ae75-4b3a-8b05-904a42a0dacb","Type":"ContainerStarted","Data":"b34d37534df006ec8757faf175dd82a20c96d0601104042465a18a42f3baf033"}
Jul 06 20:50:42.617807 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:42.617660    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbp6v" event={"ID":"b6935f66-ae75-4b3a-8b05-904a42a0dacb","Type":"ContainerStarted","Data":"7c7cef7d778eb1d7eb91c9939c4f3a1a26db09a2407efa473d697301717f504d"}
Jul 06 20:50:42.635528 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:42.635479    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbp6v" podStartSLOduration=1.335919568 podStartE2EDuration="2.635442788s" totalImagesPullingTime="1.29952322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:50:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:42.634344217 +0000 UTC m=+121.100585470" watchObservedRunningTime="2026-07-06 20:50:42.635442788 +0000 UTC m=+121.101684084"
Jul 06 20:50:44.556652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.556617    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wsr8m"]
Jul 06 20:50:44.558971 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.558953    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.563066 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.563042    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 20:50:44.563193 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.563141    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 20:50:44.565019 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.564999    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 20:50:44.565135 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.565052    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 20:50:44.565135 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.565110    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 20:50:44.565597 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.565254    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4rs94\""
Jul 06 20:50:44.565597 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.565351    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 20:50:44.568027 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.568005    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"]
Jul 06 20:50:44.570084 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.570068    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.572510 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.572495    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 06 20:50:44.572922 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.572905    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-vhwdn\""
Jul 06 20:50:44.573153 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.573138    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 06 20:50:44.573800 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.573787    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 06 20:50:44.586428 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.586395    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"]
Jul 06 20:50:44.655102 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655068    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12ddb476-403e-4335-8030-935061abc21a-sys\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.655299 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655108    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12ddb476-403e-4335-8030-935061abc21a-node-exporter-accelerators-collector-config\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.655299 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655131    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.655299 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655164    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bc829e1-259f-4427-8499-59dab0aa7a4b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.655299 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655180    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44k9x\" (UniqueName: \"kubernetes.io/projected/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-api-access-44k9x\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.655299 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655196    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12ddb476-403e-4335-8030-935061abc21a-node-exporter-tls\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.655299 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655222    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12ddb476-403e-4335-8030-935061abc21a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.655299 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655269    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12ddb476-403e-4335-8030-935061abc21a-root\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.655554 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655303    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12ddb476-403e-4335-8030-935061abc21a-node-exporter-textfile\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.655554 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655326    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12ddb476-403e-4335-8030-935061abc21a-metrics-client-ca\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.655554 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655344    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.655554 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655369    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2bc829e1-259f-4427-8499-59dab0aa7a4b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.655554 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655429    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.655554 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655484    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6wln\" (UniqueName: \"kubernetes.io/projected/12ddb476-403e-4335-8030-935061abc21a-kube-api-access-g6wln\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.655554 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.655504    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12ddb476-403e-4335-8030-935061abc21a-node-exporter-wtmp\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.755869 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.755831    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bc829e1-259f-4427-8499-59dab0aa7a4b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.755869 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.755875    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44k9x\" (UniqueName: \"kubernetes.io/projected/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-api-access-44k9x\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.756117 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.755902    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12ddb476-403e-4335-8030-935061abc21a-node-exporter-tls\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.756117 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.755940    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12ddb476-403e-4335-8030-935061abc21a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.756117 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.755968    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12ddb476-403e-4335-8030-935061abc21a-root\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.756117 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.755992    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12ddb476-403e-4335-8030-935061abc21a-node-exporter-textfile\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.756117 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756017    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12ddb476-403e-4335-8030-935061abc21a-metrics-client-ca\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.756117 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756041    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.756117 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756068    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2bc829e1-259f-4427-8499-59dab0aa7a4b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.756117 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756085    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12ddb476-403e-4335-8030-935061abc21a-root\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.756117 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756108    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.756575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756162    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6wln\" (UniqueName: \"kubernetes.io/projected/12ddb476-403e-4335-8030-935061abc21a-kube-api-access-g6wln\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.756575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756196    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12ddb476-403e-4335-8030-935061abc21a-node-exporter-wtmp\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.756575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756224    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12ddb476-403e-4335-8030-935061abc21a-sys\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.756575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756259    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12ddb476-403e-4335-8030-935061abc21a-node-exporter-accelerators-collector-config\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.756575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756371    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12ddb476-403e-4335-8030-935061abc21a-node-exporter-textfile\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.756575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756386    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12ddb476-403e-4335-8030-935061abc21a-node-exporter-wtmp\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.756575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756422    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.756575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756571    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12ddb476-403e-4335-8030-935061abc21a-sys\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.756981 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:44.756579    2566 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 06 20:50:44.756981 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:50:44.756734    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-state-metrics-tls podName:2bc829e1-259f-4427-8499-59dab0aa7a4b nodeName:}" failed. No retries permitted until 2026-07-06 20:50:45.256715901 +0000 UTC m=+123.722957144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-5xxtk" (UID: "2bc829e1-259f-4427-8499-59dab0aa7a4b") : secret "kube-state-metrics-tls" not found
Jul 06 20:50:44.756981 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756762    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bc829e1-259f-4427-8499-59dab0aa7a4b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.756981 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.756840    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12ddb476-403e-4335-8030-935061abc21a-node-exporter-accelerators-collector-config\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.757202 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.757055    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2bc829e1-259f-4427-8499-59dab0aa7a4b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.757202 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.757122    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12ddb476-403e-4335-8030-935061abc21a-metrics-client-ca\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.757299 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.757244    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.758640 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.758624    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.758722 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.758631    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12ddb476-403e-4335-8030-935061abc21a-node-exporter-tls\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.758892 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.758872    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12ddb476-403e-4335-8030-935061abc21a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.769828 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.769802    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44k9x\" (UniqueName: \"kubernetes.io/projected/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-api-access-44k9x\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:44.770317 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.770293    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6wln\" (UniqueName: \"kubernetes.io/projected/12ddb476-403e-4335-8030-935061abc21a-kube-api-access-g6wln\") pod \"node-exporter-wsr8m\" (UID: \"12ddb476-403e-4335-8030-935061abc21a\") " pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.869073 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:44.868972    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wsr8m"
Jul 06 20:50:44.879285 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:50:44.879253    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ddb476_403e_4335_8030_935061abc21a.slice/crio-d296d3acdb3582ab250dd61dc4792b39b8526f475bf47e5059e185a5efb969a0 WatchSource:0}: Error finding container d296d3acdb3582ab250dd61dc4792b39b8526f475bf47e5059e185a5efb969a0: Status 404 returned error can't find the container with id d296d3acdb3582ab250dd61dc4792b39b8526f475bf47e5059e185a5efb969a0
Jul 06 20:50:45.260799 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.260766    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:45.263221 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.263196    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bc829e1-259f-4427-8499-59dab0aa7a4b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5xxtk\" (UID: \"2bc829e1-259f-4427-8499-59dab0aa7a4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:45.478649 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.478615    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"
Jul 06 20:50:45.631165 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.631052    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wsr8m" event={"ID":"12ddb476-403e-4335-8030-935061abc21a","Type":"ContainerStarted","Data":"d296d3acdb3582ab250dd61dc4792b39b8526f475bf47e5059e185a5efb969a0"}
Jul 06 20:50:45.644106 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.644076    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 20:50:45.647572 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.647551    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.650925 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.650878    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 06 20:50:45.651101 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.651082    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rj2vq\""
Jul 06 20:50:45.651196 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.651094    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 06 20:50:45.651839 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.651816    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 06 20:50:45.652903 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.652886    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 06 20:50:45.655525 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.655506    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 06 20:50:45.656015 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.655507    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 06 20:50:45.656358 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.656338    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 06 20:50:45.659049 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.659030    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 06 20:50:45.664705 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.664683    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 06 20:50:45.670319 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.670288    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 20:50:45.765816 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.765787    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-config-volume\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.765983 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.765831    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1932c276-cb9c-428c-aa9d-b804a8bd4840-config-out\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.765983 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.765857    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.765983 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.765923    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-web-config\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.766153 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.765983    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1932c276-cb9c-428c-aa9d-b804a8bd4840-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.766153 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.766008    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1932c276-cb9c-428c-aa9d-b804a8bd4840-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.766153 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.766047    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.766153 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.766115    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1932c276-cb9c-428c-aa9d-b804a8bd4840-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.766153 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.766144    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1932c276-cb9c-428c-aa9d-b804a8bd4840-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.766326 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.766180    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.766326 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.766207    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.766326 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.766257    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.766326 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.766289    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg8ft\" (UniqueName: \"kubernetes.io/projected/1932c276-cb9c-428c-aa9d-b804a8bd4840-kube-api-access-sg8ft\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.828240 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.828207    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk"]
Jul 06 20:50:45.831579 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:50:45.831552    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc829e1_259f_4427_8499_59dab0aa7a4b.slice/crio-6128c80aa5bc5a8d90b4b94d5da4cedddf7867702f69cbaa5314dae009dd7321 WatchSource:0}: Error finding container 6128c80aa5bc5a8d90b4b94d5da4cedddf7867702f69cbaa5314dae009dd7321: Status 404 returned error can't find the container with id 6128c80aa5bc5a8d90b4b94d5da4cedddf7867702f69cbaa5314dae009dd7321
Jul 06 20:50:45.867205 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.867165    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1932c276-cb9c-428c-aa9d-b804a8bd4840-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.867205 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.867203    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1932c276-cb9c-428c-aa9d-b804a8bd4840-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.867410 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.867227    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.867410 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.867254    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.867410 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.867281    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.867410 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.867328    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg8ft\" (UniqueName: \"kubernetes.io/projected/1932c276-cb9c-428c-aa9d-b804a8bd4840-kube-api-access-sg8ft\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.867410 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.867374    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-config-volume\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.867410 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.867409    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1932c276-cb9c-428c-aa9d-b804a8bd4840-config-out\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.867745 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.867434    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.867745 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.867515    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-web-config\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.867745 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.867567    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1932c276-cb9c-428c-aa9d-b804a8bd4840-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.867745 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.867593    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1932c276-cb9c-428c-aa9d-b804a8bd4840-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.867745 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.867628    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.868199 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.868015    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1932c276-cb9c-428c-aa9d-b804a8bd4840-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.869707 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.869107    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1932c276-cb9c-428c-aa9d-b804a8bd4840-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.869707 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.869651    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1932c276-cb9c-428c-aa9d-b804a8bd4840-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.871031 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.870990    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.871031 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.870989    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1932c276-cb9c-428c-aa9d-b804a8bd4840-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.871627 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.871192    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-web-config\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.871627 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.871404    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.871906 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.871885    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.872589 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.872568    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.872720 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.872699    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.872784 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.872734    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1932c276-cb9c-428c-aa9d-b804a8bd4840-config-out\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.873885 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.873865    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-config-volume\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.881561 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.881514    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg8ft\" (UniqueName: \"kubernetes.io/projected/1932c276-cb9c-428c-aa9d-b804a8bd4840-kube-api-access-sg8ft\") pod \"alertmanager-main-0\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:45.958800 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:45.958764    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:50:46.094052 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:46.094030    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 20:50:46.096321 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:50:46.096295    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1932c276_cb9c_428c_aa9d_b804a8bd4840.slice/crio-c6976647c6227a45703c82b4fa9659f95249c46a288d8cc6ec4885e3faaebe95 WatchSource:0}: Error finding container c6976647c6227a45703c82b4fa9659f95249c46a288d8cc6ec4885e3faaebe95: Status 404 returned error can't find the container with id c6976647c6227a45703c82b4fa9659f95249c46a288d8cc6ec4885e3faaebe95
Jul 06 20:50:46.637623 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:46.637566    2566 generic.go:356] "Generic (PLEG): container finished" podID="12ddb476-403e-4335-8030-935061abc21a" containerID="153310c4418eed729c1012eca115445c3f44b8f872440210d26070d15b909613" exitCode=0
Jul 06 20:50:46.638099 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:46.637660    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wsr8m" event={"ID":"12ddb476-403e-4335-8030-935061abc21a","Type":"ContainerDied","Data":"153310c4418eed729c1012eca115445c3f44b8f872440210d26070d15b909613"}
Jul 06 20:50:46.639631 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:46.639604    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerStarted","Data":"c6976647c6227a45703c82b4fa9659f95249c46a288d8cc6ec4885e3faaebe95"}
Jul 06 20:50:46.640995 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:46.640973    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk" event={"ID":"2bc829e1-259f-4427-8499-59dab0aa7a4b","Type":"ContainerStarted","Data":"6128c80aa5bc5a8d90b4b94d5da4cedddf7867702f69cbaa5314dae009dd7321"}
Jul 06 20:50:47.646563 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:47.646438    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wsr8m" event={"ID":"12ddb476-403e-4335-8030-935061abc21a","Type":"ContainerStarted","Data":"d19c2f90bb7beaa876c550afed5728136f6df396139309de2e8208f43b0d0895"}
Jul 06 20:50:47.646964 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:47.646566    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wsr8m" event={"ID":"12ddb476-403e-4335-8030-935061abc21a","Type":"ContainerStarted","Data":"bd257e6d7c8ebb79a15c3ae8ff912e5c2da4a65a145c5caef72725dae2ff7d91"}
Jul 06 20:50:47.648005 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:47.647978    2566 generic.go:356] "Generic (PLEG): container finished" podID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerID="0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d" exitCode=0
Jul 06 20:50:47.648116 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:47.648061    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerDied","Data":"0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d"}
Jul 06 20:50:47.650143 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:47.650124    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk" event={"ID":"2bc829e1-259f-4427-8499-59dab0aa7a4b","Type":"ContainerStarted","Data":"aae40b8779d6be445557d4c209d3760935763f1dce85a854d53b64b819eec6e4"}
Jul 06 20:50:47.650213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:47.650164    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk" event={"ID":"2bc829e1-259f-4427-8499-59dab0aa7a4b","Type":"ContainerStarted","Data":"98caff8827f540a69fbd353a34b7b64b81374bd4c80412824efba875ef9d9347"}
Jul 06 20:50:47.650213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:47.650183    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk" event={"ID":"2bc829e1-259f-4427-8499-59dab0aa7a4b","Type":"ContainerStarted","Data":"ed43d8c59ce9702868b86f3ebc5f9911880fec051b5638ff39f184e2f3cb01b8"}
Jul 06 20:50:47.671598 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:47.671554    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wsr8m" podStartSLOduration=2.7663218240000003 podStartE2EDuration="3.671539084s" totalImagesPullingTime="836.716202ms" totalInitContainerRuntime="68.501058ms" isStatefulPod=true podCreationTimestamp="2026-07-06 20:50:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:47.670893361 +0000 UTC m=+126.137134628" watchObservedRunningTime="2026-07-06 20:50:47.671539084 +0000 UTC m=+126.137780338"
Jul 06 20:50:47.702259 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:47.702204    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5xxtk" podStartSLOduration=2.428694666 podStartE2EDuration="3.702186574s" totalImagesPullingTime="1.273491908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:50:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:47.70129756 +0000 UTC m=+126.167538813" watchObservedRunningTime="2026-07-06 20:50:47.702186574 +0000 UTC m=+126.168427828"
Jul 06 20:50:48.331631 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.331593    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-796bdb75ff-s74mp"]
Jul 06 20:50:48.333642 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.333612    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.336009 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.335984    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m4hzh\""
Jul 06 20:50:48.336401 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.336381    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 06 20:50:48.337025 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.337002    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 06 20:50:48.337118 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.337030    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 06 20:50:48.337170 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.337112    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 06 20:50:48.337305 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.337270    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 06 20:50:48.337436 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.337320    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 06 20:50:48.337781 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.337763    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 06 20:50:48.352876 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.352850    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-796bdb75ff-s74mp"]
Jul 06 20:50:48.393301 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.393271    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-oauth-serving-cert\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.393507 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.393320    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-config\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.393507 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.393402    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-oauth-config\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.393613 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.393506    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-serving-cert\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.393613 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.393557    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-service-ca\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.393696 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.393623    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfrs8\" (UniqueName: \"kubernetes.io/projected/02e480bf-635e-4b3e-a2a6-84db3f14ff03-kube-api-access-zfrs8\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.494236 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.494200    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-service-ca\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.494236 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.494242    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfrs8\" (UniqueName: \"kubernetes.io/projected/02e480bf-635e-4b3e-a2a6-84db3f14ff03-kube-api-access-zfrs8\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.494524 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.494274    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-oauth-serving-cert\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.494524 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.494322    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-config\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.494524 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.494352    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-oauth-config\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.494524 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.494413    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-serving-cert\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.495075 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.495051    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-oauth-serving-cert\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.495164 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.495055    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-service-ca\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.495164 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.495150    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-config\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.497034 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.497014    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-oauth-config\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.497140 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.497123    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-serving-cert\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.504029 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.504009    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfrs8\" (UniqueName: \"kubernetes.io/projected/02e480bf-635e-4b3e-a2a6-84db3f14ff03-kube-api-access-zfrs8\") pod \"console-796bdb75ff-s74mp\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") " pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.645952 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.645866    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:48.967909 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:48.967885    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-796bdb75ff-s74mp"]
Jul 06 20:50:48.972898 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:50:48.972870    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e480bf_635e_4b3e_a2a6_84db3f14ff03.slice/crio-f2100cca81b74a4f069743bbb9a7e641931643946d8cfe4eaa3860c615d2b882 WatchSource:0}: Error finding container f2100cca81b74a4f069743bbb9a7e641931643946d8cfe4eaa3860c615d2b882: Status 404 returned error can't find the container with id f2100cca81b74a4f069743bbb9a7e641931643946d8cfe4eaa3860c615d2b882
Jul 06 20:50:49.663823 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:49.663778    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerStarted","Data":"70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b"}
Jul 06 20:50:49.664009 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:49.663902    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerStarted","Data":"7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11"}
Jul 06 20:50:49.664009 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:49.663939    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerStarted","Data":"0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19"}
Jul 06 20:50:49.664009 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:49.663972    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerStarted","Data":"d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1"}
Jul 06 20:50:49.664009 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:49.664000    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerStarted","Data":"826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1"}
Jul 06 20:50:49.665091 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:49.665058    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796bdb75ff-s74mp" event={"ID":"02e480bf-635e-4b3e-a2a6-84db3f14ff03","Type":"ContainerStarted","Data":"f2100cca81b74a4f069743bbb9a7e641931643946d8cfe4eaa3860c615d2b882"}
Jul 06 20:50:50.672886 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:50.672852    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerStarted","Data":"2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea"}
Jul 06 20:50:50.724815 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:50.724752    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.683989605 podStartE2EDuration="5.724730336s" totalImagesPullingTime="3.017715532s" totalInitContainerRuntime="23.025199ms" isStatefulPod=false podCreationTimestamp="2026-07-06 20:50:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:50.723952413 +0000 UTC m=+129.190193696" watchObservedRunningTime="2026-07-06 20:50:50.724730336 +0000 UTC m=+129.190971592"
Jul 06 20:50:51.726097 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:51.725945    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:50:51.728571 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:51.728484    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/997ba244-1db8-4c94-bb4e-eda91480d511-metrics-certs\") pod \"network-metrics-daemon-w4sfv\" (UID: \"997ba244-1db8-4c94-bb4e-eda91480d511\") " pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:50:51.816999 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:51.816967    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mf9wp\""
Jul 06 20:50:51.824865 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:51.824842    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4sfv"
Jul 06 20:50:51.954710 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:51.954679    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w4sfv"]
Jul 06 20:50:51.958257 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:50:51.958215    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod997ba244_1db8_4c94_bb4e_eda91480d511.slice/crio-7b4357a47aea4a3818f730459e1fae62795a8538ce79d0d7122989f991857264 WatchSource:0}: Error finding container 7b4357a47aea4a3818f730459e1fae62795a8538ce79d0d7122989f991857264: Status 404 returned error can't find the container with id 7b4357a47aea4a3818f730459e1fae62795a8538ce79d0d7122989f991857264
Jul 06 20:50:52.682793 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:52.682754    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796bdb75ff-s74mp" event={"ID":"02e480bf-635e-4b3e-a2a6-84db3f14ff03","Type":"ContainerStarted","Data":"4db0326e29cd763cc026c411e0dc5532e224426647e7d6c2a698d52cd3f90221"}
Jul 06 20:50:52.684098 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:52.684069    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w4sfv" event={"ID":"997ba244-1db8-4c94-bb4e-eda91480d511","Type":"ContainerStarted","Data":"7b4357a47aea4a3818f730459e1fae62795a8538ce79d0d7122989f991857264"}
Jul 06 20:50:52.703867 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:52.703807    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-796bdb75ff-s74mp" podStartSLOduration=1.847136728 podStartE2EDuration="4.703791446s" totalImagesPullingTime="2.856654718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:52.702773788 +0000 UTC m=+131.169015045" watchObservedRunningTime="2026-07-06 20:50:52.703791446 +0000 UTC m=+131.170032700"
Jul 06 20:50:53.689275 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:53.689238    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w4sfv" event={"ID":"997ba244-1db8-4c94-bb4e-eda91480d511","Type":"ContainerStarted","Data":"f4a1c5e90726018c48a57a5284ec5f21f46d91788926532471b533555a8d66e2"}
Jul 06 20:50:53.689711 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:53.689295    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w4sfv" event={"ID":"997ba244-1db8-4c94-bb4e-eda91480d511","Type":"ContainerStarted","Data":"acecf43a9c8deaa569693fc8ba77b1beff057cf65d36e9f7bbfb6936ea54fe9d"}
Jul 06 20:50:53.707243 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:53.707193    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w4sfv" podStartSLOduration=130.818972247 podStartE2EDuration="2m11.707173444s" totalImagesPullingTime="888.201197ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:53.705696531 +0000 UTC m=+132.171937784" watchObservedRunningTime="2026-07-06 20:50:53.707173444 +0000 UTC m=+132.173414699"
Jul 06 20:50:54.586307 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:54.586270    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68d4f94c5d-xzkmd"]
Jul 06 20:50:56.735898 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.735861    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-777556858d-l9lns"]
Jul 06 20:50:56.738050 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.738033    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.750698 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.750675    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 06 20:50:56.759982 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.759949    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-777556858d-l9lns"]
Jul 06 20:50:56.864842 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.864801    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw4b8\" (UniqueName: \"kubernetes.io/projected/74971c84-5a01-4329-bb2d-35408f3136a8-kube-api-access-gw4b8\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.865050 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.864857    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74971c84-5a01-4329-bb2d-35408f3136a8-console-serving-cert\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.865050 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.864934    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-service-ca\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.865050 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.864976    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-oauth-serving-cert\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.865050 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.865016    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74971c84-5a01-4329-bb2d-35408f3136a8-console-oauth-config\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.865212 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.865054    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-trusted-ca-bundle\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.865212 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.865090    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-console-config\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.966030 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.965992    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw4b8\" (UniqueName: \"kubernetes.io/projected/74971c84-5a01-4329-bb2d-35408f3136a8-kube-api-access-gw4b8\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.966185 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.966043    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74971c84-5a01-4329-bb2d-35408f3136a8-console-serving-cert\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.966185 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.966095    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-service-ca\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.966185 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.966130    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-oauth-serving-cert\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.966185 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.966149    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74971c84-5a01-4329-bb2d-35408f3136a8-console-oauth-config\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.966390 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.966291    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-trusted-ca-bundle\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.966390 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.966355    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-console-config\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.966892 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.966869    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-service-ca\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.966998 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.966976    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-oauth-serving-cert\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.967062 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.966996    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-console-config\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.967393 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.967361    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-trusted-ca-bundle\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.968613 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.968593    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74971c84-5a01-4329-bb2d-35408f3136a8-console-oauth-config\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.968847 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.968829    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74971c84-5a01-4329-bb2d-35408f3136a8-console-serving-cert\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:56.975728 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:56.975705    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw4b8\" (UniqueName: \"kubernetes.io/projected/74971c84-5a01-4329-bb2d-35408f3136a8-kube-api-access-gw4b8\") pod \"console-777556858d-l9lns\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") " pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:57.046616 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:57.046523    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:50:57.174365 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:57.174340    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-777556858d-l9lns"]
Jul 06 20:50:57.176736 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:50:57.176706    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74971c84_5a01_4329_bb2d_35408f3136a8.slice/crio-0c0946e9b87a71646420a3a8acc7b90633858d48d4a85daf3699f7685078f5f0 WatchSource:0}: Error finding container 0c0946e9b87a71646420a3a8acc7b90633858d48d4a85daf3699f7685078f5f0: Status 404 returned error can't find the container with id 0c0946e9b87a71646420a3a8acc7b90633858d48d4a85daf3699f7685078f5f0
Jul 06 20:50:57.706006 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:57.705970    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777556858d-l9lns" event={"ID":"74971c84-5a01-4329-bb2d-35408f3136a8","Type":"ContainerStarted","Data":"86770cb01e78f61321d5542780a778f49466246f3010bbbe9951be3f4c811ce2"}
Jul 06 20:50:57.706156 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:57.706021    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777556858d-l9lns" event={"ID":"74971c84-5a01-4329-bb2d-35408f3136a8","Type":"ContainerStarted","Data":"0c0946e9b87a71646420a3a8acc7b90633858d48d4a85daf3699f7685078f5f0"}
Jul 06 20:50:57.730991 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:57.730915    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-777556858d-l9lns" podStartSLOduration=1.730900713 podStartE2EDuration="1.730900713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:50:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:57.729510911 +0000 UTC m=+136.195752164" watchObservedRunningTime="2026-07-06 20:50:57.730900713 +0000 UTC m=+136.197141967"
Jul 06 20:50:58.647014 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:58.646977    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:58.647384 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:58.647042    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:58.651854 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:58.651833    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:50:58.714229 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:50:58.714189    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:51:07.047515 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:07.047481    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:51:07.047973 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:07.047562    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:51:07.052104 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:07.052086    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:51:07.746355 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:07.746324    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:51:07.812858 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:07.812827    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-796bdb75ff-s74mp"]
Jul 06 20:51:19.606659 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.606595    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd" podUID="1cd6b87c-70df-459e-9f27-d458d34d1bed" containerName="registry" containerID="cri-o://c95703ad99fd1b58aadad28f21e4157d5c376b84acbb8ca509a6419fa255785a" gracePeriod=30
Jul 06 20:51:19.787612 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.787583    2566 generic.go:356] "Generic (PLEG): container finished" podID="1cd6b87c-70df-459e-9f27-d458d34d1bed" containerID="c95703ad99fd1b58aadad28f21e4157d5c376b84acbb8ca509a6419fa255785a" exitCode=0
Jul 06 20:51:19.787767 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.787630    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd" event={"ID":"1cd6b87c-70df-459e-9f27-d458d34d1bed","Type":"ContainerDied","Data":"c95703ad99fd1b58aadad28f21e4157d5c376b84acbb8ca509a6419fa255785a"}
Jul 06 20:51:19.857498 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.857418    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:51:19.960444 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.960400    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-bound-sa-token\") pod \"1cd6b87c-70df-459e-9f27-d458d34d1bed\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") "
Jul 06 20:51:19.960685 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.960488    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1cd6b87c-70df-459e-9f27-d458d34d1bed-trusted-ca\") pod \"1cd6b87c-70df-459e-9f27-d458d34d1bed\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") "
Jul 06 20:51:19.960685 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.960516    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls\") pod \"1cd6b87c-70df-459e-9f27-d458d34d1bed\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") "
Jul 06 20:51:19.960685 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.960551    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1cd6b87c-70df-459e-9f27-d458d34d1bed-image-registry-private-configuration\") pod \"1cd6b87c-70df-459e-9f27-d458d34d1bed\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") "
Jul 06 20:51:19.960685 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.960576    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1cd6b87c-70df-459e-9f27-d458d34d1bed-ca-trust-extracted\") pod \"1cd6b87c-70df-459e-9f27-d458d34d1bed\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") "
Jul 06 20:51:19.960904 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.960713    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1cd6b87c-70df-459e-9f27-d458d34d1bed-installation-pull-secrets\") pod \"1cd6b87c-70df-459e-9f27-d458d34d1bed\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") "
Jul 06 20:51:19.960904 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.960755    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-certificates\") pod \"1cd6b87c-70df-459e-9f27-d458d34d1bed\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") "
Jul 06 20:51:19.960904 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.960796    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwczg\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-kube-api-access-fwczg\") pod \"1cd6b87c-70df-459e-9f27-d458d34d1bed\" (UID: \"1cd6b87c-70df-459e-9f27-d458d34d1bed\") "
Jul 06 20:51:19.961058 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.960962    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cd6b87c-70df-459e-9f27-d458d34d1bed-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1cd6b87c-70df-459e-9f27-d458d34d1bed" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:51:19.961117 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.961076    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1cd6b87c-70df-459e-9f27-d458d34d1bed-trusted-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:19.961438 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.961402    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1cd6b87c-70df-459e-9f27-d458d34d1bed" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:51:19.963254 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.963229    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1cd6b87c-70df-459e-9f27-d458d34d1bed" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:51:19.963366 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.963228    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1cd6b87c-70df-459e-9f27-d458d34d1bed" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:51:19.963366 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.963315    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-kube-api-access-fwczg" (OuterVolumeSpecName: "kube-api-access-fwczg") pod "1cd6b87c-70df-459e-9f27-d458d34d1bed" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed"). InnerVolumeSpecName "kube-api-access-fwczg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:51:19.963528 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.963501    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cd6b87c-70df-459e-9f27-d458d34d1bed-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1cd6b87c-70df-459e-9f27-d458d34d1bed" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:51:19.963596 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.963559    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cd6b87c-70df-459e-9f27-d458d34d1bed-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1cd6b87c-70df-459e-9f27-d458d34d1bed" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:51:19.969182 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:19.969155    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd6b87c-70df-459e-9f27-d458d34d1bed-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1cd6b87c-70df-459e-9f27-d458d34d1bed" (UID: "1cd6b87c-70df-459e-9f27-d458d34d1bed"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:51:20.062001 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:20.061964    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-bound-sa-token\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:20.062001 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:20.061996    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:20.062001 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:20.062006    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1cd6b87c-70df-459e-9f27-d458d34d1bed-image-registry-private-configuration\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:20.062001 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:20.062016    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1cd6b87c-70df-459e-9f27-d458d34d1bed-ca-trust-extracted\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:20.062268 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:20.062025    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1cd6b87c-70df-459e-9f27-d458d34d1bed-installation-pull-secrets\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:20.062268 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:20.062034    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1cd6b87c-70df-459e-9f27-d458d34d1bed-registry-certificates\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:20.062268 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:20.062042    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwczg\" (UniqueName: \"kubernetes.io/projected/1cd6b87c-70df-459e-9f27-d458d34d1bed-kube-api-access-fwczg\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:20.791925 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:20.791897    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd"
Jul 06 20:51:20.791925 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:20.791916    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68d4f94c5d-xzkmd" event={"ID":"1cd6b87c-70df-459e-9f27-d458d34d1bed","Type":"ContainerDied","Data":"8ed6296e79c2a320a1342d1446d5378bda82f9e9eb5b62b8ac04e14336013386"}
Jul 06 20:51:20.792369 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:20.791974    2566 scope.go:117] "RemoveContainer" containerID="c95703ad99fd1b58aadad28f21e4157d5c376b84acbb8ca509a6419fa255785a"
Jul 06 20:51:20.810000 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:20.809972    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68d4f94c5d-xzkmd"]
Jul 06 20:51:20.814138 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:20.814117    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-68d4f94c5d-xzkmd"]
Jul 06 20:51:22.004394 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:22.004358    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cd6b87c-70df-459e-9f27-d458d34d1bed" path="/var/lib/kubelet/pods/1cd6b87c-70df-459e-9f27-d458d34d1bed/volumes"
Jul 06 20:51:32.833562 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:32.833500    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-796bdb75ff-s74mp" podUID="02e480bf-635e-4b3e-a2a6-84db3f14ff03" containerName="console" containerID="cri-o://4db0326e29cd763cc026c411e0dc5532e224426647e7d6c2a698d52cd3f90221" gracePeriod=15
Jul 06 20:51:33.108379 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.108356    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:51:33.182914 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.182877    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-oauth-config\") pod \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") "
Jul 06 20:51:33.182914 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.182917    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfrs8\" (UniqueName: \"kubernetes.io/projected/02e480bf-635e-4b3e-a2a6-84db3f14ff03-kube-api-access-zfrs8\") pod \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") "
Jul 06 20:51:33.183128 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.183006    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-config\") pod \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") "
Jul 06 20:51:33.183128 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.183036    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-oauth-serving-cert\") pod \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") "
Jul 06 20:51:33.183128 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.183060    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-serving-cert\") pod \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") "
Jul 06 20:51:33.183128 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.183099    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-service-ca\") pod \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\" (UID: \"02e480bf-635e-4b3e-a2a6-84db3f14ff03\") "
Jul 06 20:51:33.183502 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.183469    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-config" (OuterVolumeSpecName: "console-config") pod "02e480bf-635e-4b3e-a2a6-84db3f14ff03" (UID: "02e480bf-635e-4b3e-a2a6-84db3f14ff03"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:51:33.183633 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.183524    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "02e480bf-635e-4b3e-a2a6-84db3f14ff03" (UID: "02e480bf-635e-4b3e-a2a6-84db3f14ff03"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:51:33.183633 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.183573    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-service-ca" (OuterVolumeSpecName: "service-ca") pod "02e480bf-635e-4b3e-a2a6-84db3f14ff03" (UID: "02e480bf-635e-4b3e-a2a6-84db3f14ff03"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:51:33.185311 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.185285    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02e480bf-635e-4b3e-a2a6-84db3f14ff03-kube-api-access-zfrs8" (OuterVolumeSpecName: "kube-api-access-zfrs8") pod "02e480bf-635e-4b3e-a2a6-84db3f14ff03" (UID: "02e480bf-635e-4b3e-a2a6-84db3f14ff03"). InnerVolumeSpecName "kube-api-access-zfrs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:51:33.185311 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.185298    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "02e480bf-635e-4b3e-a2a6-84db3f14ff03" (UID: "02e480bf-635e-4b3e-a2a6-84db3f14ff03"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:51:33.185449 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.185375    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "02e480bf-635e-4b3e-a2a6-84db3f14ff03" (UID: "02e480bf-635e-4b3e-a2a6-84db3f14ff03"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:51:33.284557 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.284519    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfrs8\" (UniqueName: \"kubernetes.io/projected/02e480bf-635e-4b3e-a2a6-84db3f14ff03-kube-api-access-zfrs8\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:33.284557 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.284553    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:33.284557 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.284563    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-oauth-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:33.284795 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.284571    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:33.284795 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.284580    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02e480bf-635e-4b3e-a2a6-84db3f14ff03-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:33.284795 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.284588    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02e480bf-635e-4b3e-a2a6-84db3f14ff03-console-oauth-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:51:33.841291 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.841254    2566 generic.go:356] "Generic (PLEG): container finished" podID="02e480bf-635e-4b3e-a2a6-84db3f14ff03" containerID="4db0326e29cd763cc026c411e0dc5532e224426647e7d6c2a698d52cd3f90221" exitCode=0
Jul 06 20:51:33.841291 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.841287    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796bdb75ff-s74mp"
Jul 06 20:51:33.841847 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.841324    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796bdb75ff-s74mp" event={"ID":"02e480bf-635e-4b3e-a2a6-84db3f14ff03","Type":"ContainerDied","Data":"4db0326e29cd763cc026c411e0dc5532e224426647e7d6c2a698d52cd3f90221"}
Jul 06 20:51:33.841847 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.841407    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796bdb75ff-s74mp" event={"ID":"02e480bf-635e-4b3e-a2a6-84db3f14ff03","Type":"ContainerDied","Data":"f2100cca81b74a4f069743bbb9a7e641931643946d8cfe4eaa3860c615d2b882"}
Jul 06 20:51:33.841847 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.841429    2566 scope.go:117] "RemoveContainer" containerID="4db0326e29cd763cc026c411e0dc5532e224426647e7d6c2a698d52cd3f90221"
Jul 06 20:51:33.850397 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.850377    2566 scope.go:117] "RemoveContainer" containerID="4db0326e29cd763cc026c411e0dc5532e224426647e7d6c2a698d52cd3f90221"
Jul 06 20:51:33.850662 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:51:33.850643    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4db0326e29cd763cc026c411e0dc5532e224426647e7d6c2a698d52cd3f90221\": container with ID starting with 4db0326e29cd763cc026c411e0dc5532e224426647e7d6c2a698d52cd3f90221 not found: ID does not exist" containerID="4db0326e29cd763cc026c411e0dc5532e224426647e7d6c2a698d52cd3f90221"
Jul 06 20:51:33.850716 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.850671    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4db0326e29cd763cc026c411e0dc5532e224426647e7d6c2a698d52cd3f90221"} err="failed to get container status \"4db0326e29cd763cc026c411e0dc5532e224426647e7d6c2a698d52cd3f90221\": rpc error: code = NotFound desc = could not find container \"4db0326e29cd763cc026c411e0dc5532e224426647e7d6c2a698d52cd3f90221\": container with ID starting with 4db0326e29cd763cc026c411e0dc5532e224426647e7d6c2a698d52cd3f90221 not found: ID does not exist"
Jul 06 20:51:33.864138 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.864112    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-796bdb75ff-s74mp"]
Jul 06 20:51:33.867725 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:33.867706    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-796bdb75ff-s74mp"]
Jul 06 20:51:34.002798 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:51:34.002765    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02e480bf-635e-4b3e-a2a6-84db3f14ff03" path="/var/lib/kubelet/pods/02e480bf-635e-4b3e-a2a6-84db3f14ff03/volumes"
Jul 06 20:52:04.978002 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:04.977971    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 20:52:04.978552 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:04.978265    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="alertmanager" containerID="cri-o://826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1" gracePeriod=120
Jul 06 20:52:04.978552 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:04.978329    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="kube-rbac-proxy-metric" containerID="cri-o://70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b" gracePeriod=120
Jul 06 20:52:04.978552 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:04.978347    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="prom-label-proxy" containerID="cri-o://2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea" gracePeriod=120
Jul 06 20:52:04.978552 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:04.978373    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="kube-rbac-proxy-web" containerID="cri-o://0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19" gracePeriod=120
Jul 06 20:52:04.978552 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:04.978381    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="kube-rbac-proxy" containerID="cri-o://7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11" gracePeriod=120
Jul 06 20:52:04.978552 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:04.978420    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="config-reloader" containerID="cri-o://d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1" gracePeriod=120
Jul 06 20:52:05.956532 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:05.956497    2566 generic.go:356] "Generic (PLEG): container finished" podID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerID="2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea" exitCode=0
Jul 06 20:52:05.956532 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:05.956526    2566 generic.go:356] "Generic (PLEG): container finished" podID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerID="7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11" exitCode=0
Jul 06 20:52:05.956532 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:05.956532    2566 generic.go:356] "Generic (PLEG): container finished" podID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerID="d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1" exitCode=0
Jul 06 20:52:05.956532 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:05.956538    2566 generic.go:356] "Generic (PLEG): container finished" podID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerID="826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1" exitCode=0
Jul 06 20:52:05.956802 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:05.956569    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerDied","Data":"2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea"}
Jul 06 20:52:05.956802 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:05.956635    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerDied","Data":"7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11"}
Jul 06 20:52:05.956802 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:05.956661    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerDied","Data":"d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1"}
Jul 06 20:52:05.956802 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:05.956696    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerDied","Data":"826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1"}
Jul 06 20:52:06.224268 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.224245    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:06.336694 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.336650    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1932c276-cb9c-428c-aa9d-b804a8bd4840\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") "
Jul 06 20:52:06.336694 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.336700    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1932c276-cb9c-428c-aa9d-b804a8bd4840-tls-assets\") pod \"1932c276-cb9c-428c-aa9d-b804a8bd4840\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") "
Jul 06 20:52:06.336938 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.336722    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1932c276-cb9c-428c-aa9d-b804a8bd4840-config-out\") pod \"1932c276-cb9c-428c-aa9d-b804a8bd4840\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") "
Jul 06 20:52:06.336938 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.336739    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-cluster-tls-config\") pod \"1932c276-cb9c-428c-aa9d-b804a8bd4840\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") "
Jul 06 20:52:06.336938 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.336771    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy-web\") pod \"1932c276-cb9c-428c-aa9d-b804a8bd4840\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") "
Jul 06 20:52:06.336938 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.336798    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-config-volume\") pod \"1932c276-cb9c-428c-aa9d-b804a8bd4840\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") "
Jul 06 20:52:06.336938 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.336819    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-web-config\") pod \"1932c276-cb9c-428c-aa9d-b804a8bd4840\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") "
Jul 06 20:52:06.338429 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.337260    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy\") pod \"1932c276-cb9c-428c-aa9d-b804a8bd4840\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") "
Jul 06 20:52:06.338429 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.337341    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1932c276-cb9c-428c-aa9d-b804a8bd4840-alertmanager-trusted-ca-bundle\") pod \"1932c276-cb9c-428c-aa9d-b804a8bd4840\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") "
Jul 06 20:52:06.338429 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.337383    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1932c276-cb9c-428c-aa9d-b804a8bd4840-alertmanager-main-db\") pod \"1932c276-cb9c-428c-aa9d-b804a8bd4840\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") "
Jul 06 20:52:06.338429 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.337408    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1932c276-cb9c-428c-aa9d-b804a8bd4840-metrics-client-ca\") pod \"1932c276-cb9c-428c-aa9d-b804a8bd4840\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") "
Jul 06 20:52:06.338429 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.337433    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg8ft\" (UniqueName: \"kubernetes.io/projected/1932c276-cb9c-428c-aa9d-b804a8bd4840-kube-api-access-sg8ft\") pod \"1932c276-cb9c-428c-aa9d-b804a8bd4840\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") "
Jul 06 20:52:06.338429 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.337522    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-main-tls\") pod \"1932c276-cb9c-428c-aa9d-b804a8bd4840\" (UID: \"1932c276-cb9c-428c-aa9d-b804a8bd4840\") "
Jul 06 20:52:06.338429 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.338307    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1932c276-cb9c-428c-aa9d-b804a8bd4840-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1932c276-cb9c-428c-aa9d-b804a8bd4840" (UID: "1932c276-cb9c-428c-aa9d-b804a8bd4840"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:52:06.338429 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.338371    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1932c276-cb9c-428c-aa9d-b804a8bd4840-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1932c276-cb9c-428c-aa9d-b804a8bd4840" (UID: "1932c276-cb9c-428c-aa9d-b804a8bd4840"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:52:06.339081 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.338690    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1932c276-cb9c-428c-aa9d-b804a8bd4840-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1932c276-cb9c-428c-aa9d-b804a8bd4840" (UID: "1932c276-cb9c-428c-aa9d-b804a8bd4840"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:52:06.339769 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.339734    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1932c276-cb9c-428c-aa9d-b804a8bd4840" (UID: "1932c276-cb9c-428c-aa9d-b804a8bd4840"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:52:06.339977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.339939    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1932c276-cb9c-428c-aa9d-b804a8bd4840" (UID: "1932c276-cb9c-428c-aa9d-b804a8bd4840"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:52:06.340191 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.340164    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1932c276-cb9c-428c-aa9d-b804a8bd4840-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1932c276-cb9c-428c-aa9d-b804a8bd4840" (UID: "1932c276-cb9c-428c-aa9d-b804a8bd4840"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:52:06.340533 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.340507    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-config-volume" (OuterVolumeSpecName: "config-volume") pod "1932c276-cb9c-428c-aa9d-b804a8bd4840" (UID: "1932c276-cb9c-428c-aa9d-b804a8bd4840"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:52:06.340750 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.340723    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1932c276-cb9c-428c-aa9d-b804a8bd4840-config-out" (OuterVolumeSpecName: "config-out") pod "1932c276-cb9c-428c-aa9d-b804a8bd4840" (UID: "1932c276-cb9c-428c-aa9d-b804a8bd4840"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:52:06.341125 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.341099    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1932c276-cb9c-428c-aa9d-b804a8bd4840" (UID: "1932c276-cb9c-428c-aa9d-b804a8bd4840"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:52:06.341542 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.341516    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1932c276-cb9c-428c-aa9d-b804a8bd4840" (UID: "1932c276-cb9c-428c-aa9d-b804a8bd4840"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:52:06.341893 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.341863    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1932c276-cb9c-428c-aa9d-b804a8bd4840-kube-api-access-sg8ft" (OuterVolumeSpecName: "kube-api-access-sg8ft") pod "1932c276-cb9c-428c-aa9d-b804a8bd4840" (UID: "1932c276-cb9c-428c-aa9d-b804a8bd4840"). InnerVolumeSpecName "kube-api-access-sg8ft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:52:06.344374 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.344354    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1932c276-cb9c-428c-aa9d-b804a8bd4840" (UID: "1932c276-cb9c-428c-aa9d-b804a8bd4840"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:52:06.350888 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.350866    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-web-config" (OuterVolumeSpecName: "web-config") pod "1932c276-cb9c-428c-aa9d-b804a8bd4840" (UID: "1932c276-cb9c-428c-aa9d-b804a8bd4840"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:52:06.438709 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.438678    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1932c276-cb9c-428c-aa9d-b804a8bd4840-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:06.438709 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.438704    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1932c276-cb9c-428c-aa9d-b804a8bd4840-alertmanager-main-db\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:06.438709 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.438714    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1932c276-cb9c-428c-aa9d-b804a8bd4840-metrics-client-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:06.438931 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.438723    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg8ft\" (UniqueName: \"kubernetes.io/projected/1932c276-cb9c-428c-aa9d-b804a8bd4840-kube-api-access-sg8ft\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:06.438931 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.438733    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-main-tls\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:06.438931 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.438744    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:06.438931 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.438753    2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1932c276-cb9c-428c-aa9d-b804a8bd4840-tls-assets\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:06.438931 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.438761    2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1932c276-cb9c-428c-aa9d-b804a8bd4840-config-out\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:06.438931 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.438769    2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-cluster-tls-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:06.438931 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.438777    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:06.438931 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.438788    2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-config-volume\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:06.438931 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.438796    2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-web-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:06.438931 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.438804    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1932c276-cb9c-428c-aa9d-b804a8bd4840-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:06.962696 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.962660    2566 generic.go:356] "Generic (PLEG): container finished" podID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerID="70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b" exitCode=0
Jul 06 20:52:06.962696 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.962691    2566 generic.go:356] "Generic (PLEG): container finished" podID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerID="0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19" exitCode=0
Jul 06 20:52:06.962895 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.962730    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerDied","Data":"70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b"}
Jul 06 20:52:06.962895 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.962747    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:06.962895 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.962801    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerDied","Data":"0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19"}
Jul 06 20:52:06.962895 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.962829    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1932c276-cb9c-428c-aa9d-b804a8bd4840","Type":"ContainerDied","Data":"c6976647c6227a45703c82b4fa9659f95249c46a288d8cc6ec4885e3faaebe95"}
Jul 06 20:52:06.962895 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.962861    2566 scope.go:117] "RemoveContainer" containerID="2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea"
Jul 06 20:52:06.970740 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.970716    2566 scope.go:117] "RemoveContainer" containerID="70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b"
Jul 06 20:52:06.981295 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.981274    2566 scope.go:117] "RemoveContainer" containerID="7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11"
Jul 06 20:52:06.988637 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.988619    2566 scope.go:117] "RemoveContainer" containerID="0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19"
Jul 06 20:52:06.995172 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:06.995152    2566 scope.go:117] "RemoveContainer" containerID="d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1"
Jul 06 20:52:07.002193 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.002174    2566 scope.go:117] "RemoveContainer" containerID="826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1"
Jul 06 20:52:07.010196 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.010174    2566 scope.go:117] "RemoveContainer" containerID="0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d"
Jul 06 20:52:07.011203 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.011179    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 20:52:07.017467 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.017430    2566 scope.go:117] "RemoveContainer" containerID="2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea"
Jul 06 20:52:07.017712 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:52:07.017693    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea\": container with ID starting with 2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea not found: ID does not exist" containerID="2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea"
Jul 06 20:52:07.017771 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.017722    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea"} err="failed to get container status \"2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea\": rpc error: code = NotFound desc = could not find container \"2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea\": container with ID starting with 2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea not found: ID does not exist"
Jul 06 20:52:07.017771 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.017741    2566 scope.go:117] "RemoveContainer" containerID="70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b"
Jul 06 20:52:07.017971 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:52:07.017951    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b\": container with ID starting with 70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b not found: ID does not exist" containerID="70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b"
Jul 06 20:52:07.018021 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.017978    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b"} err="failed to get container status \"70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b\": rpc error: code = NotFound desc = could not find container \"70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b\": container with ID starting with 70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b not found: ID does not exist"
Jul 06 20:52:07.018021 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.017997    2566 scope.go:117] "RemoveContainer" containerID="7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11"
Jul 06 20:52:07.018242 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:52:07.018227    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11\": container with ID starting with 7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11 not found: ID does not exist" containerID="7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11"
Jul 06 20:52:07.018297 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.018248    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11"} err="failed to get container status \"7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11\": rpc error: code = NotFound desc = could not find container \"7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11\": container with ID starting with 7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11 not found: ID does not exist"
Jul 06 20:52:07.018297 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.018265    2566 scope.go:117] "RemoveContainer" containerID="0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19"
Jul 06 20:52:07.018560 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:52:07.018537    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19\": container with ID starting with 0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19 not found: ID does not exist" containerID="0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19"
Jul 06 20:52:07.018624 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.018568    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19"} err="failed to get container status \"0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19\": rpc error: code = NotFound desc = could not find container \"0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19\": container with ID starting with 0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19 not found: ID does not exist"
Jul 06 20:52:07.018624 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.018590    2566 scope.go:117] "RemoveContainer" containerID="d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1"
Jul 06 20:52:07.019049 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:52:07.019026    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1\": container with ID starting with d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1 not found: ID does not exist" containerID="d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1"
Jul 06 20:52:07.019129 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.019056    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1"} err="failed to get container status \"d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1\": rpc error: code = NotFound desc = could not find container \"d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1\": container with ID starting with d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1 not found: ID does not exist"
Jul 06 20:52:07.019129 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.019079    2566 scope.go:117] "RemoveContainer" containerID="826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1"
Jul 06 20:52:07.019504 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:52:07.019478    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1\": container with ID starting with 826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1 not found: ID does not exist" containerID="826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1"
Jul 06 20:52:07.019594 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.019509    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1"} err="failed to get container status \"826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1\": rpc error: code = NotFound desc = could not find container \"826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1\": container with ID starting with 826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1 not found: ID does not exist"
Jul 06 20:52:07.019594 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.019527    2566 scope.go:117] "RemoveContainer" containerID="0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d"
Jul 06 20:52:07.019782 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:52:07.019768    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d\": container with ID starting with 0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d not found: ID does not exist" containerID="0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d"
Jul 06 20:52:07.019828 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.019784    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d"} err="failed to get container status \"0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d\": rpc error: code = NotFound desc = could not find container \"0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d\": container with ID starting with 0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d not found: ID does not exist"
Jul 06 20:52:07.019828 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.019797    2566 scope.go:117] "RemoveContainer" containerID="2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea"
Jul 06 20:52:07.020158 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.020121    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea"} err="failed to get container status \"2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea\": rpc error: code = NotFound desc = could not find container \"2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea\": container with ID starting with 2ad6eeaf5c46d7c4f7d9d3d3d5f597a536e730515012a15021989b2e6485d8ea not found: ID does not exist"
Jul 06 20:52:07.020158 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.020142    2566 scope.go:117] "RemoveContainer" containerID="70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b"
Jul 06 20:52:07.020374 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.020353    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b"} err="failed to get container status \"70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b\": rpc error: code = NotFound desc = could not find container \"70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b\": container with ID starting with 70ff059e104dc925a158a9b68e4c86f236089c15a4c9d199bee0fc7cfa7d6a5b not found: ID does not exist"
Jul 06 20:52:07.020477 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.020376    2566 scope.go:117] "RemoveContainer" containerID="7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11"
Jul 06 20:52:07.020628 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.020609    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11"} err="failed to get container status \"7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11\": rpc error: code = NotFound desc = could not find container \"7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11\": container with ID starting with 7dc6ea89eec04bb89893da60e8ae07f3307bb9fdcf2d8c014de125bd77277e11 not found: ID does not exist"
Jul 06 20:52:07.020681 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.020629    2566 scope.go:117] "RemoveContainer" containerID="0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19"
Jul 06 20:52:07.020845 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.020828    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19"} err="failed to get container status \"0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19\": rpc error: code = NotFound desc = could not find container \"0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19\": container with ID starting with 0bcf019b2e122793648778b9b1f7720db0ea219d7ca48c3b38a78fedfc796a19 not found: ID does not exist"
Jul 06 20:52:07.020906 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.020845    2566 scope.go:117] "RemoveContainer" containerID="d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1"
Jul 06 20:52:07.021070 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.021052    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1"} err="failed to get container status \"d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1\": rpc error: code = NotFound desc = could not find container \"d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1\": container with ID starting with d3ef4015402937acc5031962d836cbfeee55c1b27e6c9b8fe0af32def70e56f1 not found: ID does not exist"
Jul 06 20:52:07.021116 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.021071    2566 scope.go:117] "RemoveContainer" containerID="826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1"
Jul 06 20:52:07.021363 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.021344    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1"} err="failed to get container status \"826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1\": rpc error: code = NotFound desc = could not find container \"826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1\": container with ID starting with 826746ebccbb0b9119826ae9632522e22fd794591f5cbdae897515d0078957a1 not found: ID does not exist"
Jul 06 20:52:07.021423 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.021364    2566 scope.go:117] "RemoveContainer" containerID="0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d"
Jul 06 20:52:07.021618 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.021600    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d"} err="failed to get container status \"0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d\": rpc error: code = NotFound desc = could not find container \"0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d\": container with ID starting with 0356ac475b2befadcc14c4ffbfbbe8176254ec643df7f177d804b50ecfb1d32d not found: ID does not exist"
Jul 06 20:52:07.021705 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.021689    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 20:52:07.076324 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076289    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 20:52:07.076561 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076549    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cd6b87c-70df-459e-9f27-d458d34d1bed" containerName="registry"
Jul 06 20:52:07.076561 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076562    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd6b87c-70df-459e-9f27-d458d34d1bed" containerName="registry"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076573    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="init-config-reloader"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076579    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="init-config-reloader"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076585    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02e480bf-635e-4b3e-a2a6-84db3f14ff03" containerName="console"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076591    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="02e480bf-635e-4b3e-a2a6-84db3f14ff03" containerName="console"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076598    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="alertmanager"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076603    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="alertmanager"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076608    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="kube-rbac-proxy-metric"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076613    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="kube-rbac-proxy-metric"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076624    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="kube-rbac-proxy-web"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076629    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="kube-rbac-proxy-web"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076635    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="kube-rbac-proxy"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076640    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="kube-rbac-proxy"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076646    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="config-reloader"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076650    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="config-reloader"
Jul 06 20:52:07.076652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076657    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="prom-label-proxy"
Jul 06 20:52:07.077062 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076662    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="prom-label-proxy"
Jul 06 20:52:07.077062 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076707    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="kube-rbac-proxy-web"
Jul 06 20:52:07.077062 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076715    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="config-reloader"
Jul 06 20:52:07.077062 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076721    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="alertmanager"
Jul 06 20:52:07.077062 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076727    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="kube-rbac-proxy-metric"
Jul 06 20:52:07.077062 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076732    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="02e480bf-635e-4b3e-a2a6-84db3f14ff03" containerName="console"
Jul 06 20:52:07.077062 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076737    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="kube-rbac-proxy"
Jul 06 20:52:07.077062 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076743    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cd6b87c-70df-459e-9f27-d458d34d1bed" containerName="registry"
Jul 06 20:52:07.077062 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.076749    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" containerName="prom-label-proxy"
Jul 06 20:52:07.080594 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.080574    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.083503 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.083472    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 06 20:52:07.083805 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.083787    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 06 20:52:07.084109 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.084094    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 06 20:52:07.084140 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.084095    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 06 20:52:07.084140 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.084098    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rj2vq\""
Jul 06 20:52:07.084767 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.084747    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 06 20:52:07.084850 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.084768    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 06 20:52:07.084850 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.084792    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 06 20:52:07.084850 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.084757    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 06 20:52:07.090676 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.090654    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 06 20:52:07.104060 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.104034    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 20:52:07.245835 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.245805    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.246244 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.245850    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-config-volume\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.246244 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.245881    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-config-out\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.246244 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.245940    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.246244 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.246000    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmqft\" (UniqueName: \"kubernetes.io/projected/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-kube-api-access-dmqft\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.246244 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.246028    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-web-config\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.246244 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.246080    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.246244 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.246109    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.246244 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.246128    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.246244 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.246151    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.246244 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.246198    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.246244 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.246218    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.246244 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.246237    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.347317 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.347267    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqft\" (UniqueName: \"kubernetes.io/projected/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-kube-api-access-dmqft\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.347487 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.347333    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-web-config\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.347487 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.347368    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.347487 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.347392    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.347487 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.347417    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.347487 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.347444    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.347724 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.347494    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.347724 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.347521    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.347724 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.347544    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.347724 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.347659    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.347724 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.347692    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-config-volume\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.347968 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.347727    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-config-out\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.347968 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.347753    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.348136 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.348108    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.348414 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.348388    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.348524 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.348492    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.350717 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.350609    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-web-config\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.350717 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.350645    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.350924 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.350901    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-config-out\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.350963 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.350947    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.351052 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.351023    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-config-volume\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.351088 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.351076    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.351326 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.351306    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.351390 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.351371    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.352817 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.352798    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.356321 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.356300    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmqft\" (UniqueName: \"kubernetes.io/projected/2dea7a7a-5804-44e6-be5e-0855f2a5f3a8-kube-api-access-dmqft\") pod \"alertmanager-main-0\" (UID: \"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.390392 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.390353    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 20:52:07.537201 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.537118    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 20:52:07.540681 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:52:07.540648    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dea7a7a_5804_44e6_be5e_0855f2a5f3a8.slice/crio-6a35e81d2bf10d96e01c6a67339dae65a348bd5769190bf2cecdc0beefcdfeb3 WatchSource:0}: Error finding container 6a35e81d2bf10d96e01c6a67339dae65a348bd5769190bf2cecdc0beefcdfeb3: Status 404 returned error can't find the container with id 6a35e81d2bf10d96e01c6a67339dae65a348bd5769190bf2cecdc0beefcdfeb3
Jul 06 20:52:07.967635 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.967539    2566 generic.go:356] "Generic (PLEG): container finished" podID="2dea7a7a-5804-44e6-be5e-0855f2a5f3a8" containerID="2a46afe53013f8a2194b0e35755a145ad1810255903bba769a4614a9947ab92e" exitCode=0
Jul 06 20:52:07.967795 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.967624    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8","Type":"ContainerDied","Data":"2a46afe53013f8a2194b0e35755a145ad1810255903bba769a4614a9947ab92e"}
Jul 06 20:52:07.967795 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:07.967698    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8","Type":"ContainerStarted","Data":"6a35e81d2bf10d96e01c6a67339dae65a348bd5769190bf2cecdc0beefcdfeb3"}
Jul 06 20:52:08.000380 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:08.000348    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1932c276-cb9c-428c-aa9d-b804a8bd4840" path="/var/lib/kubelet/pods/1932c276-cb9c-428c-aa9d-b804a8bd4840/volumes"
Jul 06 20:52:08.974884 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:08.974854    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8","Type":"ContainerStarted","Data":"00ade7f6de171cbe5af34ff9f52e88c92646cf0bc77e50413f1f253279086c1f"}
Jul 06 20:52:08.975259 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:08.974916    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8","Type":"ContainerStarted","Data":"68099da4dfce5be926f0685651a9d2a62d3756a104f5c3b1363d174fddb3eb17"}
Jul 06 20:52:08.975259 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:08.974943    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8","Type":"ContainerStarted","Data":"8e8fa0881162778383f4c7976f206362d0e5cfcdd771903824867778b4ff2095"}
Jul 06 20:52:08.975259 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:08.974967    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8","Type":"ContainerStarted","Data":"68fcede3239e360888f2afc823aaef5ae41b0e22d286574815a5202181fab2c5"}
Jul 06 20:52:08.975259 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:08.974991    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8","Type":"ContainerStarted","Data":"bee689c287f039b116c57007192c7aecbb7603092db4d2e003ed6a8a68d696f4"}
Jul 06 20:52:08.975259 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:08.975016    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2dea7a7a-5804-44e6-be5e-0855f2a5f3a8","Type":"ContainerStarted","Data":"fa6c5c79a33a848948c3b909ec9659f4936c45332f10a2e58bfd4867511bfb2b"}
Jul 06 20:52:09.008297 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.005768    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6bcbc65985-8ww22"]
Jul 06 20:52:09.010651 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.010604    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.013398 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.013371    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 06 20:52:09.013555 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.013404    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.994922782 podStartE2EDuration="2.013387626s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.464844ms" isStatefulPod=false podCreationTimestamp="2026-07-06 20:52:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:52:09.011095548 +0000 UTC m=+207.477336823" watchObservedRunningTime="2026-07-06 20:52:09.013387626 +0000 UTC m=+207.479628880"
Jul 06 20:52:09.013555 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.013495    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 06 20:52:09.013717 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.013698    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 06 20:52:09.013776 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.013755    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 06 20:52:09.013830 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.013808    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 06 20:52:09.013936 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.013922    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5s7ds\""
Jul 06 20:52:09.019352 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.019329    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 06 20:52:09.025160 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.025105    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6bcbc65985-8ww22"]
Jul 06 20:52:09.161337 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.161294    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/32b2f426-a65f-4ab1-b617-b2a3e50afe26-federate-client-tls\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.161337 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.161341    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32b2f426-a65f-4ab1-b617-b2a3e50afe26-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.161628 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.161383    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32b2f426-a65f-4ab1-b617-b2a3e50afe26-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.161628 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.161492    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/32b2f426-a65f-4ab1-b617-b2a3e50afe26-telemeter-client-tls\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.161628 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.161554    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32b2f426-a65f-4ab1-b617-b2a3e50afe26-metrics-client-ca\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.161628 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.161604    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/32b2f426-a65f-4ab1-b617-b2a3e50afe26-secret-telemeter-client\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.161776 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.161712    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tqpj\" (UniqueName: \"kubernetes.io/projected/32b2f426-a65f-4ab1-b617-b2a3e50afe26-kube-api-access-4tqpj\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.161815 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.161795    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32b2f426-a65f-4ab1-b617-b2a3e50afe26-serving-certs-ca-bundle\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.262734 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.262702    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/32b2f426-a65f-4ab1-b617-b2a3e50afe26-federate-client-tls\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.262905 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.262746    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32b2f426-a65f-4ab1-b617-b2a3e50afe26-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.262905 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.262778    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32b2f426-a65f-4ab1-b617-b2a3e50afe26-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.262905 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.262805    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/32b2f426-a65f-4ab1-b617-b2a3e50afe26-telemeter-client-tls\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.262905 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.262850    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32b2f426-a65f-4ab1-b617-b2a3e50afe26-metrics-client-ca\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.263098 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.262989    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/32b2f426-a65f-4ab1-b617-b2a3e50afe26-secret-telemeter-client\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.263098 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.263020    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tqpj\" (UniqueName: \"kubernetes.io/projected/32b2f426-a65f-4ab1-b617-b2a3e50afe26-kube-api-access-4tqpj\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.263098 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.263060    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32b2f426-a65f-4ab1-b617-b2a3e50afe26-serving-certs-ca-bundle\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.263635 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.263610    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32b2f426-a65f-4ab1-b617-b2a3e50afe26-metrics-client-ca\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.263859 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.263827    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32b2f426-a65f-4ab1-b617-b2a3e50afe26-serving-certs-ca-bundle\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.263967 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.263920    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32b2f426-a65f-4ab1-b617-b2a3e50afe26-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.265587 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.265564    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/32b2f426-a65f-4ab1-b617-b2a3e50afe26-telemeter-client-tls\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.265664 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.265574    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/32b2f426-a65f-4ab1-b617-b2a3e50afe26-secret-telemeter-client\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.265664 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.265633    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32b2f426-a65f-4ab1-b617-b2a3e50afe26-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.265791 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.265769    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/32b2f426-a65f-4ab1-b617-b2a3e50afe26-federate-client-tls\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.272085 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.272056    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tqpj\" (UniqueName: \"kubernetes.io/projected/32b2f426-a65f-4ab1-b617-b2a3e50afe26-kube-api-access-4tqpj\") pod \"telemeter-client-6bcbc65985-8ww22\" (UID: \"32b2f426-a65f-4ab1-b617-b2a3e50afe26\") " pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.321630 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.321593    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22"
Jul 06 20:52:09.462341 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.462310    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6bcbc65985-8ww22"]
Jul 06 20:52:09.465114 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:52:09.465087    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b2f426_a65f_4ab1_b617_b2a3e50afe26.slice/crio-5c47cc48cda94dc4e58b3f428f8335b5aa6b06f82ded9809adbd471bc5a1987f WatchSource:0}: Error finding container 5c47cc48cda94dc4e58b3f428f8335b5aa6b06f82ded9809adbd471bc5a1987f: Status 404 returned error can't find the container with id 5c47cc48cda94dc4e58b3f428f8335b5aa6b06f82ded9809adbd471bc5a1987f
Jul 06 20:52:09.981615 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:09.981573    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22" event={"ID":"32b2f426-a65f-4ab1-b617-b2a3e50afe26","Type":"ContainerStarted","Data":"5c47cc48cda94dc4e58b3f428f8335b5aa6b06f82ded9809adbd471bc5a1987f"}
Jul 06 20:52:10.987992 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:10.987960    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22" event={"ID":"32b2f426-a65f-4ab1-b617-b2a3e50afe26","Type":"ContainerStarted","Data":"956a436d4c41ea53cc08f0c65f974867bf3b7a67bf03d7883a8a4ff91b1afd05"}
Jul 06 20:52:10.988318 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:10.988025    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22" event={"ID":"32b2f426-a65f-4ab1-b617-b2a3e50afe26","Type":"ContainerStarted","Data":"253621508c07e4eb788238b01ca2b9fc954810ce24f32631ffa3f2ad7d388ef3"}
Jul 06 20:52:12.000467 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.000429    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22" event={"ID":"32b2f426-a65f-4ab1-b617-b2a3e50afe26","Type":"ContainerStarted","Data":"7ac0108a2b30a9f6e37f2c4818376cee65462aa11919de0b8bf630c9fb322d6f"}
Jul 06 20:52:12.023082 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.023025    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6bcbc65985-8ww22" podStartSLOduration=2.64574737 podStartE2EDuration="4.023007166s" totalImagesPullingTime="1.377259796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:52:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:52:12.020517311 +0000 UTC m=+210.486758567" watchObservedRunningTime="2026-07-06 20:52:12.023007166 +0000 UTC m=+210.489248419"
Jul 06 20:52:12.741165 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.741128    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b87558bc4-8knlv"]
Jul 06 20:52:12.744276 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.744254    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.758676 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.758652    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b87558bc4-8knlv"]
Jul 06 20:52:12.791844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.791806    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nff2g\" (UniqueName: \"kubernetes.io/projected/5474c763-ce8a-4abf-8191-900b89b307a5-kube-api-access-nff2g\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.791844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.791848    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-service-ca\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.792070 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.791888    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-trusted-ca-bundle\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.792070 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.791916    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-console-config\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.792070 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.791941    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-oauth-serving-cert\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.792070 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.791974    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5474c763-ce8a-4abf-8191-900b89b307a5-console-serving-cert\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.792070 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.792010    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5474c763-ce8a-4abf-8191-900b89b307a5-console-oauth-config\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.892319 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.892275    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5474c763-ce8a-4abf-8191-900b89b307a5-console-oauth-config\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.892497 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.892338    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nff2g\" (UniqueName: \"kubernetes.io/projected/5474c763-ce8a-4abf-8191-900b89b307a5-kube-api-access-nff2g\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.892497 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.892387    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-service-ca\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.892497 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.892412    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-trusted-ca-bundle\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.892497 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.892434    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-console-config\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.892497 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.892479    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-oauth-serving-cert\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.892752 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.892645    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5474c763-ce8a-4abf-8191-900b89b307a5-console-serving-cert\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.893301 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.893264    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-oauth-serving-cert\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.893393 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.893269    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-console-config\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.893393 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.893276    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-service-ca\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.893490 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.893432    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-trusted-ca-bundle\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.895021 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.894996    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5474c763-ce8a-4abf-8191-900b89b307a5-console-oauth-config\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.895091 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.895028    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5474c763-ce8a-4abf-8191-900b89b307a5-console-serving-cert\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:12.905794 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:12.905764    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nff2g\" (UniqueName: \"kubernetes.io/projected/5474c763-ce8a-4abf-8191-900b89b307a5-kube-api-access-nff2g\") pod \"console-6b87558bc4-8knlv\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") " pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:13.054183 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:13.054089    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:13.190932 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:13.190902    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b87558bc4-8knlv"]
Jul 06 20:52:13.194087 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:52:13.194052    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5474c763_ce8a_4abf_8191_900b89b307a5.slice/crio-e98c15f7a80fb1753ad30fa55718d4fca7af3977714b9781d5a78923e1543f57 WatchSource:0}: Error finding container e98c15f7a80fb1753ad30fa55718d4fca7af3977714b9781d5a78923e1543f57: Status 404 returned error can't find the container with id e98c15f7a80fb1753ad30fa55718d4fca7af3977714b9781d5a78923e1543f57
Jul 06 20:52:14.008191 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:14.008157    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b87558bc4-8knlv" event={"ID":"5474c763-ce8a-4abf-8191-900b89b307a5","Type":"ContainerStarted","Data":"3143d303ddedcd650ae5d960d881d15bf6f73b09914c0d8ae129b49bfacef797"}
Jul 06 20:52:14.008356 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:14.008213    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b87558bc4-8knlv" event={"ID":"5474c763-ce8a-4abf-8191-900b89b307a5","Type":"ContainerStarted","Data":"e98c15f7a80fb1753ad30fa55718d4fca7af3977714b9781d5a78923e1543f57"}
Jul 06 20:52:14.034191 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:14.034137    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b87558bc4-8knlv" podStartSLOduration=2.034118417 podStartE2EDuration="2.034118417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:52:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:52:14.029635832 +0000 UTC m=+212.495877085" watchObservedRunningTime="2026-07-06 20:52:14.034118417 +0000 UTC m=+212.500359670"
Jul 06 20:52:23.054443 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:23.054408    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:23.054908 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:23.054510    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:23.059176 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:23.059153    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:24.049802 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:24.049774    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:52:24.111939 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:24.111904    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-777556858d-l9lns"]
Jul 06 20:52:49.133313 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.133271    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-777556858d-l9lns" podUID="74971c84-5a01-4329-bb2d-35408f3136a8" containerName="console" containerID="cri-o://86770cb01e78f61321d5542780a778f49466246f3010bbbe9951be3f4c811ce2" gracePeriod=15
Jul 06 20:52:49.373240 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.373210    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:52:49.492238 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.492202    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-service-ca\") pod \"74971c84-5a01-4329-bb2d-35408f3136a8\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") "
Jul 06 20:52:49.492425 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.492271    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74971c84-5a01-4329-bb2d-35408f3136a8-console-oauth-config\") pod \"74971c84-5a01-4329-bb2d-35408f3136a8\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") "
Jul 06 20:52:49.492425 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.492305    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-console-config\") pod \"74971c84-5a01-4329-bb2d-35408f3136a8\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") "
Jul 06 20:52:49.492425 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.492339    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gw4b8\" (UniqueName: \"kubernetes.io/projected/74971c84-5a01-4329-bb2d-35408f3136a8-kube-api-access-gw4b8\") pod \"74971c84-5a01-4329-bb2d-35408f3136a8\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") "
Jul 06 20:52:49.492631 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.492491    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-trusted-ca-bundle\") pod \"74971c84-5a01-4329-bb2d-35408f3136a8\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") "
Jul 06 20:52:49.492631 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.492558    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74971c84-5a01-4329-bb2d-35408f3136a8-console-serving-cert\") pod \"74971c84-5a01-4329-bb2d-35408f3136a8\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") "
Jul 06 20:52:49.492631 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.492586    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-oauth-serving-cert\") pod \"74971c84-5a01-4329-bb2d-35408f3136a8\" (UID: \"74971c84-5a01-4329-bb2d-35408f3136a8\") "
Jul 06 20:52:49.492806 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.492630    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-service-ca" (OuterVolumeSpecName: "service-ca") pod "74971c84-5a01-4329-bb2d-35408f3136a8" (UID: "74971c84-5a01-4329-bb2d-35408f3136a8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:52:49.492806 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.492746    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-console-config" (OuterVolumeSpecName: "console-config") pod "74971c84-5a01-4329-bb2d-35408f3136a8" (UID: "74971c84-5a01-4329-bb2d-35408f3136a8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:52:49.492897 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.492859    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:49.492897 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.492879    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-console-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:49.492981 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.492886    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "74971c84-5a01-4329-bb2d-35408f3136a8" (UID: "74971c84-5a01-4329-bb2d-35408f3136a8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:52:49.493063 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.493046    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "74971c84-5a01-4329-bb2d-35408f3136a8" (UID: "74971c84-5a01-4329-bb2d-35408f3136a8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:52:49.494503 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.494481    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74971c84-5a01-4329-bb2d-35408f3136a8-kube-api-access-gw4b8" (OuterVolumeSpecName: "kube-api-access-gw4b8") pod "74971c84-5a01-4329-bb2d-35408f3136a8" (UID: "74971c84-5a01-4329-bb2d-35408f3136a8"). InnerVolumeSpecName "kube-api-access-gw4b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:52:49.494959 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.494931    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74971c84-5a01-4329-bb2d-35408f3136a8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "74971c84-5a01-4329-bb2d-35408f3136a8" (UID: "74971c84-5a01-4329-bb2d-35408f3136a8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:52:49.495051 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.494954    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74971c84-5a01-4329-bb2d-35408f3136a8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "74971c84-5a01-4329-bb2d-35408f3136a8" (UID: "74971c84-5a01-4329-bb2d-35408f3136a8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:52:49.593538 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.593496    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74971c84-5a01-4329-bb2d-35408f3136a8-console-oauth-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:49.593538 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.593530    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gw4b8\" (UniqueName: \"kubernetes.io/projected/74971c84-5a01-4329-bb2d-35408f3136a8-kube-api-access-gw4b8\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:49.593538 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.593540    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-trusted-ca-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:49.593538 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.593548    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74971c84-5a01-4329-bb2d-35408f3136a8-console-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:49.593802 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:49.593557    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74971c84-5a01-4329-bb2d-35408f3136a8-oauth-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:52:50.145425 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:50.145386    2566 generic.go:356] "Generic (PLEG): container finished" podID="74971c84-5a01-4329-bb2d-35408f3136a8" containerID="86770cb01e78f61321d5542780a778f49466246f3010bbbe9951be3f4c811ce2" exitCode=0
Jul 06 20:52:50.145866 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:50.145435    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777556858d-l9lns"
Jul 06 20:52:50.145866 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:50.145490    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777556858d-l9lns" event={"ID":"74971c84-5a01-4329-bb2d-35408f3136a8","Type":"ContainerDied","Data":"86770cb01e78f61321d5542780a778f49466246f3010bbbe9951be3f4c811ce2"}
Jul 06 20:52:50.145866 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:50.145549    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777556858d-l9lns" event={"ID":"74971c84-5a01-4329-bb2d-35408f3136a8","Type":"ContainerDied","Data":"0c0946e9b87a71646420a3a8acc7b90633858d48d4a85daf3699f7685078f5f0"}
Jul 06 20:52:50.145866 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:50.145570    2566 scope.go:117] "RemoveContainer" containerID="86770cb01e78f61321d5542780a778f49466246f3010bbbe9951be3f4c811ce2"
Jul 06 20:52:50.153489 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:50.153467    2566 scope.go:117] "RemoveContainer" containerID="86770cb01e78f61321d5542780a778f49466246f3010bbbe9951be3f4c811ce2"
Jul 06 20:52:50.153776 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:52:50.153756    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86770cb01e78f61321d5542780a778f49466246f3010bbbe9951be3f4c811ce2\": container with ID starting with 86770cb01e78f61321d5542780a778f49466246f3010bbbe9951be3f4c811ce2 not found: ID does not exist" containerID="86770cb01e78f61321d5542780a778f49466246f3010bbbe9951be3f4c811ce2"
Jul 06 20:52:50.153843 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:50.153785    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86770cb01e78f61321d5542780a778f49466246f3010bbbe9951be3f4c811ce2"} err="failed to get container status \"86770cb01e78f61321d5542780a778f49466246f3010bbbe9951be3f4c811ce2\": rpc error: code = NotFound desc = could not find container \"86770cb01e78f61321d5542780a778f49466246f3010bbbe9951be3f4c811ce2\": container with ID starting with 86770cb01e78f61321d5542780a778f49466246f3010bbbe9951be3f4c811ce2 not found: ID does not exist"
Jul 06 20:52:50.163540 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:50.163513    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-777556858d-l9lns"]
Jul 06 20:52:50.167091 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:50.167059    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-777556858d-l9lns"]
Jul 06 20:52:52.001209 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:52:52.001173    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74971c84-5a01-4329-bb2d-35408f3136a8" path="/var/lib/kubelet/pods/74971c84-5a01-4329-bb2d-35408f3136a8/volumes"
Jul 06 20:53:39.021874 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.021843    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d58bcc545-684v8"]
Jul 06 20:53:39.022405 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.022273    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74971c84-5a01-4329-bb2d-35408f3136a8" containerName="console"
Jul 06 20:53:39.022405 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.022292    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="74971c84-5a01-4329-bb2d-35408f3136a8" containerName="console"
Jul 06 20:53:39.022405 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.022361    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="74971c84-5a01-4329-bb2d-35408f3136a8" containerName="console"
Jul 06 20:53:39.025073 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.025049    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.041383 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.041359    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d58bcc545-684v8"]
Jul 06 20:53:39.058806 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.058780    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-oauth-config\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.058940 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.058815    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-oauth-serving-cert\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.058940 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.058837    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-config\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.058940 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.058918    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-serving-cert\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.059128 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.058966    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-service-ca\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.059128 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.059034    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjm9x\" (UniqueName: \"kubernetes.io/projected/a95354bb-38cb-4186-bb3b-bb1669fd5a86-kube-api-access-hjm9x\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.059128 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.059066    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-trusted-ca-bundle\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.159450 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.159415    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjm9x\" (UniqueName: \"kubernetes.io/projected/a95354bb-38cb-4186-bb3b-bb1669fd5a86-kube-api-access-hjm9x\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.159655 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.159474    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-trusted-ca-bundle\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.159655 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.159516    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-oauth-config\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.159655 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.159544    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-oauth-serving-cert\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.159655 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.159576    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-config\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.159655 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.159616    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-serving-cert\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.159655 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.159648    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-service-ca\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.160361 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.160330    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-oauth-serving-cert\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.160513 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.160383    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-service-ca\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.160513 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.160413    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-config\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.160513 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.160483    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-trusted-ca-bundle\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.162007 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.161984    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-oauth-config\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.162561 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.162527    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-serving-cert\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.168596 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.168575    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjm9x\" (UniqueName: \"kubernetes.io/projected/a95354bb-38cb-4186-bb3b-bb1669fd5a86-kube-api-access-hjm9x\") pod \"console-d58bcc545-684v8\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") " pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.334407 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.334323    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:39.460827 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:39.460645    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d58bcc545-684v8"]
Jul 06 20:53:39.463523 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:53:39.463495    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda95354bb_38cb_4186_bb3b_bb1669fd5a86.slice/crio-ae788df24341821ca44a96b8fbcb3afad7d0238929a6a1427c6488e49b217ee6 WatchSource:0}: Error finding container ae788df24341821ca44a96b8fbcb3afad7d0238929a6a1427c6488e49b217ee6: Status 404 returned error can't find the container with id ae788df24341821ca44a96b8fbcb3afad7d0238929a6a1427c6488e49b217ee6
Jul 06 20:53:40.328679 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:40.328648    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d58bcc545-684v8" event={"ID":"a95354bb-38cb-4186-bb3b-bb1669fd5a86","Type":"ContainerStarted","Data":"d18b1154d7f2a7cf01356ea049e7aba2c53a99acb9117778c9b5172e2407bfd5"}
Jul 06 20:53:40.329131 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:40.328694    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d58bcc545-684v8" event={"ID":"a95354bb-38cb-4186-bb3b-bb1669fd5a86","Type":"ContainerStarted","Data":"ae788df24341821ca44a96b8fbcb3afad7d0238929a6a1427c6488e49b217ee6"}
Jul 06 20:53:40.356531 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:40.356478    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d58bcc545-684v8" podStartSLOduration=2.35643923 podStartE2EDuration="2.35643923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:53:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:53:40.355768654 +0000 UTC m=+298.822009907" watchObservedRunningTime="2026-07-06 20:53:40.35643923 +0000 UTC m=+298.822680483"
Jul 06 20:53:41.913889 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:41.913864    2566 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 20:53:49.335166 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:49.335123    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:49.337685 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:49.335195    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:49.339873 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:49.339852    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:49.365740 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:49.365717    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:53:49.428517 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:53:49.428486    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b87558bc4-8knlv"]
Jul 06 20:54:14.450741 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.450677    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6b87558bc4-8knlv" podUID="5474c763-ce8a-4abf-8191-900b89b307a5" containerName="console" containerID="cri-o://3143d303ddedcd650ae5d960d881d15bf6f73b09914c0d8ae129b49bfacef797" gracePeriod=15
Jul 06 20:54:14.691049 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.691016    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:54:14.836803 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.836746    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5474c763-ce8a-4abf-8191-900b89b307a5-console-oauth-config\") pod \"5474c763-ce8a-4abf-8191-900b89b307a5\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") "
Jul 06 20:54:14.836803 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.836812    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-service-ca\") pod \"5474c763-ce8a-4abf-8191-900b89b307a5\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") "
Jul 06 20:54:14.837073 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.836836    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nff2g\" (UniqueName: \"kubernetes.io/projected/5474c763-ce8a-4abf-8191-900b89b307a5-kube-api-access-nff2g\") pod \"5474c763-ce8a-4abf-8191-900b89b307a5\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") "
Jul 06 20:54:14.837073 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.836857    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-console-config\") pod \"5474c763-ce8a-4abf-8191-900b89b307a5\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") "
Jul 06 20:54:14.837073 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.836888    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-trusted-ca-bundle\") pod \"5474c763-ce8a-4abf-8191-900b89b307a5\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") "
Jul 06 20:54:14.837073 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.836910    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-oauth-serving-cert\") pod \"5474c763-ce8a-4abf-8191-900b89b307a5\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") "
Jul 06 20:54:14.837073 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.836965    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5474c763-ce8a-4abf-8191-900b89b307a5-console-serving-cert\") pod \"5474c763-ce8a-4abf-8191-900b89b307a5\" (UID: \"5474c763-ce8a-4abf-8191-900b89b307a5\") "
Jul 06 20:54:14.837394 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.837365    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-console-config" (OuterVolumeSpecName: "console-config") pod "5474c763-ce8a-4abf-8191-900b89b307a5" (UID: "5474c763-ce8a-4abf-8191-900b89b307a5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:54:14.837508 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.837383    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5474c763-ce8a-4abf-8191-900b89b307a5" (UID: "5474c763-ce8a-4abf-8191-900b89b307a5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:54:14.837508 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.837381    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-service-ca" (OuterVolumeSpecName: "service-ca") pod "5474c763-ce8a-4abf-8191-900b89b307a5" (UID: "5474c763-ce8a-4abf-8191-900b89b307a5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:54:14.837508 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.837490    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5474c763-ce8a-4abf-8191-900b89b307a5" (UID: "5474c763-ce8a-4abf-8191-900b89b307a5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:54:14.839042 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.839014    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5474c763-ce8a-4abf-8191-900b89b307a5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5474c763-ce8a-4abf-8191-900b89b307a5" (UID: "5474c763-ce8a-4abf-8191-900b89b307a5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:54:14.839129 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.839083    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5474c763-ce8a-4abf-8191-900b89b307a5-kube-api-access-nff2g" (OuterVolumeSpecName: "kube-api-access-nff2g") pod "5474c763-ce8a-4abf-8191-900b89b307a5" (UID: "5474c763-ce8a-4abf-8191-900b89b307a5"). InnerVolumeSpecName "kube-api-access-nff2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:54:14.839172 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.839122    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5474c763-ce8a-4abf-8191-900b89b307a5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5474c763-ce8a-4abf-8191-900b89b307a5" (UID: "5474c763-ce8a-4abf-8191-900b89b307a5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:54:14.937665 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.937617    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:54:14.937665 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.937660    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nff2g\" (UniqueName: \"kubernetes.io/projected/5474c763-ce8a-4abf-8191-900b89b307a5-kube-api-access-nff2g\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:54:14.937665 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.937670    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-console-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:54:14.937665 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.937679    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-trusted-ca-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:54:14.937924 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.937688    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5474c763-ce8a-4abf-8191-900b89b307a5-oauth-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:54:14.937924 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.937697    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5474c763-ce8a-4abf-8191-900b89b307a5-console-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:54:14.937924 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:14.937705    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5474c763-ce8a-4abf-8191-900b89b307a5-console-oauth-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:54:15.455828 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:15.455786    2566 generic.go:356] "Generic (PLEG): container finished" podID="5474c763-ce8a-4abf-8191-900b89b307a5" containerID="3143d303ddedcd650ae5d960d881d15bf6f73b09914c0d8ae129b49bfacef797" exitCode=0
Jul 06 20:54:15.455828 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:15.455830    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b87558bc4-8knlv"
Jul 06 20:54:15.456314 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:15.455873    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b87558bc4-8knlv" event={"ID":"5474c763-ce8a-4abf-8191-900b89b307a5","Type":"ContainerDied","Data":"3143d303ddedcd650ae5d960d881d15bf6f73b09914c0d8ae129b49bfacef797"}
Jul 06 20:54:15.456314 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:15.455925    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b87558bc4-8knlv" event={"ID":"5474c763-ce8a-4abf-8191-900b89b307a5","Type":"ContainerDied","Data":"e98c15f7a80fb1753ad30fa55718d4fca7af3977714b9781d5a78923e1543f57"}
Jul 06 20:54:15.456314 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:15.455945    2566 scope.go:117] "RemoveContainer" containerID="3143d303ddedcd650ae5d960d881d15bf6f73b09914c0d8ae129b49bfacef797"
Jul 06 20:54:15.464290 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:15.464271    2566 scope.go:117] "RemoveContainer" containerID="3143d303ddedcd650ae5d960d881d15bf6f73b09914c0d8ae129b49bfacef797"
Jul 06 20:54:15.464559 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:54:15.464538    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3143d303ddedcd650ae5d960d881d15bf6f73b09914c0d8ae129b49bfacef797\": container with ID starting with 3143d303ddedcd650ae5d960d881d15bf6f73b09914c0d8ae129b49bfacef797 not found: ID does not exist" containerID="3143d303ddedcd650ae5d960d881d15bf6f73b09914c0d8ae129b49bfacef797"
Jul 06 20:54:15.464633 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:15.464576    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3143d303ddedcd650ae5d960d881d15bf6f73b09914c0d8ae129b49bfacef797"} err="failed to get container status \"3143d303ddedcd650ae5d960d881d15bf6f73b09914c0d8ae129b49bfacef797\": rpc error: code = NotFound desc = could not find container \"3143d303ddedcd650ae5d960d881d15bf6f73b09914c0d8ae129b49bfacef797\": container with ID starting with 3143d303ddedcd650ae5d960d881d15bf6f73b09914c0d8ae129b49bfacef797 not found: ID does not exist"
Jul 06 20:54:15.475918 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:15.475890    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b87558bc4-8knlv"]
Jul 06 20:54:15.479666 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:15.479635    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b87558bc4-8knlv"]
Jul 06 20:54:16.001300 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:16.001266    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5474c763-ce8a-4abf-8191-900b89b307a5" path="/var/lib/kubelet/pods/5474c763-ce8a-4abf-8191-900b89b307a5/volumes"
Jul 06 20:54:26.094388 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.094353    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"]
Jul 06 20:54:26.094814 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.094662    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5474c763-ce8a-4abf-8191-900b89b307a5" containerName="console"
Jul 06 20:54:26.094814 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.094673    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5474c763-ce8a-4abf-8191-900b89b307a5" containerName="console"
Jul 06 20:54:26.094814 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.094714    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5474c763-ce8a-4abf-8191-900b89b307a5" containerName="console"
Jul 06 20:54:26.100671 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.100638    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"
Jul 06 20:54:26.104116 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.104094    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kl48c\""
Jul 06 20:54:26.104753 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.104737    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 20:54:26.104851 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.104834    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 20:54:26.115506 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.115474    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"]
Jul 06 20:54:26.225934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.225892    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600ad30e-6aa0-47ca-8d02-2ec9bc526208-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd\" (UID: \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"
Jul 06 20:54:26.225934 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.225938    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600ad30e-6aa0-47ca-8d02-2ec9bc526208-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd\" (UID: \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"
Jul 06 20:54:26.226152 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.226010    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l9z8\" (UniqueName: \"kubernetes.io/projected/600ad30e-6aa0-47ca-8d02-2ec9bc526208-kube-api-access-7l9z8\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd\" (UID: \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"
Jul 06 20:54:26.327260 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.327214    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600ad30e-6aa0-47ca-8d02-2ec9bc526208-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd\" (UID: \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"
Jul 06 20:54:26.327260 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.327264    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600ad30e-6aa0-47ca-8d02-2ec9bc526208-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd\" (UID: \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"
Jul 06 20:54:26.327521 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.327299    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9z8\" (UniqueName: \"kubernetes.io/projected/600ad30e-6aa0-47ca-8d02-2ec9bc526208-kube-api-access-7l9z8\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd\" (UID: \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"
Jul 06 20:54:26.327667 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.327645    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600ad30e-6aa0-47ca-8d02-2ec9bc526208-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd\" (UID: \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"
Jul 06 20:54:26.327703 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.327670    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600ad30e-6aa0-47ca-8d02-2ec9bc526208-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd\" (UID: \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"
Jul 06 20:54:26.335775 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.335746    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l9z8\" (UniqueName: \"kubernetes.io/projected/600ad30e-6aa0-47ca-8d02-2ec9bc526208-kube-api-access-7l9z8\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd\" (UID: \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"
Jul 06 20:54:26.409962 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.409867    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"
Jul 06 20:54:26.542515 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.542334    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"]
Jul 06 20:54:26.544599 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:54:26.544573    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600ad30e_6aa0_47ca_8d02_2ec9bc526208.slice/crio-1a5cdedcff23d09761e1a446fc172bdf54f796d04d41b88a0940cfe1363b5c5d WatchSource:0}: Error finding container 1a5cdedcff23d09761e1a446fc172bdf54f796d04d41b88a0940cfe1363b5c5d: Status 404 returned error can't find the container with id 1a5cdedcff23d09761e1a446fc172bdf54f796d04d41b88a0940cfe1363b5c5d
Jul 06 20:54:26.546470 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:26.546439    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:54:27.501738 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:27.501700    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd" event={"ID":"600ad30e-6aa0-47ca-8d02-2ec9bc526208","Type":"ContainerStarted","Data":"1a5cdedcff23d09761e1a446fc172bdf54f796d04d41b88a0940cfe1363b5c5d"}
Jul 06 20:54:33.527474 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:33.527433    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd" event={"ID":"600ad30e-6aa0-47ca-8d02-2ec9bc526208","Type":"ContainerStarted","Data":"f0959b83bcb784c39322e4d20f6f32fb3d66a5101fe70fa8f72d8bbe9940a8a4"}
Jul 06 20:54:34.532974 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:34.532898    2566 generic.go:356] "Generic (PLEG): container finished" podID="600ad30e-6aa0-47ca-8d02-2ec9bc526208" containerID="f0959b83bcb784c39322e4d20f6f32fb3d66a5101fe70fa8f72d8bbe9940a8a4" exitCode=0
Jul 06 20:54:34.533354 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:34.532975    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd" event={"ID":"600ad30e-6aa0-47ca-8d02-2ec9bc526208","Type":"ContainerDied","Data":"f0959b83bcb784c39322e4d20f6f32fb3d66a5101fe70fa8f72d8bbe9940a8a4"}
Jul 06 20:54:36.542124 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:36.542087    2566 generic.go:356] "Generic (PLEG): container finished" podID="600ad30e-6aa0-47ca-8d02-2ec9bc526208" containerID="f006900c016a54b57e4b0c769f4031d475b78c8ee38057b33cc3111dedfa5681" exitCode=0
Jul 06 20:54:36.542517 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:36.542144    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd" event={"ID":"600ad30e-6aa0-47ca-8d02-2ec9bc526208","Type":"ContainerDied","Data":"f006900c016a54b57e4b0c769f4031d475b78c8ee38057b33cc3111dedfa5681"}
Jul 06 20:54:44.577747 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:44.577715    2566 generic.go:356] "Generic (PLEG): container finished" podID="600ad30e-6aa0-47ca-8d02-2ec9bc526208" containerID="f841cb2d5fb0b7d8b370fcc0a42baa98de27f0b960d4bf0b6d8414cfb626033b" exitCode=0
Jul 06 20:54:44.578138 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:44.577801    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd" event={"ID":"600ad30e-6aa0-47ca-8d02-2ec9bc526208","Type":"ContainerDied","Data":"f841cb2d5fb0b7d8b370fcc0a42baa98de27f0b960d4bf0b6d8414cfb626033b"}
Jul 06 20:54:45.706617 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:45.706585    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"
Jul 06 20:54:45.778804 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:45.778773    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l9z8\" (UniqueName: \"kubernetes.io/projected/600ad30e-6aa0-47ca-8d02-2ec9bc526208-kube-api-access-7l9z8\") pod \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\" (UID: \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\") "
Jul 06 20:54:45.778976 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:45.778821    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600ad30e-6aa0-47ca-8d02-2ec9bc526208-util\") pod \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\" (UID: \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\") "
Jul 06 20:54:45.778976 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:45.778845    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600ad30e-6aa0-47ca-8d02-2ec9bc526208-bundle\") pod \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\" (UID: \"600ad30e-6aa0-47ca-8d02-2ec9bc526208\") "
Jul 06 20:54:45.779470 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:45.779424    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600ad30e-6aa0-47ca-8d02-2ec9bc526208-bundle" (OuterVolumeSpecName: "bundle") pod "600ad30e-6aa0-47ca-8d02-2ec9bc526208" (UID: "600ad30e-6aa0-47ca-8d02-2ec9bc526208"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:54:45.781020 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:45.780989    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/600ad30e-6aa0-47ca-8d02-2ec9bc526208-kube-api-access-7l9z8" (OuterVolumeSpecName: "kube-api-access-7l9z8") pod "600ad30e-6aa0-47ca-8d02-2ec9bc526208" (UID: "600ad30e-6aa0-47ca-8d02-2ec9bc526208"). InnerVolumeSpecName "kube-api-access-7l9z8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:54:45.783312 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:45.783286    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600ad30e-6aa0-47ca-8d02-2ec9bc526208-util" (OuterVolumeSpecName: "util") pod "600ad30e-6aa0-47ca-8d02-2ec9bc526208" (UID: "600ad30e-6aa0-47ca-8d02-2ec9bc526208"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:54:45.880416 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:45.880328    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l9z8\" (UniqueName: \"kubernetes.io/projected/600ad30e-6aa0-47ca-8d02-2ec9bc526208-kube-api-access-7l9z8\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:54:45.880416 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:45.880360    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600ad30e-6aa0-47ca-8d02-2ec9bc526208-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:54:45.880416 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:45.880371    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600ad30e-6aa0-47ca-8d02-2ec9bc526208-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:54:46.586471 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:46.586422    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd"
Jul 06 20:54:46.586471 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:46.586419    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589r45jd" event={"ID":"600ad30e-6aa0-47ca-8d02-2ec9bc526208","Type":"ContainerDied","Data":"1a5cdedcff23d09761e1a446fc172bdf54f796d04d41b88a0940cfe1363b5c5d"}
Jul 06 20:54:46.586743 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:46.586513    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a5cdedcff23d09761e1a446fc172bdf54f796d04d41b88a0940cfe1363b5c5d"
Jul 06 20:54:49.500870 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.500835    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6"]
Jul 06 20:54:49.501270 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.501237    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600ad30e-6aa0-47ca-8d02-2ec9bc526208" containerName="pull"
Jul 06 20:54:49.501270 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.501252    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="600ad30e-6aa0-47ca-8d02-2ec9bc526208" containerName="pull"
Jul 06 20:54:49.501270 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.501268    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600ad30e-6aa0-47ca-8d02-2ec9bc526208" containerName="extract"
Jul 06 20:54:49.501386 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.501276    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="600ad30e-6aa0-47ca-8d02-2ec9bc526208" containerName="extract"
Jul 06 20:54:49.501386 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.501308    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600ad30e-6aa0-47ca-8d02-2ec9bc526208" containerName="util"
Jul 06 20:54:49.501386 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.501317    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="600ad30e-6aa0-47ca-8d02-2ec9bc526208" containerName="util"
Jul 06 20:54:49.501386 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.501385    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="600ad30e-6aa0-47ca-8d02-2ec9bc526208" containerName="extract"
Jul 06 20:54:49.524226 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.524198    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6"
Jul 06 20:54:49.528985 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.528960    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 06 20:54:49.529131 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.529003    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 06 20:54:49.529131 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.529052    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-k9mvk\""
Jul 06 20:54:49.529308 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.529291    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 06 20:54:49.529521 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.529502    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6"]
Jul 06 20:54:49.608465 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.608430    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/c86b6ef3-8124-4edc-bf86-294bd3ce9d23-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-s58g6\" (UID: \"c86b6ef3-8124-4edc-bf86-294bd3ce9d23\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6"
Jul 06 20:54:49.608670 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.608552    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26ppt\" (UniqueName: \"kubernetes.io/projected/c86b6ef3-8124-4edc-bf86-294bd3ce9d23-kube-api-access-26ppt\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-s58g6\" (UID: \"c86b6ef3-8124-4edc-bf86-294bd3ce9d23\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6"
Jul 06 20:54:49.709116 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.709073    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26ppt\" (UniqueName: \"kubernetes.io/projected/c86b6ef3-8124-4edc-bf86-294bd3ce9d23-kube-api-access-26ppt\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-s58g6\" (UID: \"c86b6ef3-8124-4edc-bf86-294bd3ce9d23\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6"
Jul 06 20:54:49.709299 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.709233    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/c86b6ef3-8124-4edc-bf86-294bd3ce9d23-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-s58g6\" (UID: \"c86b6ef3-8124-4edc-bf86-294bd3ce9d23\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6"
Jul 06 20:54:49.711725 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.711696    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/c86b6ef3-8124-4edc-bf86-294bd3ce9d23-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-s58g6\" (UID: \"c86b6ef3-8124-4edc-bf86-294bd3ce9d23\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6"
Jul 06 20:54:49.718598 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.718574    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26ppt\" (UniqueName: \"kubernetes.io/projected/c86b6ef3-8124-4edc-bf86-294bd3ce9d23-kube-api-access-26ppt\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-s58g6\" (UID: \"c86b6ef3-8124-4edc-bf86-294bd3ce9d23\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6"
Jul 06 20:54:49.836071 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.835981    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6"
Jul 06 20:54:49.995952 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:49.995918    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6"]
Jul 06 20:54:49.999774 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:54:49.999750    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc86b6ef3_8124_4edc_bf86_294bd3ce9d23.slice/crio-faabd27c96fba867671435912967fc53c9c8ec48c1085fde08fe837456c432b5 WatchSource:0}: Error finding container faabd27c96fba867671435912967fc53c9c8ec48c1085fde08fe837456c432b5: Status 404 returned error can't find the container with id faabd27c96fba867671435912967fc53c9c8ec48c1085fde08fe837456c432b5
Jul 06 20:54:50.603019 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:50.602979    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6" event={"ID":"c86b6ef3-8124-4edc-bf86-294bd3ce9d23","Type":"ContainerStarted","Data":"faabd27c96fba867671435912967fc53c9c8ec48c1085fde08fe837456c432b5"}
Jul 06 20:54:53.616844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:53.616807    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6" event={"ID":"c86b6ef3-8124-4edc-bf86-294bd3ce9d23","Type":"ContainerStarted","Data":"ec98178ecfef1ae90496c005a8dbe867d8466fccaf719d48494ceb6c7f22bea2"}
Jul 06 20:54:53.617231 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:53.616873    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6"
Jul 06 20:54:53.638962 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:53.638908    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6" podStartSLOduration=1.74233697 podStartE2EDuration="4.638892263s" totalImagesPullingTime="2.896555293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:54:53.63697755 +0000 UTC m=+372.103218803" watchObservedRunningTime="2026-07-06 20:54:53.638892263 +0000 UTC m=+372.105133516"
Jul 06 20:54:54.627991 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:54.627959    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-h66dd"]
Jul 06 20:54:54.630149 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:54.630128    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-h66dd"
Jul 06 20:54:54.632428 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:54.632407    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 06 20:54:54.632620 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:54.632595    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-swzgz\""
Jul 06 20:54:54.632686 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:54.632631    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 06 20:54:54.641686 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:54.641664    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-h66dd"]
Jul 06 20:54:54.755667 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:54.755627    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbn4v\" (UniqueName: \"kubernetes.io/projected/07bb69d1-8127-4059-8fb6-f267c560ff94-kube-api-access-wbn4v\") pod \"keda-admission-5fbd5c4644-h66dd\" (UID: \"07bb69d1-8127-4059-8fb6-f267c560ff94\") " pod="openshift-keda/keda-admission-5fbd5c4644-h66dd"
Jul 06 20:54:54.755833 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:54.755685    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/07bb69d1-8127-4059-8fb6-f267c560ff94-certificates\") pod \"keda-admission-5fbd5c4644-h66dd\" (UID: \"07bb69d1-8127-4059-8fb6-f267c560ff94\") " pod="openshift-keda/keda-admission-5fbd5c4644-h66dd"
Jul 06 20:54:54.856755 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:54.856714    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbn4v\" (UniqueName: \"kubernetes.io/projected/07bb69d1-8127-4059-8fb6-f267c560ff94-kube-api-access-wbn4v\") pod \"keda-admission-5fbd5c4644-h66dd\" (UID: \"07bb69d1-8127-4059-8fb6-f267c560ff94\") " pod="openshift-keda/keda-admission-5fbd5c4644-h66dd"
Jul 06 20:54:54.856930 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:54.856888    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/07bb69d1-8127-4059-8fb6-f267c560ff94-certificates\") pod \"keda-admission-5fbd5c4644-h66dd\" (UID: \"07bb69d1-8127-4059-8fb6-f267c560ff94\") " pod="openshift-keda/keda-admission-5fbd5c4644-h66dd"
Jul 06 20:54:54.857037 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:54:54.857019    2566 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Jul 06 20:54:54.857100 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:54:54.857049    2566 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-5fbd5c4644-h66dd: secret "keda-admission-webhooks-certs" not found
Jul 06 20:54:54.857150 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:54:54.857143    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07bb69d1-8127-4059-8fb6-f267c560ff94-certificates podName:07bb69d1-8127-4059-8fb6-f267c560ff94 nodeName:}" failed. No retries permitted until 2026-07-06 20:54:55.35711448 +0000 UTC m=+373.823355727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/07bb69d1-8127-4059-8fb6-f267c560ff94-certificates") pod "keda-admission-5fbd5c4644-h66dd" (UID: "07bb69d1-8127-4059-8fb6-f267c560ff94") : secret "keda-admission-webhooks-certs" not found
Jul 06 20:54:54.869205 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:54.869175    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbn4v\" (UniqueName: \"kubernetes.io/projected/07bb69d1-8127-4059-8fb6-f267c560ff94-kube-api-access-wbn4v\") pod \"keda-admission-5fbd5c4644-h66dd\" (UID: \"07bb69d1-8127-4059-8fb6-f267c560ff94\") " pod="openshift-keda/keda-admission-5fbd5c4644-h66dd"
Jul 06 20:54:55.361881 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:55.361839    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/07bb69d1-8127-4059-8fb6-f267c560ff94-certificates\") pod \"keda-admission-5fbd5c4644-h66dd\" (UID: \"07bb69d1-8127-4059-8fb6-f267c560ff94\") " pod="openshift-keda/keda-admission-5fbd5c4644-h66dd"
Jul 06 20:54:55.364370 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:55.364340    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/07bb69d1-8127-4059-8fb6-f267c560ff94-certificates\") pod \"keda-admission-5fbd5c4644-h66dd\" (UID: \"07bb69d1-8127-4059-8fb6-f267c560ff94\") " pod="openshift-keda/keda-admission-5fbd5c4644-h66dd"
Jul 06 20:54:55.540613 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:55.540574    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-h66dd"
Jul 06 20:54:55.670144 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:55.670108    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-h66dd"]
Jul 06 20:54:55.672957 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:54:55.672925    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07bb69d1_8127_4059_8fb6_f267c560ff94.slice/crio-98028a46ff9d89546be0d83ebceb433f664b289204e52115e18ec8b61e682a6e WatchSource:0}: Error finding container 98028a46ff9d89546be0d83ebceb433f664b289204e52115e18ec8b61e682a6e: Status 404 returned error can't find the container with id 98028a46ff9d89546be0d83ebceb433f664b289204e52115e18ec8b61e682a6e
Jul 06 20:54:56.630509 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:56.630473    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-h66dd" event={"ID":"07bb69d1-8127-4059-8fb6-f267c560ff94","Type":"ContainerStarted","Data":"98028a46ff9d89546be0d83ebceb433f664b289204e52115e18ec8b61e682a6e"}
Jul 06 20:54:57.636086 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:57.636051    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-h66dd" event={"ID":"07bb69d1-8127-4059-8fb6-f267c560ff94","Type":"ContainerStarted","Data":"b529328c0971c8a5f6ec677ec7966df2fc246c8cc712d2170a316164a3eb6553"}
Jul 06 20:54:57.636520 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:57.636161    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-h66dd"
Jul 06 20:54:57.657815 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:54:57.657753    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-h66dd" podStartSLOduration=2.308657245 podStartE2EDuration="3.657734949s" totalImagesPullingTime="1.349077704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:54:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:54:57.656769073 +0000 UTC m=+376.123010327" watchObservedRunningTime="2026-07-06 20:54:57.657734949 +0000 UTC m=+376.123976203"
Jul 06 20:55:14.623476 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:55:14.623421    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-s58g6"
Jul 06 20:55:18.642296 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:55:18.642265    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-h66dd"
Jul 06 20:56:05.924427 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:05.924346    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-664cc55bd7-fjmx5"]
Jul 06 20:56:05.927304 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:05.927283    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5"
Jul 06 20:56:05.929857 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:05.929834    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 06 20:56:05.930402 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:05.930379    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 06 20:56:05.930402 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:05.930399    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-lz5px\""
Jul 06 20:56:05.930605 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:05.930585    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 06 20:56:05.957356 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:05.957323    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-664cc55bd7-fjmx5"]
Jul 06 20:56:05.961198 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:05.961170    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"]
Jul 06 20:56:05.964208 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:05.964187    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"
Jul 06 20:56:05.967903 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:05.967881    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 06 20:56:05.968037 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:05.967929    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-dk5nc\""
Jul 06 20:56:05.974816 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:05.974790    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"]
Jul 06 20:56:06.037685 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.037646    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26-cert\") pod \"llmisvc-controller-manager-5bcc55b7f6-2glkl\" (UID: \"ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26\") " pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"
Jul 06 20:56:06.037894 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.037697    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5587ef3c-daa5-45e7-80f2-95004041d4d3-cert\") pod \"kserve-controller-manager-664cc55bd7-fjmx5\" (UID: \"5587ef3c-daa5-45e7-80f2-95004041d4d3\") " pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5"
Jul 06 20:56:06.037894 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.037793    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqg6c\" (UniqueName: \"kubernetes.io/projected/ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26-kube-api-access-fqg6c\") pod \"llmisvc-controller-manager-5bcc55b7f6-2glkl\" (UID: \"ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26\") " pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"
Jul 06 20:56:06.037894 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.037837    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pw4s\" (UniqueName: \"kubernetes.io/projected/5587ef3c-daa5-45e7-80f2-95004041d4d3-kube-api-access-7pw4s\") pod \"kserve-controller-manager-664cc55bd7-fjmx5\" (UID: \"5587ef3c-daa5-45e7-80f2-95004041d4d3\") " pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5"
Jul 06 20:56:06.139196 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.139157    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26-cert\") pod \"llmisvc-controller-manager-5bcc55b7f6-2glkl\" (UID: \"ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26\") " pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"
Jul 06 20:56:06.139409 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.139218    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5587ef3c-daa5-45e7-80f2-95004041d4d3-cert\") pod \"kserve-controller-manager-664cc55bd7-fjmx5\" (UID: \"5587ef3c-daa5-45e7-80f2-95004041d4d3\") " pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5"
Jul 06 20:56:06.139409 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.139293    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqg6c\" (UniqueName: \"kubernetes.io/projected/ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26-kube-api-access-fqg6c\") pod \"llmisvc-controller-manager-5bcc55b7f6-2glkl\" (UID: \"ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26\") " pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"
Jul 06 20:56:06.139409 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.139347    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pw4s\" (UniqueName: \"kubernetes.io/projected/5587ef3c-daa5-45e7-80f2-95004041d4d3-kube-api-access-7pw4s\") pod \"kserve-controller-manager-664cc55bd7-fjmx5\" (UID: \"5587ef3c-daa5-45e7-80f2-95004041d4d3\") " pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5"
Jul 06 20:56:06.139409 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:56:06.139369    2566 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 06 20:56:06.139687 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:56:06.139671    2566 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 06 20:56:06.139736 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:56:06.139695    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26-cert podName:ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26 nodeName:}" failed. No retries permitted until 2026-07-06 20:56:06.63967089 +0000 UTC m=+445.105912124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26-cert") pod "llmisvc-controller-manager-5bcc55b7f6-2glkl" (UID: "ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26") : secret "llmisvc-webhook-server-cert" not found
Jul 06 20:56:06.139736 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:56:06.139717    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5587ef3c-daa5-45e7-80f2-95004041d4d3-cert podName:5587ef3c-daa5-45e7-80f2-95004041d4d3 nodeName:}" failed. No retries permitted until 2026-07-06 20:56:06.639706596 +0000 UTC m=+445.105947828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5587ef3c-daa5-45e7-80f2-95004041d4d3-cert") pod "kserve-controller-manager-664cc55bd7-fjmx5" (UID: "5587ef3c-daa5-45e7-80f2-95004041d4d3") : secret "kserve-webhook-server-cert" not found
Jul 06 20:56:06.155760 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.155730    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqg6c\" (UniqueName: \"kubernetes.io/projected/ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26-kube-api-access-fqg6c\") pod \"llmisvc-controller-manager-5bcc55b7f6-2glkl\" (UID: \"ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26\") " pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"
Jul 06 20:56:06.157441 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.157415    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pw4s\" (UniqueName: \"kubernetes.io/projected/5587ef3c-daa5-45e7-80f2-95004041d4d3-kube-api-access-7pw4s\") pod \"kserve-controller-manager-664cc55bd7-fjmx5\" (UID: \"5587ef3c-daa5-45e7-80f2-95004041d4d3\") " pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5"
Jul 06 20:56:06.644676 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.644632    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26-cert\") pod \"llmisvc-controller-manager-5bcc55b7f6-2glkl\" (UID: \"ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26\") " pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"
Jul 06 20:56:06.644878 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.644698    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5587ef3c-daa5-45e7-80f2-95004041d4d3-cert\") pod \"kserve-controller-manager-664cc55bd7-fjmx5\" (UID: \"5587ef3c-daa5-45e7-80f2-95004041d4d3\") " pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5"
Jul 06 20:56:06.647052 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.647021    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5587ef3c-daa5-45e7-80f2-95004041d4d3-cert\") pod \"kserve-controller-manager-664cc55bd7-fjmx5\" (UID: \"5587ef3c-daa5-45e7-80f2-95004041d4d3\") " pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5"
Jul 06 20:56:06.647213 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.647192    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26-cert\") pod \"llmisvc-controller-manager-5bcc55b7f6-2glkl\" (UID: \"ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26\") " pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"
Jul 06 20:56:06.837852 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.837812    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5"
Jul 06 20:56:06.875793 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.875155    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"
Jul 06 20:56:06.973971 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:06.973939    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-664cc55bd7-fjmx5"]
Jul 06 20:56:06.977967 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:56:06.977938    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5587ef3c_daa5_45e7_80f2_95004041d4d3.slice/crio-911b6d2953d2888e151d93acdbea1cd2bb4bea19000577210ec5f4dca1c90f1a WatchSource:0}: Error finding container 911b6d2953d2888e151d93acdbea1cd2bb4bea19000577210ec5f4dca1c90f1a: Status 404 returned error can't find the container with id 911b6d2953d2888e151d93acdbea1cd2bb4bea19000577210ec5f4dca1c90f1a
Jul 06 20:56:07.019086 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:07.019057    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"]
Jul 06 20:56:07.019898 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:56:07.019873    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podff0c4ffd_7f29_4a03_9fd0_d0f204d09a26.slice/crio-9bda566e961da41f5bfa7de03de7fb0c9c0d26c4d2124d0f26f8ddbf8b66f4f6 WatchSource:0}: Error finding container 9bda566e961da41f5bfa7de03de7fb0c9c0d26c4d2124d0f26f8ddbf8b66f4f6: Status 404 returned error can't find the container with id 9bda566e961da41f5bfa7de03de7fb0c9c0d26c4d2124d0f26f8ddbf8b66f4f6
Jul 06 20:56:07.904121 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:07.904038    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5" event={"ID":"5587ef3c-daa5-45e7-80f2-95004041d4d3","Type":"ContainerStarted","Data":"911b6d2953d2888e151d93acdbea1cd2bb4bea19000577210ec5f4dca1c90f1a"}
Jul 06 20:56:07.906168 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:07.906106    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl" event={"ID":"ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26","Type":"ContainerStarted","Data":"9bda566e961da41f5bfa7de03de7fb0c9c0d26c4d2124d0f26f8ddbf8b66f4f6"}
Jul 06 20:56:09.919299 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:09.919260    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5" event={"ID":"5587ef3c-daa5-45e7-80f2-95004041d4d3","Type":"ContainerStarted","Data":"49b465582f5c7a7c4ecd8160527a5ff0885d486e6fffa8b9435e741cd33eaf47"}
Jul 06 20:56:09.919751 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:09.919418    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5"
Jul 06 20:56:09.941035 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:09.940979    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5" podStartSLOduration=2.126978334 podStartE2EDuration="4.940957948s" totalImagesPullingTime="2.813979614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:56:09.9398455 +0000 UTC m=+448.406086753" watchObservedRunningTime="2026-07-06 20:56:09.940957948 +0000 UTC m=+448.407199202"
Jul 06 20:56:10.924546 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:10.924500    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl" event={"ID":"ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26","Type":"ContainerStarted","Data":"39f3f45d255bd411cdcefbdc5d577889c00ef4d206a2f4e5b640652935913ee3"}
Jul 06 20:56:10.925005 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:10.924669    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"
Jul 06 20:56:10.945037 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:10.944980    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl" podStartSLOduration=2.502562522 podStartE2EDuration="5.944964418s" totalImagesPullingTime="3.442401896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:56:10.944316674 +0000 UTC m=+449.410557928" watchObservedRunningTime="2026-07-06 20:56:10.944964418 +0000 UTC m=+449.411205671"
Jul 06 20:56:40.930345 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:40.930314    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5"
Jul 06 20:56:41.931848 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:41.931814    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-5bcc55b7f6-2glkl"
Jul 06 20:56:43.829171 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:43.829131    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"]
Jul 06 20:56:43.831590 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:43.831572    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"
Jul 06 20:56:43.834201 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:43.834175    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 20:56:43.834334 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:43.834175    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kl48c\""
Jul 06 20:56:43.834334 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:43.834215    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 20:56:43.844189 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:43.844164    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"]
Jul 06 20:56:43.958447 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:43.958409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7lkp\" (UniqueName: \"kubernetes.io/projected/8e59d841-f8bd-4181-8657-f5ab50bc69db-kube-api-access-p7lkp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr\" (UID: \"8e59d841-f8bd-4181-8657-f5ab50bc69db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"
Jul 06 20:56:43.958447 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:43.958450    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e59d841-f8bd-4181-8657-f5ab50bc69db-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr\" (UID: \"8e59d841-f8bd-4181-8657-f5ab50bc69db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"
Jul 06 20:56:43.958702 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:43.958514    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e59d841-f8bd-4181-8657-f5ab50bc69db-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr\" (UID: \"8e59d841-f8bd-4181-8657-f5ab50bc69db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"
Jul 06 20:56:44.059812 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:44.059771    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7lkp\" (UniqueName: \"kubernetes.io/projected/8e59d841-f8bd-4181-8657-f5ab50bc69db-kube-api-access-p7lkp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr\" (UID: \"8e59d841-f8bd-4181-8657-f5ab50bc69db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"
Jul 06 20:56:44.059812 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:44.059814    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e59d841-f8bd-4181-8657-f5ab50bc69db-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr\" (UID: \"8e59d841-f8bd-4181-8657-f5ab50bc69db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"
Jul 06 20:56:44.060031 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:44.059847    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e59d841-f8bd-4181-8657-f5ab50bc69db-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr\" (UID: \"8e59d841-f8bd-4181-8657-f5ab50bc69db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"
Jul 06 20:56:44.060205 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:44.060185    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e59d841-f8bd-4181-8657-f5ab50bc69db-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr\" (UID: \"8e59d841-f8bd-4181-8657-f5ab50bc69db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"
Jul 06 20:56:44.060247 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:44.060211    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e59d841-f8bd-4181-8657-f5ab50bc69db-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr\" (UID: \"8e59d841-f8bd-4181-8657-f5ab50bc69db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"
Jul 06 20:56:44.069519 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:44.069450    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7lkp\" (UniqueName: \"kubernetes.io/projected/8e59d841-f8bd-4181-8657-f5ab50bc69db-kube-api-access-p7lkp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr\" (UID: \"8e59d841-f8bd-4181-8657-f5ab50bc69db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"
Jul 06 20:56:44.140834 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:44.140734    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"
Jul 06 20:56:44.279292 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:44.279221    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"]
Jul 06 20:56:44.281988 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:56:44.281956    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e59d841_f8bd_4181_8657_f5ab50bc69db.slice/crio-23275d76a7b6a0f8bdca5a9e3d83577202b67b629365fed244d8a9ee308c26e9 WatchSource:0}: Error finding container 23275d76a7b6a0f8bdca5a9e3d83577202b67b629365fed244d8a9ee308c26e9: Status 404 returned error can't find the container with id 23275d76a7b6a0f8bdca5a9e3d83577202b67b629365fed244d8a9ee308c26e9
Jul 06 20:56:45.084123 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:45.084089    2566 generic.go:356] "Generic (PLEG): container finished" podID="8e59d841-f8bd-4181-8657-f5ab50bc69db" containerID="573d275da358b5e745a4ce606a04c6b06e0f2053affcac878a904ce661cbd4e0" exitCode=0
Jul 06 20:56:45.084437 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:45.084171    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr" event={"ID":"8e59d841-f8bd-4181-8657-f5ab50bc69db","Type":"ContainerDied","Data":"573d275da358b5e745a4ce606a04c6b06e0f2053affcac878a904ce661cbd4e0"}
Jul 06 20:56:45.084437 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:45.084221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr" event={"ID":"8e59d841-f8bd-4181-8657-f5ab50bc69db","Type":"ContainerStarted","Data":"23275d76a7b6a0f8bdca5a9e3d83577202b67b629365fed244d8a9ee308c26e9"}
Jul 06 20:56:52.115964 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:52.115923    2566 generic.go:356] "Generic (PLEG): container finished" podID="8e59d841-f8bd-4181-8657-f5ab50bc69db" containerID="41faf0eb8edb3e959fe942ea9b3c540b91dd1caf9e201edd1b3282efb94cbfda" exitCode=0
Jul 06 20:56:52.116386 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:52.115987    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr" event={"ID":"8e59d841-f8bd-4181-8657-f5ab50bc69db","Type":"ContainerDied","Data":"41faf0eb8edb3e959fe942ea9b3c540b91dd1caf9e201edd1b3282efb94cbfda"}
Jul 06 20:56:53.122242 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:53.122203    2566 generic.go:356] "Generic (PLEG): container finished" podID="8e59d841-f8bd-4181-8657-f5ab50bc69db" containerID="2d88a828755ce39738609558e5d79026b3a8ac55af0d0393a6d8997df91ecfbf" exitCode=0
Jul 06 20:56:53.122651 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:53.122284    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr" event={"ID":"8e59d841-f8bd-4181-8657-f5ab50bc69db","Type":"ContainerDied","Data":"2d88a828755ce39738609558e5d79026b3a8ac55af0d0393a6d8997df91ecfbf"}
Jul 06 20:56:54.254844 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:54.254820    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"
Jul 06 20:56:54.350811 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:54.350777    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e59d841-f8bd-4181-8657-f5ab50bc69db-bundle\") pod \"8e59d841-f8bd-4181-8657-f5ab50bc69db\" (UID: \"8e59d841-f8bd-4181-8657-f5ab50bc69db\") "
Jul 06 20:56:54.350982 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:54.350824    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e59d841-f8bd-4181-8657-f5ab50bc69db-util\") pod \"8e59d841-f8bd-4181-8657-f5ab50bc69db\" (UID: \"8e59d841-f8bd-4181-8657-f5ab50bc69db\") "
Jul 06 20:56:54.350982 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:54.350869    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7lkp\" (UniqueName: \"kubernetes.io/projected/8e59d841-f8bd-4181-8657-f5ab50bc69db-kube-api-access-p7lkp\") pod \"8e59d841-f8bd-4181-8657-f5ab50bc69db\" (UID: \"8e59d841-f8bd-4181-8657-f5ab50bc69db\") "
Jul 06 20:56:54.351466 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:54.351434    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e59d841-f8bd-4181-8657-f5ab50bc69db-bundle" (OuterVolumeSpecName: "bundle") pod "8e59d841-f8bd-4181-8657-f5ab50bc69db" (UID: "8e59d841-f8bd-4181-8657-f5ab50bc69db"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:56:54.353023 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:54.352993    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e59d841-f8bd-4181-8657-f5ab50bc69db-kube-api-access-p7lkp" (OuterVolumeSpecName: "kube-api-access-p7lkp") pod "8e59d841-f8bd-4181-8657-f5ab50bc69db" (UID: "8e59d841-f8bd-4181-8657-f5ab50bc69db"). InnerVolumeSpecName "kube-api-access-p7lkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:56:54.355561 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:54.355536    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e59d841-f8bd-4181-8657-f5ab50bc69db-util" (OuterVolumeSpecName: "util") pod "8e59d841-f8bd-4181-8657-f5ab50bc69db" (UID: "8e59d841-f8bd-4181-8657-f5ab50bc69db"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:56:54.451528 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:54.451416    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e59d841-f8bd-4181-8657-f5ab50bc69db-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:56:54.451528 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:54.451443    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e59d841-f8bd-4181-8657-f5ab50bc69db-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:56:54.451528 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:54.451472    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7lkp\" (UniqueName: \"kubernetes.io/projected/8e59d841-f8bd-4181-8657-f5ab50bc69db-kube-api-access-p7lkp\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:56:55.132294 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:55.132266    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr"
Jul 06 20:56:55.132294 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:55.132278    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540slvhr" event={"ID":"8e59d841-f8bd-4181-8657-f5ab50bc69db","Type":"ContainerDied","Data":"23275d76a7b6a0f8bdca5a9e3d83577202b67b629365fed244d8a9ee308c26e9"}
Jul 06 20:56:55.132534 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:56:55.132320    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23275d76a7b6a0f8bdca5a9e3d83577202b67b629365fed244d8a9ee308c26e9"
Jul 06 20:57:10.647080 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.647037    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"]
Jul 06 20:57:10.647580 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.647352    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e59d841-f8bd-4181-8657-f5ab50bc69db" containerName="pull"
Jul 06 20:57:10.647580 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.647362    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e59d841-f8bd-4181-8657-f5ab50bc69db" containerName="pull"
Jul 06 20:57:10.647580 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.647382    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e59d841-f8bd-4181-8657-f5ab50bc69db" containerName="util"
Jul 06 20:57:10.647580 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.647387    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e59d841-f8bd-4181-8657-f5ab50bc69db" containerName="util"
Jul 06 20:57:10.647580 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.647393    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e59d841-f8bd-4181-8657-f5ab50bc69db" containerName="extract"
Jul 06 20:57:10.647580 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.647398    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e59d841-f8bd-4181-8657-f5ab50bc69db" containerName="extract"
Jul 06 20:57:10.647580 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.647449    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e59d841-f8bd-4181-8657-f5ab50bc69db" containerName="extract"
Jul 06 20:57:10.649488 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.649447    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"
Jul 06 20:57:10.651817 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.651795    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kl48c\""
Jul 06 20:57:10.651817 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.651805    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 20:57:10.652524 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.652503    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 20:57:10.659092 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.659068    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"]
Jul 06 20:57:10.789021 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.788976    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlqld\" (UniqueName: \"kubernetes.io/projected/e1fa3e17-9d3f-4807-af16-05602a7fe693-kube-api-access-hlqld\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd\" (UID: \"e1fa3e17-9d3f-4807-af16-05602a7fe693\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"
Jul 06 20:57:10.789202 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.789029    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1fa3e17-9d3f-4807-af16-05602a7fe693-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd\" (UID: \"e1fa3e17-9d3f-4807-af16-05602a7fe693\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"
Jul 06 20:57:10.789202 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.789063    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1fa3e17-9d3f-4807-af16-05602a7fe693-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd\" (UID: \"e1fa3e17-9d3f-4807-af16-05602a7fe693\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"
Jul 06 20:57:10.890297 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.890261    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlqld\" (UniqueName: \"kubernetes.io/projected/e1fa3e17-9d3f-4807-af16-05602a7fe693-kube-api-access-hlqld\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd\" (UID: \"e1fa3e17-9d3f-4807-af16-05602a7fe693\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"
Jul 06 20:57:10.890297 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.890303    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1fa3e17-9d3f-4807-af16-05602a7fe693-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd\" (UID: \"e1fa3e17-9d3f-4807-af16-05602a7fe693\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"
Jul 06 20:57:10.890535 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.890330    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1fa3e17-9d3f-4807-af16-05602a7fe693-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd\" (UID: \"e1fa3e17-9d3f-4807-af16-05602a7fe693\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"
Jul 06 20:57:10.890677 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.890661    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1fa3e17-9d3f-4807-af16-05602a7fe693-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd\" (UID: \"e1fa3e17-9d3f-4807-af16-05602a7fe693\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"
Jul 06 20:57:10.890726 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.890707    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1fa3e17-9d3f-4807-af16-05602a7fe693-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd\" (UID: \"e1fa3e17-9d3f-4807-af16-05602a7fe693\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"
Jul 06 20:57:10.899634 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.899566    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlqld\" (UniqueName: \"kubernetes.io/projected/e1fa3e17-9d3f-4807-af16-05602a7fe693-kube-api-access-hlqld\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd\" (UID: \"e1fa3e17-9d3f-4807-af16-05602a7fe693\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"
Jul 06 20:57:10.959514 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:10.959473    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"
Jul 06 20:57:11.093251 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:11.093220    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"]
Jul 06 20:57:11.095614 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:57:11.095581    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1fa3e17_9d3f_4807_af16_05602a7fe693.slice/crio-2ef54bfe1c3d976438053fd8e470c88612ed1de49d1af9f1e76466679771447b WatchSource:0}: Error finding container 2ef54bfe1c3d976438053fd8e470c88612ed1de49d1af9f1e76466679771447b: Status 404 returned error can't find the container with id 2ef54bfe1c3d976438053fd8e470c88612ed1de49d1af9f1e76466679771447b
Jul 06 20:57:11.203496 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:11.203441    2566 generic.go:356] "Generic (PLEG): container finished" podID="e1fa3e17-9d3f-4807-af16-05602a7fe693" containerID="765cf40ec3939b74a6332cc05c38cea7a6d759e767c4294c46b2b908ab3b6a80" exitCode=0
Jul 06 20:57:11.203624 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:11.203490    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd" event={"ID":"e1fa3e17-9d3f-4807-af16-05602a7fe693","Type":"ContainerDied","Data":"765cf40ec3939b74a6332cc05c38cea7a6d759e767c4294c46b2b908ab3b6a80"}
Jul 06 20:57:11.203624 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:11.203544    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd" event={"ID":"e1fa3e17-9d3f-4807-af16-05602a7fe693","Type":"ContainerStarted","Data":"2ef54bfe1c3d976438053fd8e470c88612ed1de49d1af9f1e76466679771447b"}
Jul 06 20:57:14.220859 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:14.220814    2566 generic.go:356] "Generic (PLEG): container finished" podID="e1fa3e17-9d3f-4807-af16-05602a7fe693" containerID="ad82b8adaec17550e9d0f9a1bb4da11548d53f150aa5c2763987b1ee9f824418" exitCode=0
Jul 06 20:57:14.221249 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:14.220903    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd" event={"ID":"e1fa3e17-9d3f-4807-af16-05602a7fe693","Type":"ContainerDied","Data":"ad82b8adaec17550e9d0f9a1bb4da11548d53f150aa5c2763987b1ee9f824418"}
Jul 06 20:57:15.227723 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:15.227688    2566 generic.go:356] "Generic (PLEG): container finished" podID="e1fa3e17-9d3f-4807-af16-05602a7fe693" containerID="177814da5266e29f82ab9e27559343f7493b3e41eea1797de4bf14dce1793ddc" exitCode=0
Jul 06 20:57:15.228112 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:15.227737    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd" event={"ID":"e1fa3e17-9d3f-4807-af16-05602a7fe693","Type":"ContainerDied","Data":"177814da5266e29f82ab9e27559343f7493b3e41eea1797de4bf14dce1793ddc"}
Jul 06 20:57:16.369682 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:16.369579    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"
Jul 06 20:57:16.544206 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:16.544120    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlqld\" (UniqueName: \"kubernetes.io/projected/e1fa3e17-9d3f-4807-af16-05602a7fe693-kube-api-access-hlqld\") pod \"e1fa3e17-9d3f-4807-af16-05602a7fe693\" (UID: \"e1fa3e17-9d3f-4807-af16-05602a7fe693\") "
Jul 06 20:57:16.544206 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:16.544176    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1fa3e17-9d3f-4807-af16-05602a7fe693-bundle\") pod \"e1fa3e17-9d3f-4807-af16-05602a7fe693\" (UID: \"e1fa3e17-9d3f-4807-af16-05602a7fe693\") "
Jul 06 20:57:16.544206 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:16.544200    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1fa3e17-9d3f-4807-af16-05602a7fe693-util\") pod \"e1fa3e17-9d3f-4807-af16-05602a7fe693\" (UID: \"e1fa3e17-9d3f-4807-af16-05602a7fe693\") "
Jul 06 20:57:16.544614 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:16.544577    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1fa3e17-9d3f-4807-af16-05602a7fe693-bundle" (OuterVolumeSpecName: "bundle") pod "e1fa3e17-9d3f-4807-af16-05602a7fe693" (UID: "e1fa3e17-9d3f-4807-af16-05602a7fe693"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:57:16.546219 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:16.546190    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1fa3e17-9d3f-4807-af16-05602a7fe693-kube-api-access-hlqld" (OuterVolumeSpecName: "kube-api-access-hlqld") pod "e1fa3e17-9d3f-4807-af16-05602a7fe693" (UID: "e1fa3e17-9d3f-4807-af16-05602a7fe693"). InnerVolumeSpecName "kube-api-access-hlqld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:57:16.558944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:16.558915    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1fa3e17-9d3f-4807-af16-05602a7fe693-util" (OuterVolumeSpecName: "util") pod "e1fa3e17-9d3f-4807-af16-05602a7fe693" (UID: "e1fa3e17-9d3f-4807-af16-05602a7fe693"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:57:16.644746 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:16.644704    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlqld\" (UniqueName: \"kubernetes.io/projected/e1fa3e17-9d3f-4807-af16-05602a7fe693-kube-api-access-hlqld\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:16.644746 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:16.644736    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1fa3e17-9d3f-4807-af16-05602a7fe693-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:16.644746 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:16.644746    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1fa3e17-9d3f-4807-af16-05602a7fe693-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:17.238065 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:17.238028    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd" event={"ID":"e1fa3e17-9d3f-4807-af16-05602a7fe693","Type":"ContainerDied","Data":"2ef54bfe1c3d976438053fd8e470c88612ed1de49d1af9f1e76466679771447b"}
Jul 06 20:57:17.238235 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:17.238075    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfxvd"
Jul 06 20:57:17.238235 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:17.238079    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ef54bfe1c3d976438053fd8e470c88612ed1de49d1af9f1e76466679771447b"
Jul 06 20:57:19.627055 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.627022    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84f7f87c5-64vbr"]
Jul 06 20:57:19.627438 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.627328    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1fa3e17-9d3f-4807-af16-05602a7fe693" containerName="pull"
Jul 06 20:57:19.627438 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.627338    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1fa3e17-9d3f-4807-af16-05602a7fe693" containerName="pull"
Jul 06 20:57:19.627438 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.627353    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1fa3e17-9d3f-4807-af16-05602a7fe693" containerName="util"
Jul 06 20:57:19.627438 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.627360    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1fa3e17-9d3f-4807-af16-05602a7fe693" containerName="util"
Jul 06 20:57:19.627438 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.627371    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1fa3e17-9d3f-4807-af16-05602a7fe693" containerName="extract"
Jul 06 20:57:19.627438 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.627376    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1fa3e17-9d3f-4807-af16-05602a7fe693" containerName="extract"
Jul 06 20:57:19.627438 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.627422    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1fa3e17-9d3f-4807-af16-05602a7fe693" containerName="extract"
Jul 06 20:57:19.629236 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.629220    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.644243 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.644215    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84f7f87c5-64vbr"]
Jul 06 20:57:19.771103 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.771067    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c388ef5c-704f-4ce1-b290-62f7b1da4669-service-ca\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.771281 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.771115    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c388ef5c-704f-4ce1-b290-62f7b1da4669-trusted-ca-bundle\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.771281 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.771140    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c388ef5c-704f-4ce1-b290-62f7b1da4669-console-serving-cert\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.771281 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.771157    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v24kt\" (UniqueName: \"kubernetes.io/projected/c388ef5c-704f-4ce1-b290-62f7b1da4669-kube-api-access-v24kt\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.771281 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.771175    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c388ef5c-704f-4ce1-b290-62f7b1da4669-console-config\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.771281 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.771232    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c388ef5c-704f-4ce1-b290-62f7b1da4669-oauth-serving-cert\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.771281 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.771274    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c388ef5c-704f-4ce1-b290-62f7b1da4669-console-oauth-config\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.872439 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.872395    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c388ef5c-704f-4ce1-b290-62f7b1da4669-service-ca\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.872665 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.872483    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c388ef5c-704f-4ce1-b290-62f7b1da4669-trusted-ca-bundle\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.872665 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.872515    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c388ef5c-704f-4ce1-b290-62f7b1da4669-console-serving-cert\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.872665 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.872643    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v24kt\" (UniqueName: \"kubernetes.io/projected/c388ef5c-704f-4ce1-b290-62f7b1da4669-kube-api-access-v24kt\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.872843 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.872682    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c388ef5c-704f-4ce1-b290-62f7b1da4669-console-config\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.872843 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.872709    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c388ef5c-704f-4ce1-b290-62f7b1da4669-oauth-serving-cert\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.872843 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.872737    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c388ef5c-704f-4ce1-b290-62f7b1da4669-console-oauth-config\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.873351 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.873326    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c388ef5c-704f-4ce1-b290-62f7b1da4669-console-config\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.873477 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.873354    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c388ef5c-704f-4ce1-b290-62f7b1da4669-service-ca\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.873477 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.873371    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c388ef5c-704f-4ce1-b290-62f7b1da4669-trusted-ca-bundle\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.873606 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.873493    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c388ef5c-704f-4ce1-b290-62f7b1da4669-oauth-serving-cert\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.875117 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.875096    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c388ef5c-704f-4ce1-b290-62f7b1da4669-console-oauth-config\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.875215 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.875164    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c388ef5c-704f-4ce1-b290-62f7b1da4669-console-serving-cert\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.881107 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.881057    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v24kt\" (UniqueName: \"kubernetes.io/projected/c388ef5c-704f-4ce1-b290-62f7b1da4669-kube-api-access-v24kt\") pod \"console-84f7f87c5-64vbr\" (UID: \"c388ef5c-704f-4ce1-b290-62f7b1da4669\") " pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:19.939060 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:19.939020    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:20.077026 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:20.077000    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84f7f87c5-64vbr"]
Jul 06 20:57:20.079715 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:57:20.079680    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc388ef5c_704f_4ce1_b290_62f7b1da4669.slice/crio-32b6042f825b616f2cf32e7dd99f54dde9efaf6a5ade5726a2428a36fa8a959e WatchSource:0}: Error finding container 32b6042f825b616f2cf32e7dd99f54dde9efaf6a5ade5726a2428a36fa8a959e: Status 404 returned error can't find the container with id 32b6042f825b616f2cf32e7dd99f54dde9efaf6a5ade5726a2428a36fa8a959e
Jul 06 20:57:20.252571 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:20.252529    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f7f87c5-64vbr" event={"ID":"c388ef5c-704f-4ce1-b290-62f7b1da4669","Type":"ContainerStarted","Data":"d09ce65f878a9b54775c36a170a53a1ef4f3e6148194cf6bac98df99202a09cd"}
Jul 06 20:57:20.252761 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:20.252586    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f7f87c5-64vbr" event={"ID":"c388ef5c-704f-4ce1-b290-62f7b1da4669","Type":"ContainerStarted","Data":"32b6042f825b616f2cf32e7dd99f54dde9efaf6a5ade5726a2428a36fa8a959e"}
Jul 06 20:57:20.273646 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:20.273580    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84f7f87c5-64vbr" podStartSLOduration=1.273559673 podStartE2EDuration="1.273559673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:57:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:57:20.272971485 +0000 UTC m=+518.739212738" watchObservedRunningTime="2026-07-06 20:57:20.273559673 +0000 UTC m=+518.739800928"
Jul 06 20:57:29.939958 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:29.939917    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:29.940444 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:29.939983    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:29.945331 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:29.945305    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:30.303713 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:30.303683    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84f7f87c5-64vbr"
Jul 06 20:57:30.358159 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:30.358062    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d58bcc545-684v8"]
Jul 06 20:57:39.117644 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.117562    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"]
Jul 06 20:57:39.120750 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.120730    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.123257 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.123237    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 06 20:57:39.123383 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.123242    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 06 20:57:39.123383 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.123253    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 20:57:39.123518 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.123502    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 06 20:57:39.123899 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.123884    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-p5n2z\""
Jul 06 20:57:39.137286 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.137259    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"]
Jul 06 20:57:39.235793 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.235748    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.235984 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.235807    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.235984 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.235857    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.235984 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.235878    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.235984 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.235914    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.235984 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.235949    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwpkl\" (UniqueName: \"kubernetes.io/projected/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-kube-api-access-nwpkl\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.235984 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.235981    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.336570 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.336535    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.336743 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.336592    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwpkl\" (UniqueName: \"kubernetes.io/projected/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-kube-api-access-nwpkl\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.336743 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.336636    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.336743 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.336676    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.336743 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.336711    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.336947 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.336748    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.336947 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.336773    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.337427 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.337396    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.339211 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.339178    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.339329 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.339182    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.339486 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.339437    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.339575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.339499    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.344829 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.344797    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.344925 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.344806    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwpkl\" (UniqueName: \"kubernetes.io/projected/c5fe011c-ee58-46d2-ab73-1ca4d34aff1c-kube-api-access-nwpkl\") pod \"istiod-openshift-gateway-94bb8fbfd-x2st2\" (UID: \"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.430771 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.430665    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:39.567249 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:39.567223    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"]
Jul 06 20:57:39.569309 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:57:39.569281    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5fe011c_ee58_46d2_ab73_1ca4d34aff1c.slice/crio-9bb622396c00ce66f5da47ff5f85e55e6dd3c926b53dc60b61472cc69ba43f54 WatchSource:0}: Error finding container 9bb622396c00ce66f5da47ff5f85e55e6dd3c926b53dc60b61472cc69ba43f54: Status 404 returned error can't find the container with id 9bb622396c00ce66f5da47ff5f85e55e6dd3c926b53dc60b61472cc69ba43f54
Jul 06 20:57:40.346164 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:40.346118    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2" event={"ID":"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c","Type":"ContainerStarted","Data":"9bb622396c00ce66f5da47ff5f85e55e6dd3c926b53dc60b61472cc69ba43f54"}
Jul 06 20:57:42.084321 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:42.084279    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 20:57:42.084616 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:42.084352    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 20:57:42.360724 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:42.360627    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2" event={"ID":"c5fe011c-ee58-46d2-ab73-1ca4d34aff1c","Type":"ContainerStarted","Data":"1f52d9177229e1c34944941a53f06662273c36b81e73cf553721016e1b21458d"}
Jul 06 20:57:42.360724 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:42.360688    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:42.383316 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:42.383269    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2" podStartSLOduration=0.870616138 podStartE2EDuration="3.383252294s" totalImagesPullingTime="2.512636156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:57:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:57:42.380773884 +0000 UTC m=+540.847015141" watchObservedRunningTime="2026-07-06 20:57:42.383252294 +0000 UTC m=+540.849493548"
Jul 06 20:57:43.366753 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:43.366720    2566 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-x2st2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 20:57:43.367146 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:43.366770    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2" podUID="c5fe011c-ee58-46d2-ab73-1ca4d34aff1c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 20:57:46.365831 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:46.365801    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x2st2"
Jul 06 20:57:55.008508 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.008447    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"]
Jul 06 20:57:55.012283 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.012266    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"
Jul 06 20:57:55.015251 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.015231    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 20:57:55.015736 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.015720    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 20:57:55.015793 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.015752    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kl48c\""
Jul 06 20:57:55.020740 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.020716    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"]
Jul 06 20:57:55.109694 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.109657    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"]
Jul 06 20:57:55.113054 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.113035    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"
Jul 06 20:57:55.123837 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.123812    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"]
Jul 06 20:57:55.181725 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.181686    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffaf008b-3d37-42b3-9413-5be74de2fd76-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f\" (UID: \"ffaf008b-3d37-42b3-9413-5be74de2fd76\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"
Jul 06 20:57:55.181894 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.181733    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffaf008b-3d37-42b3-9413-5be74de2fd76-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f\" (UID: \"ffaf008b-3d37-42b3-9413-5be74de2fd76\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"
Jul 06 20:57:55.181894 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.181826    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zbmm\" (UniqueName: \"kubernetes.io/projected/ffaf008b-3d37-42b3-9413-5be74de2fd76-kube-api-access-6zbmm\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f\" (UID: \"ffaf008b-3d37-42b3-9413-5be74de2fd76\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"
Jul 06 20:57:55.231931 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.231892    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"]
Jul 06 20:57:55.235145 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.235128    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"
Jul 06 20:57:55.247209 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.247188    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"]
Jul 06 20:57:55.282925 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.282853    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffaf008b-3d37-42b3-9413-5be74de2fd76-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f\" (UID: \"ffaf008b-3d37-42b3-9413-5be74de2fd76\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"
Jul 06 20:57:55.282925 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.282889    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffaf008b-3d37-42b3-9413-5be74de2fd76-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f\" (UID: \"ffaf008b-3d37-42b3-9413-5be74de2fd76\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"
Jul 06 20:57:55.283114 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.282929    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de287466-8fe3-4f12-b7f0-e2994ce2d425-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl\" (UID: \"de287466-8fe3-4f12-b7f0-e2994ce2d425\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"
Jul 06 20:57:55.283114 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.282951    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhmz4\" (UniqueName: \"kubernetes.io/projected/de287466-8fe3-4f12-b7f0-e2994ce2d425-kube-api-access-nhmz4\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl\" (UID: \"de287466-8fe3-4f12-b7f0-e2994ce2d425\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"
Jul 06 20:57:55.283114 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.282978    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zbmm\" (UniqueName: \"kubernetes.io/projected/ffaf008b-3d37-42b3-9413-5be74de2fd76-kube-api-access-6zbmm\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f\" (UID: \"ffaf008b-3d37-42b3-9413-5be74de2fd76\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"
Jul 06 20:57:55.283114 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.283044    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de287466-8fe3-4f12-b7f0-e2994ce2d425-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl\" (UID: \"de287466-8fe3-4f12-b7f0-e2994ce2d425\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"
Jul 06 20:57:55.283322 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.283303    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffaf008b-3d37-42b3-9413-5be74de2fd76-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f\" (UID: \"ffaf008b-3d37-42b3-9413-5be74de2fd76\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"
Jul 06 20:57:55.283361 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.283313    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffaf008b-3d37-42b3-9413-5be74de2fd76-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f\" (UID: \"ffaf008b-3d37-42b3-9413-5be74de2fd76\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"
Jul 06 20:57:55.294867 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.294836    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zbmm\" (UniqueName: \"kubernetes.io/projected/ffaf008b-3d37-42b3-9413-5be74de2fd76-kube-api-access-6zbmm\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f\" (UID: \"ffaf008b-3d37-42b3-9413-5be74de2fd76\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"
Jul 06 20:57:55.315670 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.315637    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"]
Jul 06 20:57:55.319214 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.319192    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"
Jul 06 20:57:55.322110 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.322082    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"
Jul 06 20:57:55.329729 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.329707    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"]
Jul 06 20:57:55.383907 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.383875    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de287466-8fe3-4f12-b7f0-e2994ce2d425-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl\" (UID: \"de287466-8fe3-4f12-b7f0-e2994ce2d425\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"
Jul 06 20:57:55.384070 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.383926    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhmz4\" (UniqueName: \"kubernetes.io/projected/de287466-8fe3-4f12-b7f0-e2994ce2d425-kube-api-access-nhmz4\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl\" (UID: \"de287466-8fe3-4f12-b7f0-e2994ce2d425\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"
Jul 06 20:57:55.384070 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.383978    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e062c74-15ac-4f54-bd84-238844371913-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd\" (UID: \"3e062c74-15ac-4f54-bd84-238844371913\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"
Jul 06 20:57:55.384070 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.384020    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de287466-8fe3-4f12-b7f0-e2994ce2d425-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl\" (UID: \"de287466-8fe3-4f12-b7f0-e2994ce2d425\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"
Jul 06 20:57:55.384070 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.384056    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e062c74-15ac-4f54-bd84-238844371913-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd\" (UID: \"3e062c74-15ac-4f54-bd84-238844371913\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"
Jul 06 20:57:55.384286 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.384112    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm62p\" (UniqueName: \"kubernetes.io/projected/3e062c74-15ac-4f54-bd84-238844371913-kube-api-access-lm62p\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd\" (UID: \"3e062c74-15ac-4f54-bd84-238844371913\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"
Jul 06 20:57:55.384609 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.384586    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de287466-8fe3-4f12-b7f0-e2994ce2d425-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl\" (UID: \"de287466-8fe3-4f12-b7f0-e2994ce2d425\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"
Jul 06 20:57:55.384688 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.384643    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de287466-8fe3-4f12-b7f0-e2994ce2d425-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl\" (UID: \"de287466-8fe3-4f12-b7f0-e2994ce2d425\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"
Jul 06 20:57:55.389261 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.388993    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-d58bcc545-684v8" podUID="a95354bb-38cb-4186-bb3b-bb1669fd5a86" containerName="console" containerID="cri-o://d18b1154d7f2a7cf01356ea049e7aba2c53a99acb9117778c9b5172e2407bfd5" gracePeriod=15
Jul 06 20:57:55.394951 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.394929    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhmz4\" (UniqueName: \"kubernetes.io/projected/de287466-8fe3-4f12-b7f0-e2994ce2d425-kube-api-access-nhmz4\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl\" (UID: \"de287466-8fe3-4f12-b7f0-e2994ce2d425\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"
Jul 06 20:57:55.422571 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.422544    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"
Jul 06 20:57:55.460062 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.460037    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"]
Jul 06 20:57:55.462319 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:57:55.462294    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffaf008b_3d37_42b3_9413_5be74de2fd76.slice/crio-87fc35d0e82f169791d6a94732c1d000c25b5e8c37757f3ff15b27002885096e WatchSource:0}: Error finding container 87fc35d0e82f169791d6a94732c1d000c25b5e8c37757f3ff15b27002885096e: Status 404 returned error can't find the container with id 87fc35d0e82f169791d6a94732c1d000c25b5e8c37757f3ff15b27002885096e
Jul 06 20:57:55.485079 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.485046    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e062c74-15ac-4f54-bd84-238844371913-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd\" (UID: \"3e062c74-15ac-4f54-bd84-238844371913\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"
Jul 06 20:57:55.485235 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.485108    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e062c74-15ac-4f54-bd84-238844371913-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd\" (UID: \"3e062c74-15ac-4f54-bd84-238844371913\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"
Jul 06 20:57:55.485235 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.485135    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75flh\" (UniqueName: \"kubernetes.io/projected/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-kube-api-access-75flh\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp\" (UID: \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"
Jul 06 20:57:55.485235 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.485176    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm62p\" (UniqueName: \"kubernetes.io/projected/3e062c74-15ac-4f54-bd84-238844371913-kube-api-access-lm62p\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd\" (UID: \"3e062c74-15ac-4f54-bd84-238844371913\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"
Jul 06 20:57:55.485235 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.485200    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp\" (UID: \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"
Jul 06 20:57:55.485597 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.485289    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp\" (UID: \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"
Jul 06 20:57:55.485597 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.485533    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e062c74-15ac-4f54-bd84-238844371913-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd\" (UID: \"3e062c74-15ac-4f54-bd84-238844371913\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"
Jul 06 20:57:55.485597 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.485552    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e062c74-15ac-4f54-bd84-238844371913-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd\" (UID: \"3e062c74-15ac-4f54-bd84-238844371913\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"
Jul 06 20:57:55.496561 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.496538    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm62p\" (UniqueName: \"kubernetes.io/projected/3e062c74-15ac-4f54-bd84-238844371913-kube-api-access-lm62p\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd\" (UID: \"3e062c74-15ac-4f54-bd84-238844371913\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"
Jul 06 20:57:55.543580 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.543548    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"
Jul 06 20:57:55.568991 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.568890    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"]
Jul 06 20:57:55.573813 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:57:55.573765    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde287466_8fe3_4f12_b7f0_e2994ce2d425.slice/crio-5db517c0e81692e775bb918644f8bca7e75b2c5b5e3da8a28175b23bfeee2400 WatchSource:0}: Error finding container 5db517c0e81692e775bb918644f8bca7e75b2c5b5e3da8a28175b23bfeee2400: Status 404 returned error can't find the container with id 5db517c0e81692e775bb918644f8bca7e75b2c5b5e3da8a28175b23bfeee2400
Jul 06 20:57:55.587351 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.587056    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp\" (UID: \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"
Jul 06 20:57:55.587351 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.587114    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp\" (UID: \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"
Jul 06 20:57:55.587351 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.587228    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75flh\" (UniqueName: \"kubernetes.io/projected/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-kube-api-access-75flh\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp\" (UID: \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"
Jul 06 20:57:55.587718 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.587690    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp\" (UID: \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"
Jul 06 20:57:55.587816 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.587742    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp\" (UID: \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"
Jul 06 20:57:55.598551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.598486    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75flh\" (UniqueName: \"kubernetes.io/projected/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-kube-api-access-75flh\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp\" (UID: \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"
Jul 06 20:57:55.629594 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.629562    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"
Jul 06 20:57:55.679325 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.679284    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:57:55.699775 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.699744    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"]
Jul 06 20:57:55.704819 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:57:55.704780    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e062c74_15ac_4f54_bd84_238844371913.slice/crio-3178b8afc49a9df6530469b017d715d2637f651dcd3f1b244b1f135d692f73fc WatchSource:0}: Error finding container 3178b8afc49a9df6530469b017d715d2637f651dcd3f1b244b1f135d692f73fc: Status 404 returned error can't find the container with id 3178b8afc49a9df6530469b017d715d2637f651dcd3f1b244b1f135d692f73fc
Jul 06 20:57:55.781180 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.781142    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"]
Jul 06 20:57:55.789364 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.789341    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-oauth-config\") pod \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") "
Jul 06 20:57:55.789493 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.789404    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-serving-cert\") pod \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") "
Jul 06 20:57:55.789493 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.789429    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-config\") pod \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") "
Jul 06 20:57:55.789493 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.789474    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-service-ca\") pod \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") "
Jul 06 20:57:55.789668 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.789506    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjm9x\" (UniqueName: \"kubernetes.io/projected/a95354bb-38cb-4186-bb3b-bb1669fd5a86-kube-api-access-hjm9x\") pod \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") "
Jul 06 20:57:55.789668 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.789525    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-trusted-ca-bundle\") pod \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") "
Jul 06 20:57:55.789668 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.789544    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-oauth-serving-cert\") pod \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\" (UID: \"a95354bb-38cb-4186-bb3b-bb1669fd5a86\") "
Jul 06 20:57:55.789893 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.789862    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-config" (OuterVolumeSpecName: "console-config") pod "a95354bb-38cb-4186-bb3b-bb1669fd5a86" (UID: "a95354bb-38cb-4186-bb3b-bb1669fd5a86"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:57:55.789995 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.789974    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-service-ca" (OuterVolumeSpecName: "service-ca") pod "a95354bb-38cb-4186-bb3b-bb1669fd5a86" (UID: "a95354bb-38cb-4186-bb3b-bb1669fd5a86"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:57:55.790060 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.790026    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a95354bb-38cb-4186-bb3b-bb1669fd5a86" (UID: "a95354bb-38cb-4186-bb3b-bb1669fd5a86"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:57:55.790115 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.790056    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a95354bb-38cb-4186-bb3b-bb1669fd5a86" (UID: "a95354bb-38cb-4186-bb3b-bb1669fd5a86"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:57:55.791641 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.791583    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a95354bb-38cb-4186-bb3b-bb1669fd5a86" (UID: "a95354bb-38cb-4186-bb3b-bb1669fd5a86"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:57:55.791641 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.791622    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a95354bb-38cb-4186-bb3b-bb1669fd5a86" (UID: "a95354bb-38cb-4186-bb3b-bb1669fd5a86"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:57:55.791771 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.791664    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a95354bb-38cb-4186-bb3b-bb1669fd5a86-kube-api-access-hjm9x" (OuterVolumeSpecName: "kube-api-access-hjm9x") pod "a95354bb-38cb-4186-bb3b-bb1669fd5a86" (UID: "a95354bb-38cb-4186-bb3b-bb1669fd5a86"). InnerVolumeSpecName "kube-api-access-hjm9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:57:55.796655 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:57:55.796619    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb530006_ce16_4bf1_b2ea_67bfcf2aad3f.slice/crio-882b935a656615ec224bf46d89c5b6f1b31a4098f924980b4fd293296ca8cb35 WatchSource:0}: Error finding container 882b935a656615ec224bf46d89c5b6f1b31a4098f924980b4fd293296ca8cb35: Status 404 returned error can't find the container with id 882b935a656615ec224bf46d89c5b6f1b31a4098f924980b4fd293296ca8cb35
Jul 06 20:57:55.890726 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.890692    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:55.890726 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.890723    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:55.890726 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.890733    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-service-ca\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:55.890944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.890741    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjm9x\" (UniqueName: \"kubernetes.io/projected/a95354bb-38cb-4186-bb3b-bb1669fd5a86-kube-api-access-hjm9x\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:55.890944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.890752    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-trusted-ca-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:55.890944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.890760    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a95354bb-38cb-4186-bb3b-bb1669fd5a86-oauth-serving-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:55.890944 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:55.890768    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a95354bb-38cb-4186-bb3b-bb1669fd5a86-console-oauth-config\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:56.423222 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.423184    2566 generic.go:356] "Generic (PLEG): container finished" podID="a95354bb-38cb-4186-bb3b-bb1669fd5a86" containerID="d18b1154d7f2a7cf01356ea049e7aba2c53a99acb9117778c9b5172e2407bfd5" exitCode=0
Jul 06 20:57:56.423695 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.423230    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d58bcc545-684v8"
Jul 06 20:57:56.423695 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.423273    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d58bcc545-684v8" event={"ID":"a95354bb-38cb-4186-bb3b-bb1669fd5a86","Type":"ContainerDied","Data":"d18b1154d7f2a7cf01356ea049e7aba2c53a99acb9117778c9b5172e2407bfd5"}
Jul 06 20:57:56.423695 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.423323    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d58bcc545-684v8" event={"ID":"a95354bb-38cb-4186-bb3b-bb1669fd5a86","Type":"ContainerDied","Data":"ae788df24341821ca44a96b8fbcb3afad7d0238929a6a1427c6488e49b217ee6"}
Jul 06 20:57:56.423695 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.423344    2566 scope.go:117] "RemoveContainer" containerID="d18b1154d7f2a7cf01356ea049e7aba2c53a99acb9117778c9b5172e2407bfd5"
Jul 06 20:57:56.425040 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.425008    2566 generic.go:356] "Generic (PLEG): container finished" podID="ffaf008b-3d37-42b3-9413-5be74de2fd76" containerID="78f1d55555f17a781d8d734c77cd25afe52aa205bd2a3b04e202a7a1a2a8ee50" exitCode=0
Jul 06 20:57:56.425154 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.425101    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f" event={"ID":"ffaf008b-3d37-42b3-9413-5be74de2fd76","Type":"ContainerDied","Data":"78f1d55555f17a781d8d734c77cd25afe52aa205bd2a3b04e202a7a1a2a8ee50"}
Jul 06 20:57:56.425154 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.425147    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f" event={"ID":"ffaf008b-3d37-42b3-9413-5be74de2fd76","Type":"ContainerStarted","Data":"87fc35d0e82f169791d6a94732c1d000c25b5e8c37757f3ff15b27002885096e"}
Jul 06 20:57:56.426875 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.426849    2566 generic.go:356] "Generic (PLEG): container finished" podID="db530006-ce16-4bf1-b2ea-67bfcf2aad3f" containerID="3c3ab7d27b32181265873c624fe6eccbec9d5c009725dec0552e5cdd345214e3" exitCode=0
Jul 06 20:57:56.426971 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.426936    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp" event={"ID":"db530006-ce16-4bf1-b2ea-67bfcf2aad3f","Type":"ContainerDied","Data":"3c3ab7d27b32181265873c624fe6eccbec9d5c009725dec0552e5cdd345214e3"}
Jul 06 20:57:56.427040 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.426983    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp" event={"ID":"db530006-ce16-4bf1-b2ea-67bfcf2aad3f","Type":"ContainerStarted","Data":"882b935a656615ec224bf46d89c5b6f1b31a4098f924980b4fd293296ca8cb35"}
Jul 06 20:57:56.428786 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.428765    2566 generic.go:356] "Generic (PLEG): container finished" podID="3e062c74-15ac-4f54-bd84-238844371913" containerID="5f4ce7a9209bd43713665c2249e241e604679ee770340ed1c48e71ac7f15f161" exitCode=0
Jul 06 20:57:56.428881 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.428792    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd" event={"ID":"3e062c74-15ac-4f54-bd84-238844371913","Type":"ContainerDied","Data":"5f4ce7a9209bd43713665c2249e241e604679ee770340ed1c48e71ac7f15f161"}
Jul 06 20:57:56.428881 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.428833    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd" event={"ID":"3e062c74-15ac-4f54-bd84-238844371913","Type":"ContainerStarted","Data":"3178b8afc49a9df6530469b017d715d2637f651dcd3f1b244b1f135d692f73fc"}
Jul 06 20:57:56.430384 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.430361    2566 generic.go:356] "Generic (PLEG): container finished" podID="de287466-8fe3-4f12-b7f0-e2994ce2d425" containerID="6181ad444166bdbee10f5018ef2680f7e53dd8570f905f5fe79851e165442e82" exitCode=0
Jul 06 20:57:56.430513 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.430401    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl" event={"ID":"de287466-8fe3-4f12-b7f0-e2994ce2d425","Type":"ContainerDied","Data":"6181ad444166bdbee10f5018ef2680f7e53dd8570f905f5fe79851e165442e82"}
Jul 06 20:57:56.430513 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.430427    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl" event={"ID":"de287466-8fe3-4f12-b7f0-e2994ce2d425","Type":"ContainerStarted","Data":"5db517c0e81692e775bb918644f8bca7e75b2c5b5e3da8a28175b23bfeee2400"}
Jul 06 20:57:56.433255 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.433237    2566 scope.go:117] "RemoveContainer" containerID="d18b1154d7f2a7cf01356ea049e7aba2c53a99acb9117778c9b5172e2407bfd5"
Jul 06 20:57:56.433575 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:57:56.433521    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d18b1154d7f2a7cf01356ea049e7aba2c53a99acb9117778c9b5172e2407bfd5\": container with ID starting with d18b1154d7f2a7cf01356ea049e7aba2c53a99acb9117778c9b5172e2407bfd5 not found: ID does not exist" containerID="d18b1154d7f2a7cf01356ea049e7aba2c53a99acb9117778c9b5172e2407bfd5"
Jul 06 20:57:56.433575 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.433547    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d18b1154d7f2a7cf01356ea049e7aba2c53a99acb9117778c9b5172e2407bfd5"} err="failed to get container status \"d18b1154d7f2a7cf01356ea049e7aba2c53a99acb9117778c9b5172e2407bfd5\": rpc error: code = NotFound desc = could not find container \"d18b1154d7f2a7cf01356ea049e7aba2c53a99acb9117778c9b5172e2407bfd5\": container with ID starting with d18b1154d7f2a7cf01356ea049e7aba2c53a99acb9117778c9b5172e2407bfd5 not found: ID does not exist"
Jul 06 20:57:56.470115 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.470080    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d58bcc545-684v8"]
Jul 06 20:57:56.475747 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:56.475719    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d58bcc545-684v8"]
Jul 06 20:57:57.438499 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:57.438400    2566 generic.go:356] "Generic (PLEG): container finished" podID="ffaf008b-3d37-42b3-9413-5be74de2fd76" containerID="9b2e79c86e50dbf57aac92780d5e320f4f39d7f72cb416183dab545ad3341390" exitCode=0
Jul 06 20:57:57.438905 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:57.438488    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f" event={"ID":"ffaf008b-3d37-42b3-9413-5be74de2fd76","Type":"ContainerDied","Data":"9b2e79c86e50dbf57aac92780d5e320f4f39d7f72cb416183dab545ad3341390"}
Jul 06 20:57:57.440524 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:57.440502    2566 generic.go:356] "Generic (PLEG): container finished" podID="de287466-8fe3-4f12-b7f0-e2994ce2d425" containerID="56cc7336c800ca1d1bbb07342c7cf95e1e99634ee7d1671d9902d9a3a09a2084" exitCode=0
Jul 06 20:57:57.440613 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:57.440568    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl" event={"ID":"de287466-8fe3-4f12-b7f0-e2994ce2d425","Type":"ContainerDied","Data":"56cc7336c800ca1d1bbb07342c7cf95e1e99634ee7d1671d9902d9a3a09a2084"}
Jul 06 20:57:58.001629 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:58.001590    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a95354bb-38cb-4186-bb3b-bb1669fd5a86" path="/var/lib/kubelet/pods/a95354bb-38cb-4186-bb3b-bb1669fd5a86/volumes"
Jul 06 20:57:58.454745 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:58.454663    2566 generic.go:356] "Generic (PLEG): container finished" podID="db530006-ce16-4bf1-b2ea-67bfcf2aad3f" containerID="c96053026a996bb09d1506ac0680843ddea3eeb45110f1dad71c480b95e92779" exitCode=0
Jul 06 20:57:58.455176 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:58.454745    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp" event={"ID":"db530006-ce16-4bf1-b2ea-67bfcf2aad3f","Type":"ContainerDied","Data":"c96053026a996bb09d1506ac0680843ddea3eeb45110f1dad71c480b95e92779"}
Jul 06 20:57:58.456573 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:58.456552    2566 generic.go:356] "Generic (PLEG): container finished" podID="3e062c74-15ac-4f54-bd84-238844371913" containerID="fef28fd2f6ae6b494def7cec98584296e28fbf80094287d2e2922f964d41d803" exitCode=0
Jul 06 20:57:58.456667 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:58.456641    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd" event={"ID":"3e062c74-15ac-4f54-bd84-238844371913","Type":"ContainerDied","Data":"fef28fd2f6ae6b494def7cec98584296e28fbf80094287d2e2922f964d41d803"}
Jul 06 20:57:58.458873 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:58.458846    2566 generic.go:356] "Generic (PLEG): container finished" podID="de287466-8fe3-4f12-b7f0-e2994ce2d425" containerID="9e86e9d6e646fb385142000bfd8e5405edcfe8defeb2a7e3cb7336eef5844424" exitCode=0
Jul 06 20:57:58.458970 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:58.458918    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl" event={"ID":"de287466-8fe3-4f12-b7f0-e2994ce2d425","Type":"ContainerDied","Data":"9e86e9d6e646fb385142000bfd8e5405edcfe8defeb2a7e3cb7336eef5844424"}
Jul 06 20:57:58.461106 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:58.461083    2566 generic.go:356] "Generic (PLEG): container finished" podID="ffaf008b-3d37-42b3-9413-5be74de2fd76" containerID="838f4bcd9834b0a861039f87833b485c2414310e8c3f843f95e54d45e0725fa4" exitCode=0
Jul 06 20:57:58.461177 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:58.461114    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f" event={"ID":"ffaf008b-3d37-42b3-9413-5be74de2fd76","Type":"ContainerDied","Data":"838f4bcd9834b0a861039f87833b485c2414310e8c3f843f95e54d45e0725fa4"}
Jul 06 20:57:59.467842 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.467810    2566 generic.go:356] "Generic (PLEG): container finished" podID="db530006-ce16-4bf1-b2ea-67bfcf2aad3f" containerID="7bfc33e8a205a94ad71b058be15df3e5df89ba2acd951edcbb1cb87890db5a22" exitCode=0
Jul 06 20:57:59.468316 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.467845    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp" event={"ID":"db530006-ce16-4bf1-b2ea-67bfcf2aad3f","Type":"ContainerDied","Data":"7bfc33e8a205a94ad71b058be15df3e5df89ba2acd951edcbb1cb87890db5a22"}
Jul 06 20:57:59.469882 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.469863    2566 generic.go:356] "Generic (PLEG): container finished" podID="3e062c74-15ac-4f54-bd84-238844371913" containerID="dd9422722a467dcc1c832efed2e5b6763e66ab9d9db6f1323202913036c63527" exitCode=0
Jul 06 20:57:59.469995 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.469946    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd" event={"ID":"3e062c74-15ac-4f54-bd84-238844371913","Type":"ContainerDied","Data":"dd9422722a467dcc1c832efed2e5b6763e66ab9d9db6f1323202913036c63527"}
Jul 06 20:57:59.624760 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.624734    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"
Jul 06 20:57:59.627777 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.627756    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"
Jul 06 20:57:59.727967 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.727869    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffaf008b-3d37-42b3-9413-5be74de2fd76-bundle\") pod \"ffaf008b-3d37-42b3-9413-5be74de2fd76\" (UID: \"ffaf008b-3d37-42b3-9413-5be74de2fd76\") "
Jul 06 20:57:59.728143 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.727971    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zbmm\" (UniqueName: \"kubernetes.io/projected/ffaf008b-3d37-42b3-9413-5be74de2fd76-kube-api-access-6zbmm\") pod \"ffaf008b-3d37-42b3-9413-5be74de2fd76\" (UID: \"ffaf008b-3d37-42b3-9413-5be74de2fd76\") "
Jul 06 20:57:59.728143 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.728002    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhmz4\" (UniqueName: \"kubernetes.io/projected/de287466-8fe3-4f12-b7f0-e2994ce2d425-kube-api-access-nhmz4\") pod \"de287466-8fe3-4f12-b7f0-e2994ce2d425\" (UID: \"de287466-8fe3-4f12-b7f0-e2994ce2d425\") "
Jul 06 20:57:59.728143 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.728082    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de287466-8fe3-4f12-b7f0-e2994ce2d425-util\") pod \"de287466-8fe3-4f12-b7f0-e2994ce2d425\" (UID: \"de287466-8fe3-4f12-b7f0-e2994ce2d425\") "
Jul 06 20:57:59.728143 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.728140    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffaf008b-3d37-42b3-9413-5be74de2fd76-util\") pod \"ffaf008b-3d37-42b3-9413-5be74de2fd76\" (UID: \"ffaf008b-3d37-42b3-9413-5be74de2fd76\") "
Jul 06 20:57:59.728356 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.728175    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de287466-8fe3-4f12-b7f0-e2994ce2d425-bundle\") pod \"de287466-8fe3-4f12-b7f0-e2994ce2d425\" (UID: \"de287466-8fe3-4f12-b7f0-e2994ce2d425\") "
Jul 06 20:57:59.728752 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.728722    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de287466-8fe3-4f12-b7f0-e2994ce2d425-bundle" (OuterVolumeSpecName: "bundle") pod "de287466-8fe3-4f12-b7f0-e2994ce2d425" (UID: "de287466-8fe3-4f12-b7f0-e2994ce2d425"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:57:59.728886 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.728859    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffaf008b-3d37-42b3-9413-5be74de2fd76-bundle" (OuterVolumeSpecName: "bundle") pod "ffaf008b-3d37-42b3-9413-5be74de2fd76" (UID: "ffaf008b-3d37-42b3-9413-5be74de2fd76"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:57:59.730221 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.730193    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffaf008b-3d37-42b3-9413-5be74de2fd76-kube-api-access-6zbmm" (OuterVolumeSpecName: "kube-api-access-6zbmm") pod "ffaf008b-3d37-42b3-9413-5be74de2fd76" (UID: "ffaf008b-3d37-42b3-9413-5be74de2fd76"). InnerVolumeSpecName "kube-api-access-6zbmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:57:59.730303 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.730254    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de287466-8fe3-4f12-b7f0-e2994ce2d425-kube-api-access-nhmz4" (OuterVolumeSpecName: "kube-api-access-nhmz4") pod "de287466-8fe3-4f12-b7f0-e2994ce2d425" (UID: "de287466-8fe3-4f12-b7f0-e2994ce2d425"). InnerVolumeSpecName "kube-api-access-nhmz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:57:59.737045 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.737019    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffaf008b-3d37-42b3-9413-5be74de2fd76-util" (OuterVolumeSpecName: "util") pod "ffaf008b-3d37-42b3-9413-5be74de2fd76" (UID: "ffaf008b-3d37-42b3-9413-5be74de2fd76"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:57:59.737231 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.737215    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de287466-8fe3-4f12-b7f0-e2994ce2d425-util" (OuterVolumeSpecName: "util") pod "de287466-8fe3-4f12-b7f0-e2994ce2d425" (UID: "de287466-8fe3-4f12-b7f0-e2994ce2d425"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:57:59.829805 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.829767    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de287466-8fe3-4f12-b7f0-e2994ce2d425-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:59.829805 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.829801    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffaf008b-3d37-42b3-9413-5be74de2fd76-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:59.829805 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.829810    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zbmm\" (UniqueName: \"kubernetes.io/projected/ffaf008b-3d37-42b3-9413-5be74de2fd76-kube-api-access-6zbmm\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:59.829988 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.829820    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhmz4\" (UniqueName: \"kubernetes.io/projected/de287466-8fe3-4f12-b7f0-e2994ce2d425-kube-api-access-nhmz4\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:59.829988 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.829828    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de287466-8fe3-4f12-b7f0-e2994ce2d425-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:57:59.829988 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:57:59.829836    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffaf008b-3d37-42b3-9413-5be74de2fd76-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:58:00.476340 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.476261    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f"
Jul 06 20:58:00.476340 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.476292    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76ngn6f" event={"ID":"ffaf008b-3d37-42b3-9413-5be74de2fd76","Type":"ContainerDied","Data":"87fc35d0e82f169791d6a94732c1d000c25b5e8c37757f3ff15b27002885096e"}
Jul 06 20:58:00.476805 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.476343    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87fc35d0e82f169791d6a94732c1d000c25b5e8c37757f3ff15b27002885096e"
Jul 06 20:58:00.478514 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.478395    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl"
Jul 06 20:58:00.478514 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.478387    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efdb7zl" event={"ID":"de287466-8fe3-4f12-b7f0-e2994ce2d425","Type":"ContainerDied","Data":"5db517c0e81692e775bb918644f8bca7e75b2c5b5e3da8a28175b23bfeee2400"}
Jul 06 20:58:00.478514 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.478483    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5db517c0e81692e775bb918644f8bca7e75b2c5b5e3da8a28175b23bfeee2400"
Jul 06 20:58:00.595540 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.595511    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"
Jul 06 20:58:00.640754 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.640727    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"
Jul 06 20:58:00.737291 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.737259    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-bundle\") pod \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\" (UID: \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\") "
Jul 06 20:58:00.737502 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.737299    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-util\") pod \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\" (UID: \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\") "
Jul 06 20:58:00.737502 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.737325    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lm62p\" (UniqueName: \"kubernetes.io/projected/3e062c74-15ac-4f54-bd84-238844371913-kube-api-access-lm62p\") pod \"3e062c74-15ac-4f54-bd84-238844371913\" (UID: \"3e062c74-15ac-4f54-bd84-238844371913\") "
Jul 06 20:58:00.737502 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.737343    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75flh\" (UniqueName: \"kubernetes.io/projected/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-kube-api-access-75flh\") pod \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\" (UID: \"db530006-ce16-4bf1-b2ea-67bfcf2aad3f\") "
Jul 06 20:58:00.737502 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.737366    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e062c74-15ac-4f54-bd84-238844371913-util\") pod \"3e062c74-15ac-4f54-bd84-238844371913\" (UID: \"3e062c74-15ac-4f54-bd84-238844371913\") "
Jul 06 20:58:00.737502 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.737385    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e062c74-15ac-4f54-bd84-238844371913-bundle\") pod \"3e062c74-15ac-4f54-bd84-238844371913\" (UID: \"3e062c74-15ac-4f54-bd84-238844371913\") "
Jul 06 20:58:00.737972 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.737940    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e062c74-15ac-4f54-bd84-238844371913-bundle" (OuterVolumeSpecName: "bundle") pod "3e062c74-15ac-4f54-bd84-238844371913" (UID: "3e062c74-15ac-4f54-bd84-238844371913"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:58:00.738121 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.737991    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-bundle" (OuterVolumeSpecName: "bundle") pod "db530006-ce16-4bf1-b2ea-67bfcf2aad3f" (UID: "db530006-ce16-4bf1-b2ea-67bfcf2aad3f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:58:00.739606 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.739582    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e062c74-15ac-4f54-bd84-238844371913-kube-api-access-lm62p" (OuterVolumeSpecName: "kube-api-access-lm62p") pod "3e062c74-15ac-4f54-bd84-238844371913" (UID: "3e062c74-15ac-4f54-bd84-238844371913"). InnerVolumeSpecName "kube-api-access-lm62p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:58:00.739892 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.739862    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-kube-api-access-75flh" (OuterVolumeSpecName: "kube-api-access-75flh") pod "db530006-ce16-4bf1-b2ea-67bfcf2aad3f" (UID: "db530006-ce16-4bf1-b2ea-67bfcf2aad3f"). InnerVolumeSpecName "kube-api-access-75flh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:58:00.743337 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.743318    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e062c74-15ac-4f54-bd84-238844371913-util" (OuterVolumeSpecName: "util") pod "3e062c74-15ac-4f54-bd84-238844371913" (UID: "3e062c74-15ac-4f54-bd84-238844371913"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:58:00.743582 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.743562    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-util" (OuterVolumeSpecName: "util") pod "db530006-ce16-4bf1-b2ea-67bfcf2aad3f" (UID: "db530006-ce16-4bf1-b2ea-67bfcf2aad3f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:58:00.838802 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.838766    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lm62p\" (UniqueName: \"kubernetes.io/projected/3e062c74-15ac-4f54-bd84-238844371913-kube-api-access-lm62p\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:58:00.838802 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.838795    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75flh\" (UniqueName: \"kubernetes.io/projected/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-kube-api-access-75flh\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:58:00.838802 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.838806    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e062c74-15ac-4f54-bd84-238844371913-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:58:00.839024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.838816    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e062c74-15ac-4f54-bd84-238844371913-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:58:00.839024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.838823    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-bundle\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:58:00.839024 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:00.838831    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db530006-ce16-4bf1-b2ea-67bfcf2aad3f-util\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:58:01.485734 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:01.485703    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp"
Jul 06 20:58:01.486130 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:01.485700    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabb7mjp" event={"ID":"db530006-ce16-4bf1-b2ea-67bfcf2aad3f","Type":"ContainerDied","Data":"882b935a656615ec224bf46d89c5b6f1b31a4098f924980b4fd293296ca8cb35"}
Jul 06 20:58:01.486130 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:01.485855    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="882b935a656615ec224bf46d89c5b6f1b31a4098f924980b4fd293296ca8cb35"
Jul 06 20:58:01.487540 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:01.487516    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd" event={"ID":"3e062c74-15ac-4f54-bd84-238844371913","Type":"ContainerDied","Data":"3178b8afc49a9df6530469b017d715d2637f651dcd3f1b244b1f135d692f73fc"}
Jul 06 20:58:01.487645 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:01.487556    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3178b8afc49a9df6530469b017d715d2637f651dcd3f1b244b1f135d692f73fc"
Jul 06 20:58:01.487645 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:01.487556    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36592qd"
Jul 06 20:58:06.319188 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319155    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-l2xrg"]
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319472    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db530006-ce16-4bf1-b2ea-67bfcf2aad3f" containerName="extract"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319483    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="db530006-ce16-4bf1-b2ea-67bfcf2aad3f" containerName="extract"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319493    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db530006-ce16-4bf1-b2ea-67bfcf2aad3f" containerName="util"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319500    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="db530006-ce16-4bf1-b2ea-67bfcf2aad3f" containerName="util"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319505    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e062c74-15ac-4f54-bd84-238844371913" containerName="pull"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319510    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e062c74-15ac-4f54-bd84-238844371913" containerName="pull"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319524    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a95354bb-38cb-4186-bb3b-bb1669fd5a86" containerName="console"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319530    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a95354bb-38cb-4186-bb3b-bb1669fd5a86" containerName="console"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319537    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffaf008b-3d37-42b3-9413-5be74de2fd76" containerName="extract"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319542    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffaf008b-3d37-42b3-9413-5be74de2fd76" containerName="extract"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319550    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffaf008b-3d37-42b3-9413-5be74de2fd76" containerName="pull"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319555    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffaf008b-3d37-42b3-9413-5be74de2fd76" containerName="pull"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319563    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de287466-8fe3-4f12-b7f0-e2994ce2d425" containerName="extract"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319568    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="de287466-8fe3-4f12-b7f0-e2994ce2d425" containerName="extract"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319574    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e062c74-15ac-4f54-bd84-238844371913" containerName="extract"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319579    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e062c74-15ac-4f54-bd84-238844371913" containerName="extract"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319586    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de287466-8fe3-4f12-b7f0-e2994ce2d425" containerName="pull"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319591    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="de287466-8fe3-4f12-b7f0-e2994ce2d425" containerName="pull"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319597    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de287466-8fe3-4f12-b7f0-e2994ce2d425" containerName="util"
Jul 06 20:58:06.319593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319602    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="de287466-8fe3-4f12-b7f0-e2994ce2d425" containerName="util"
Jul 06 20:58:06.320181 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319608    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffaf008b-3d37-42b3-9413-5be74de2fd76" containerName="util"
Jul 06 20:58:06.320181 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319613    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffaf008b-3d37-42b3-9413-5be74de2fd76" containerName="util"
Jul 06 20:58:06.320181 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319618    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e062c74-15ac-4f54-bd84-238844371913" containerName="util"
Jul 06 20:58:06.320181 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319623    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e062c74-15ac-4f54-bd84-238844371913" containerName="util"
Jul 06 20:58:06.320181 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319630    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db530006-ce16-4bf1-b2ea-67bfcf2aad3f" containerName="pull"
Jul 06 20:58:06.320181 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319634    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="db530006-ce16-4bf1-b2ea-67bfcf2aad3f" containerName="pull"
Jul 06 20:58:06.320181 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319694    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e062c74-15ac-4f54-bd84-238844371913" containerName="extract"
Jul 06 20:58:06.320181 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319702    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a95354bb-38cb-4186-bb3b-bb1669fd5a86" containerName="console"
Jul 06 20:58:06.320181 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319708    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="db530006-ce16-4bf1-b2ea-67bfcf2aad3f" containerName="extract"
Jul 06 20:58:06.320181 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319714    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffaf008b-3d37-42b3-9413-5be74de2fd76" containerName="extract"
Jul 06 20:58:06.320181 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.319719    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="de287466-8fe3-4f12-b7f0-e2994ce2d425" containerName="extract"
Jul 06 20:58:06.322428 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.322412    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-l2xrg"
Jul 06 20:58:06.325325 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.325308    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-fvx2c\""
Jul 06 20:58:06.325450 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.325412    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 06 20:58:06.325450 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.325430    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 20:58:06.325450 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.325417    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 20:58:06.335878 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.335857    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-l2xrg"]
Jul 06 20:58:06.382656 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.382619    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-827lm\" (UniqueName: \"kubernetes.io/projected/b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb-kube-api-access-827lm\") pod \"dns-operator-controller-manager-65b49595d7-l2xrg\" (UID: \"b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-l2xrg"
Jul 06 20:58:06.485684 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.484150    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-827lm\" (UniqueName: \"kubernetes.io/projected/b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb-kube-api-access-827lm\") pod \"dns-operator-controller-manager-65b49595d7-l2xrg\" (UID: \"b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-l2xrg"
Jul 06 20:58:06.495941 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.495912    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-827lm\" (UniqueName: \"kubernetes.io/projected/b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb-kube-api-access-827lm\") pod \"dns-operator-controller-manager-65b49595d7-l2xrg\" (UID: \"b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-l2xrg"
Jul 06 20:58:06.632260 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.632168    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-l2xrg"
Jul 06 20:58:06.768935 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:06.768906    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-l2xrg"]
Jul 06 20:58:06.770919 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:58:06.770894    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5b1cf4b_6117_4fcd_b3f7_97ece99b88fb.slice/crio-b3518008d211034061d2a0e1604a83968fd694c8621042425a83d13523897018 WatchSource:0}: Error finding container b3518008d211034061d2a0e1604a83968fd694c8621042425a83d13523897018: Status 404 returned error can't find the container with id b3518008d211034061d2a0e1604a83968fd694c8621042425a83d13523897018
Jul 06 20:58:07.520977 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:07.520937    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-l2xrg" event={"ID":"b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb","Type":"ContainerStarted","Data":"b3518008d211034061d2a0e1604a83968fd694c8621042425a83d13523897018"}
Jul 06 20:58:08.527152 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:08.527116    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-l2xrg" event={"ID":"b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb","Type":"ContainerStarted","Data":"3bba6a224395ab64d05bd8d583d569334d4a4feafed436c1afc89a615dcf20e8"}
Jul 06 20:58:08.527531 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:08.527211    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-l2xrg"
Jul 06 20:58:08.585935 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:08.585881    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-l2xrg" podStartSLOduration=1.1658022080000001 podStartE2EDuration="2.585859458s" totalImagesPullingTime="1.42005725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:58:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:58:08.584053163 +0000 UTC m=+567.050294418" watchObservedRunningTime="2026-07-06 20:58:08.585859458 +0000 UTC m=+567.052100715"
Jul 06 20:58:09.238715 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:09.238681    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-6d85f6564-kv2wg"]
Jul 06 20:58:09.242137 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:09.242116    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6d85f6564-kv2wg"
Jul 06 20:58:09.245566 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:09.245549    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-ct8jh\""
Jul 06 20:58:09.257642 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:09.257618    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6d85f6564-kv2wg"]
Jul 06 20:58:09.311166 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:09.311125    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgffg\" (UniqueName: \"kubernetes.io/projected/06e719d0-5657-4579-b88e-7a9c83065efc-kube-api-access-zgffg\") pod \"authorino-operator-6d85f6564-kv2wg\" (UID: \"06e719d0-5657-4579-b88e-7a9c83065efc\") " pod="kuadrant-system/authorino-operator-6d85f6564-kv2wg"
Jul 06 20:58:09.411734 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:09.411688    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgffg\" (UniqueName: \"kubernetes.io/projected/06e719d0-5657-4579-b88e-7a9c83065efc-kube-api-access-zgffg\") pod \"authorino-operator-6d85f6564-kv2wg\" (UID: \"06e719d0-5657-4579-b88e-7a9c83065efc\") " pod="kuadrant-system/authorino-operator-6d85f6564-kv2wg"
Jul 06 20:58:09.430203 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:09.430154    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgffg\" (UniqueName: \"kubernetes.io/projected/06e719d0-5657-4579-b88e-7a9c83065efc-kube-api-access-zgffg\") pod \"authorino-operator-6d85f6564-kv2wg\" (UID: \"06e719d0-5657-4579-b88e-7a9c83065efc\") " pod="kuadrant-system/authorino-operator-6d85f6564-kv2wg"
Jul 06 20:58:09.552139 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:09.552045    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6d85f6564-kv2wg"
Jul 06 20:58:09.694357 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:09.694333    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6d85f6564-kv2wg"]
Jul 06 20:58:09.696771 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:58:09.696744    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e719d0_5657_4579_b88e_7a9c83065efc.slice/crio-c0e51d1246bbb8372d8e43dc875e901c22496546bb1c74a8c543bae29ba4bbcd WatchSource:0}: Error finding container c0e51d1246bbb8372d8e43dc875e901c22496546bb1c74a8c543bae29ba4bbcd: Status 404 returned error can't find the container with id c0e51d1246bbb8372d8e43dc875e901c22496546bb1c74a8c543bae29ba4bbcd
Jul 06 20:58:10.540850 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:10.540808    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6d85f6564-kv2wg" event={"ID":"06e719d0-5657-4579-b88e-7a9c83065efc","Type":"ContainerStarted","Data":"c0e51d1246bbb8372d8e43dc875e901c22496546bb1c74a8c543bae29ba4bbcd"}
Jul 06 20:58:12.555679 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:12.555644    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6d85f6564-kv2wg" event={"ID":"06e719d0-5657-4579-b88e-7a9c83065efc","Type":"ContainerStarted","Data":"911e21a07ac70505a6b648851c449126361e59a676eaa9bf5e71af71b831e6bf"}
Jul 06 20:58:12.556065 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:12.555775    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-6d85f6564-kv2wg"
Jul 06 20:58:12.582468 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:12.582415    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-6d85f6564-kv2wg" podStartSLOduration=1.182482595 podStartE2EDuration="3.582400143s" totalImagesPullingTime="2.399917548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:58:12.58212256 +0000 UTC m=+571.048363815" watchObservedRunningTime="2026-07-06 20:58:12.582400143 +0000 UTC m=+571.048641397"
Jul 06 20:58:13.869789 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:13.869748    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf"]
Jul 06 20:58:13.872035 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:13.872017    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf"
Jul 06 20:58:13.874445 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:13.874422    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2mr22\""
Jul 06 20:58:13.893814 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:13.893782    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf"]
Jul 06 20:58:13.953618 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:13.953584    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5221abb2-42b1-4dbe-abeb-0c1ed9d57d76-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf\" (UID: \"5221abb2-42b1-4dbe-abeb-0c1ed9d57d76\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf"
Jul 06 20:58:13.953781 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:13.953676    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc4d6\" (UniqueName: \"kubernetes.io/projected/5221abb2-42b1-4dbe-abeb-0c1ed9d57d76-kube-api-access-vc4d6\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf\" (UID: \"5221abb2-42b1-4dbe-abeb-0c1ed9d57d76\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf"
Jul 06 20:58:14.054070 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:14.054025    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc4d6\" (UniqueName: \"kubernetes.io/projected/5221abb2-42b1-4dbe-abeb-0c1ed9d57d76-kube-api-access-vc4d6\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf\" (UID: \"5221abb2-42b1-4dbe-abeb-0c1ed9d57d76\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf"
Jul 06 20:58:14.054247 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:14.054118    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5221abb2-42b1-4dbe-abeb-0c1ed9d57d76-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf\" (UID: \"5221abb2-42b1-4dbe-abeb-0c1ed9d57d76\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf"
Jul 06 20:58:14.054536 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:14.054519    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5221abb2-42b1-4dbe-abeb-0c1ed9d57d76-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf\" (UID: \"5221abb2-42b1-4dbe-abeb-0c1ed9d57d76\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf"
Jul 06 20:58:14.063739 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:14.063712    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc4d6\" (UniqueName: \"kubernetes.io/projected/5221abb2-42b1-4dbe-abeb-0c1ed9d57d76-kube-api-access-vc4d6\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf\" (UID: \"5221abb2-42b1-4dbe-abeb-0c1ed9d57d76\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf"
Jul 06 20:58:14.182392 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:14.182298    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf"
Jul 06 20:58:14.364652 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:14.364610    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf"]
Jul 06 20:58:14.365055 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:58:14.365033    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5221abb2_42b1_4dbe_abeb_0c1ed9d57d76.slice/crio-3498d9a31f68b24d77690e273498c1a34ea0e118dba7b872fd9dc7d1127a237c WatchSource:0}: Error finding container 3498d9a31f68b24d77690e273498c1a34ea0e118dba7b872fd9dc7d1127a237c: Status 404 returned error can't find the container with id 3498d9a31f68b24d77690e273498c1a34ea0e118dba7b872fd9dc7d1127a237c
Jul 06 20:58:14.566102 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:14.566067    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf" event={"ID":"5221abb2-42b1-4dbe-abeb-0c1ed9d57d76","Type":"ContainerStarted","Data":"3498d9a31f68b24d77690e273498c1a34ea0e118dba7b872fd9dc7d1127a237c"}
Jul 06 20:58:18.590017 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:18.589965    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf" event={"ID":"5221abb2-42b1-4dbe-abeb-0c1ed9d57d76","Type":"ContainerStarted","Data":"b40a5d19ae05357e88e0cbece22a3150eae0e6af4f5adc691c2a59e625fe8537"}
Jul 06 20:58:18.590513 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:18.590062    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf"
Jul 06 20:58:18.618240 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:18.618185    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf" podStartSLOduration=1.6099763999999999 podStartE2EDuration="5.618168837s" totalImagesPullingTime="4.008192437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:58:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:58:18.615769689 +0000 UTC m=+577.082010940" watchObservedRunningTime="2026-07-06 20:58:18.618168837 +0000 UTC m=+577.084410091"
Jul 06 20:58:19.534291 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:19.534259    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-l2xrg"
Jul 06 20:58:20.844981 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:20.844951    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"]
Jul 06 20:58:20.847520 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:20.847495    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"
Jul 06 20:58:20.850807 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:20.850790    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9zp7s\""
Jul 06 20:58:20.851008 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:20.850994    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 06 20:58:20.851090 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:20.851046    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 06 20:58:20.862528 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:20.862504    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"]
Jul 06 20:58:20.917076 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:20.917042    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/699c1536-62a3-4b51-a035-d30962929ec8-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-fq2x9\" (UID: \"699c1536-62a3-4b51-a035-d30962929ec8\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"
Jul 06 20:58:20.917251 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:20.917090    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/699c1536-62a3-4b51-a035-d30962929ec8-nginx-conf\") pod \"kuadrant-console-plugin-85fd4b8b9d-fq2x9\" (UID: \"699c1536-62a3-4b51-a035-d30962929ec8\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"
Jul 06 20:58:20.917251 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:20.917178    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99rbw\" (UniqueName: \"kubernetes.io/projected/699c1536-62a3-4b51-a035-d30962929ec8-kube-api-access-99rbw\") pod \"kuadrant-console-plugin-85fd4b8b9d-fq2x9\" (UID: \"699c1536-62a3-4b51-a035-d30962929ec8\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"
Jul 06 20:58:21.017665 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:21.017628    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99rbw\" (UniqueName: \"kubernetes.io/projected/699c1536-62a3-4b51-a035-d30962929ec8-kube-api-access-99rbw\") pod \"kuadrant-console-plugin-85fd4b8b9d-fq2x9\" (UID: \"699c1536-62a3-4b51-a035-d30962929ec8\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"
Jul 06 20:58:21.017842 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:21.017705    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/699c1536-62a3-4b51-a035-d30962929ec8-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-fq2x9\" (UID: \"699c1536-62a3-4b51-a035-d30962929ec8\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"
Jul 06 20:58:21.017842 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:21.017756    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/699c1536-62a3-4b51-a035-d30962929ec8-nginx-conf\") pod \"kuadrant-console-plugin-85fd4b8b9d-fq2x9\" (UID: \"699c1536-62a3-4b51-a035-d30962929ec8\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"
Jul 06 20:58:21.018468 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:21.018425    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/699c1536-62a3-4b51-a035-d30962929ec8-nginx-conf\") pod \"kuadrant-console-plugin-85fd4b8b9d-fq2x9\" (UID: \"699c1536-62a3-4b51-a035-d30962929ec8\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"
Jul 06 20:58:21.020179 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:21.020158    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/699c1536-62a3-4b51-a035-d30962929ec8-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-fq2x9\" (UID: \"699c1536-62a3-4b51-a035-d30962929ec8\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"
Jul 06 20:58:21.026981 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:21.026958    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99rbw\" (UniqueName: \"kubernetes.io/projected/699c1536-62a3-4b51-a035-d30962929ec8-kube-api-access-99rbw\") pod \"kuadrant-console-plugin-85fd4b8b9d-fq2x9\" (UID: \"699c1536-62a3-4b51-a035-d30962929ec8\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"
Jul 06 20:58:21.156593 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:21.156554    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"
Jul 06 20:58:21.288148 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:21.288124    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9"]
Jul 06 20:58:21.290012 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:58:21.289982    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod699c1536_62a3_4b51_a035_d30962929ec8.slice/crio-4385073d8378d075f17f9318d1fb0359e637c37a84dea5238dbdd56a078f4ff7 WatchSource:0}: Error finding container 4385073d8378d075f17f9318d1fb0359e637c37a84dea5238dbdd56a078f4ff7: Status 404 returned error can't find the container with id 4385073d8378d075f17f9318d1fb0359e637c37a84dea5238dbdd56a078f4ff7
Jul 06 20:58:21.605833 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:21.605738    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9" event={"ID":"699c1536-62a3-4b51-a035-d30962929ec8","Type":"ContainerStarted","Data":"4385073d8378d075f17f9318d1fb0359e637c37a84dea5238dbdd56a078f4ff7"}
Jul 06 20:58:23.563340 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:23.563305    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-6d85f6564-kv2wg"
Jul 06 20:58:26.638573 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:26.638537    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9" event={"ID":"699c1536-62a3-4b51-a035-d30962929ec8","Type":"ContainerStarted","Data":"2e86455fdd9da9c1de4eb1a5fd13483b4f7da811b0cf2db21ed4c40e03e76090"}
Jul 06 20:58:26.657811 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:26.657763    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-fq2x9" podStartSLOduration=1.799041569 podStartE2EDuration="6.657748422s" totalImagesPullingTime="4.858706853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:58:26.656124263 +0000 UTC m=+585.122365527" watchObservedRunningTime="2026-07-06 20:58:26.657748422 +0000 UTC m=+585.123989678"
Jul 06 20:58:29.597190 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:58:29.597156    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf"
Jul 06 20:59:05.611080 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:05.610994    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"]
Jul 06 20:59:05.629920 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:05.629891    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"]
Jul 06 20:59:05.630034 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:05.630023    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"
Jul 06 20:59:05.632745 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:05.632718    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 06 20:59:05.700225 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:05.700190    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"]
Jul 06 20:59:05.780341 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:05.780298    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxqwj\" (UniqueName: \"kubernetes.io/projected/73521454-7232-4122-9b9b-345c942d7ea0-kube-api-access-vxqwj\") pod \"limitador-limitador-ff68ff5cb-gf8sq\" (UID: \"73521454-7232-4122-9b9b-345c942d7ea0\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"
Jul 06 20:59:05.780538 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:05.780399    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/73521454-7232-4122-9b9b-345c942d7ea0-config-file\") pod \"limitador-limitador-ff68ff5cb-gf8sq\" (UID: \"73521454-7232-4122-9b9b-345c942d7ea0\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"
Jul 06 20:59:05.881727 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:05.881633    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxqwj\" (UniqueName: \"kubernetes.io/projected/73521454-7232-4122-9b9b-345c942d7ea0-kube-api-access-vxqwj\") pod \"limitador-limitador-ff68ff5cb-gf8sq\" (UID: \"73521454-7232-4122-9b9b-345c942d7ea0\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"
Jul 06 20:59:05.881727 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:05.881713    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/73521454-7232-4122-9b9b-345c942d7ea0-config-file\") pod \"limitador-limitador-ff68ff5cb-gf8sq\" (UID: \"73521454-7232-4122-9b9b-345c942d7ea0\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"
Jul 06 20:59:05.882361 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:05.882342    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/73521454-7232-4122-9b9b-345c942d7ea0-config-file\") pod \"limitador-limitador-ff68ff5cb-gf8sq\" (UID: \"73521454-7232-4122-9b9b-345c942d7ea0\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"
Jul 06 20:59:05.899503 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:05.899470    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxqwj\" (UniqueName: \"kubernetes.io/projected/73521454-7232-4122-9b9b-345c942d7ea0-kube-api-access-vxqwj\") pod \"limitador-limitador-ff68ff5cb-gf8sq\" (UID: \"73521454-7232-4122-9b9b-345c942d7ea0\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"
Jul 06 20:59:05.940827 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:05.940791    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"
Jul 06 20:59:06.097176 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:06.097153    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"]
Jul 06 20:59:06.099261 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:59:06.099229    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73521454_7232_4122_9b9b_345c942d7ea0.slice/crio-47604da3baf9a48743b89555af8d59c869d6abe75f7d79ac6ab57e979fbab4e6 WatchSource:0}: Error finding container 47604da3baf9a48743b89555af8d59c869d6abe75f7d79ac6ab57e979fbab4e6: Status 404 returned error can't find the container with id 47604da3baf9a48743b89555af8d59c869d6abe75f7d79ac6ab57e979fbab4e6
Jul 06 20:59:06.845624 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:06.845341    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq" event={"ID":"73521454-7232-4122-9b9b-345c942d7ea0","Type":"ContainerStarted","Data":"47604da3baf9a48743b89555af8d59c869d6abe75f7d79ac6ab57e979fbab4e6"}
Jul 06 20:59:09.863428 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:09.863385    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq" event={"ID":"73521454-7232-4122-9b9b-345c942d7ea0","Type":"ContainerStarted","Data":"e601a5b67a88c28738f49cf86d3c052d4fcfd13157aeba1e24d179bd0212d327"}
Jul 06 20:59:09.863842 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:09.863509    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"
Jul 06 20:59:09.898694 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:09.898634    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq" podStartSLOduration=1.256707765 podStartE2EDuration="4.898617607s" totalImagesPullingTime="3.641909842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:59:09.897806068 +0000 UTC m=+628.364047323" watchObservedRunningTime="2026-07-06 20:59:09.898617607 +0000 UTC m=+628.364858861"
Jul 06 20:59:20.870081 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:20.870052    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"
Jul 06 20:59:23.374551 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:23.374470    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"]
Jul 06 20:59:23.375288 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:23.374782    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq" podUID="73521454-7232-4122-9b9b-345c942d7ea0" containerName="limitador" containerID="cri-o://e601a5b67a88c28738f49cf86d3c052d4fcfd13157aeba1e24d179bd0212d327" gracePeriod=30
Jul 06 20:59:23.934023 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:23.933939    2566 generic.go:356] "Generic (PLEG): container finished" podID="73521454-7232-4122-9b9b-345c942d7ea0" containerID="e601a5b67a88c28738f49cf86d3c052d4fcfd13157aeba1e24d179bd0212d327" exitCode=0
Jul 06 20:59:23.934023 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:23.934013    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq" event={"ID":"73521454-7232-4122-9b9b-345c942d7ea0","Type":"ContainerDied","Data":"e601a5b67a88c28738f49cf86d3c052d4fcfd13157aeba1e24d179bd0212d327"}
Jul 06 20:59:24.314927 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:24.314899    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"
Jul 06 20:59:24.443104 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:24.443019    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/73521454-7232-4122-9b9b-345c942d7ea0-config-file\") pod \"73521454-7232-4122-9b9b-345c942d7ea0\" (UID: \"73521454-7232-4122-9b9b-345c942d7ea0\") "
Jul 06 20:59:24.443104 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:24.443074    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxqwj\" (UniqueName: \"kubernetes.io/projected/73521454-7232-4122-9b9b-345c942d7ea0-kube-api-access-vxqwj\") pod \"73521454-7232-4122-9b9b-345c942d7ea0\" (UID: \"73521454-7232-4122-9b9b-345c942d7ea0\") "
Jul 06 20:59:24.443637 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:24.443359    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73521454-7232-4122-9b9b-345c942d7ea0-config-file" (OuterVolumeSpecName: "config-file") pod "73521454-7232-4122-9b9b-345c942d7ea0" (UID: "73521454-7232-4122-9b9b-345c942d7ea0"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:59:24.445438 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:24.445412    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73521454-7232-4122-9b9b-345c942d7ea0-kube-api-access-vxqwj" (OuterVolumeSpecName: "kube-api-access-vxqwj") pod "73521454-7232-4122-9b9b-345c942d7ea0" (UID: "73521454-7232-4122-9b9b-345c942d7ea0"). InnerVolumeSpecName "kube-api-access-vxqwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:59:24.544312 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:24.544282    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxqwj\" (UniqueName: \"kubernetes.io/projected/73521454-7232-4122-9b9b-345c942d7ea0-kube-api-access-vxqwj\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:59:24.544312 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:24.544309    2566 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/73521454-7232-4122-9b9b-345c942d7ea0-config-file\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:59:24.940862 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:24.940827    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq" event={"ID":"73521454-7232-4122-9b9b-345c942d7ea0","Type":"ContainerDied","Data":"47604da3baf9a48743b89555af8d59c869d6abe75f7d79ac6ab57e979fbab4e6"}
Jul 06 20:59:24.940862 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:24.940870    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"
Jul 06 20:59:24.941114 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:24.940890    2566 scope.go:117] "RemoveContainer" containerID="e601a5b67a88c28738f49cf86d3c052d4fcfd13157aeba1e24d179bd0212d327"
Jul 06 20:59:24.962919 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:24.962867    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"]
Jul 06 20:59:24.965310 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:24.965284    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-gf8sq"]
Jul 06 20:59:26.002811 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:26.002776    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73521454-7232-4122-9b9b-345c942d7ea0" path="/var/lib/kubelet/pods/73521454-7232-4122-9b9b-345c942d7ea0/volumes"
Jul 06 20:59:40.175860 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.175822    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-664cc55bd7-fjmx5"]
Jul 06 20:59:40.176337 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.176037    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5" podUID="5587ef3c-daa5-45e7-80f2-95004041d4d3" containerName="manager" containerID="cri-o://49b465582f5c7a7c4ecd8160527a5ff0885d486e6fffa8b9435e741cd33eaf47" gracePeriod=10
Jul 06 20:59:40.217826 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.217790    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-664cc55bd7-rblpn"]
Jul 06 20:59:40.218233 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.218210    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73521454-7232-4122-9b9b-345c942d7ea0" containerName="limitador"
Jul 06 20:59:40.218233 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.218231    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="73521454-7232-4122-9b9b-345c942d7ea0" containerName="limitador"
Jul 06 20:59:40.218406 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.218309    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="73521454-7232-4122-9b9b-345c942d7ea0" containerName="limitador"
Jul 06 20:59:40.220231 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.220213    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-664cc55bd7-rblpn"
Jul 06 20:59:40.239260 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.239232    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-664cc55bd7-rblpn"]
Jul 06 20:59:40.374548 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.374507    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b77v\" (UniqueName: \"kubernetes.io/projected/0cf9ea48-0611-4266-acfb-65a78d0d63d0-kube-api-access-2b77v\") pod \"kserve-controller-manager-664cc55bd7-rblpn\" (UID: \"0cf9ea48-0611-4266-acfb-65a78d0d63d0\") " pod="kserve/kserve-controller-manager-664cc55bd7-rblpn"
Jul 06 20:59:40.374736 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.374679    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cf9ea48-0611-4266-acfb-65a78d0d63d0-cert\") pod \"kserve-controller-manager-664cc55bd7-rblpn\" (UID: \"0cf9ea48-0611-4266-acfb-65a78d0d63d0\") " pod="kserve/kserve-controller-manager-664cc55bd7-rblpn"
Jul 06 20:59:40.425695 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.425670    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5"
Jul 06 20:59:40.475507 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.475376    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cf9ea48-0611-4266-acfb-65a78d0d63d0-cert\") pod \"kserve-controller-manager-664cc55bd7-rblpn\" (UID: \"0cf9ea48-0611-4266-acfb-65a78d0d63d0\") " pod="kserve/kserve-controller-manager-664cc55bd7-rblpn"
Jul 06 20:59:40.475507 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.475490    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b77v\" (UniqueName: \"kubernetes.io/projected/0cf9ea48-0611-4266-acfb-65a78d0d63d0-kube-api-access-2b77v\") pod \"kserve-controller-manager-664cc55bd7-rblpn\" (UID: \"0cf9ea48-0611-4266-acfb-65a78d0d63d0\") " pod="kserve/kserve-controller-manager-664cc55bd7-rblpn"
Jul 06 20:59:40.478266 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.478239    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cf9ea48-0611-4266-acfb-65a78d0d63d0-cert\") pod \"kserve-controller-manager-664cc55bd7-rblpn\" (UID: \"0cf9ea48-0611-4266-acfb-65a78d0d63d0\") " pod="kserve/kserve-controller-manager-664cc55bd7-rblpn"
Jul 06 20:59:40.484832 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.484806    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b77v\" (UniqueName: \"kubernetes.io/projected/0cf9ea48-0611-4266-acfb-65a78d0d63d0-kube-api-access-2b77v\") pod \"kserve-controller-manager-664cc55bd7-rblpn\" (UID: \"0cf9ea48-0611-4266-acfb-65a78d0d63d0\") " pod="kserve/kserve-controller-manager-664cc55bd7-rblpn"
Jul 06 20:59:40.575893 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.575852    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-664cc55bd7-rblpn"
Jul 06 20:59:40.576089 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.575947    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pw4s\" (UniqueName: \"kubernetes.io/projected/5587ef3c-daa5-45e7-80f2-95004041d4d3-kube-api-access-7pw4s\") pod \"5587ef3c-daa5-45e7-80f2-95004041d4d3\" (UID: \"5587ef3c-daa5-45e7-80f2-95004041d4d3\") "
Jul 06 20:59:40.576089 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.576010    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5587ef3c-daa5-45e7-80f2-95004041d4d3-cert\") pod \"5587ef3c-daa5-45e7-80f2-95004041d4d3\" (UID: \"5587ef3c-daa5-45e7-80f2-95004041d4d3\") "
Jul 06 20:59:40.578371 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.578219    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5587ef3c-daa5-45e7-80f2-95004041d4d3-cert" (OuterVolumeSpecName: "cert") pod "5587ef3c-daa5-45e7-80f2-95004041d4d3" (UID: "5587ef3c-daa5-45e7-80f2-95004041d4d3"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:59:40.578371 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.578249    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5587ef3c-daa5-45e7-80f2-95004041d4d3-kube-api-access-7pw4s" (OuterVolumeSpecName: "kube-api-access-7pw4s") pod "5587ef3c-daa5-45e7-80f2-95004041d4d3" (UID: "5587ef3c-daa5-45e7-80f2-95004041d4d3"). InnerVolumeSpecName "kube-api-access-7pw4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:59:40.677022 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.676990    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pw4s\" (UniqueName: \"kubernetes.io/projected/5587ef3c-daa5-45e7-80f2-95004041d4d3-kube-api-access-7pw4s\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:59:40.677022 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.677020    2566 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5587ef3c-daa5-45e7-80f2-95004041d4d3-cert\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 20:59:40.707330 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.707306    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-664cc55bd7-rblpn"]
Jul 06 20:59:40.709743 ip-10-0-131-165 kubenswrapper[2566]: W0706 20:59:40.709716    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf9ea48_0611_4266_acfb_65a78d0d63d0.slice/crio-4271af3a5206bc10bcad3fe4d20046fe520de13d807653d0b41741c93c9788fe WatchSource:0}: Error finding container 4271af3a5206bc10bcad3fe4d20046fe520de13d807653d0b41741c93c9788fe: Status 404 returned error can't find the container with id 4271af3a5206bc10bcad3fe4d20046fe520de13d807653d0b41741c93c9788fe
Jul 06 20:59:40.711059 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:40.711042    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:59:41.022937 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:41.022896    2566 generic.go:356] "Generic (PLEG): container finished" podID="5587ef3c-daa5-45e7-80f2-95004041d4d3" containerID="49b465582f5c7a7c4ecd8160527a5ff0885d486e6fffa8b9435e741cd33eaf47" exitCode=0
Jul 06 20:59:41.023176 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:41.022947    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5"
Jul 06 20:59:41.023176 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:41.022979    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5" event={"ID":"5587ef3c-daa5-45e7-80f2-95004041d4d3","Type":"ContainerDied","Data":"49b465582f5c7a7c4ecd8160527a5ff0885d486e6fffa8b9435e741cd33eaf47"}
Jul 06 20:59:41.023176 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:41.023039    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-664cc55bd7-fjmx5" event={"ID":"5587ef3c-daa5-45e7-80f2-95004041d4d3","Type":"ContainerDied","Data":"911b6d2953d2888e151d93acdbea1cd2bb4bea19000577210ec5f4dca1c90f1a"}
Jul 06 20:59:41.023176 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:41.023071    2566 scope.go:117] "RemoveContainer" containerID="49b465582f5c7a7c4ecd8160527a5ff0885d486e6fffa8b9435e741cd33eaf47"
Jul 06 20:59:41.024342 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:41.024318    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-664cc55bd7-rblpn" event={"ID":"0cf9ea48-0611-4266-acfb-65a78d0d63d0","Type":"ContainerStarted","Data":"4271af3a5206bc10bcad3fe4d20046fe520de13d807653d0b41741c93c9788fe"}
Jul 06 20:59:41.031929 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:41.031911    2566 scope.go:117] "RemoveContainer" containerID="49b465582f5c7a7c4ecd8160527a5ff0885d486e6fffa8b9435e741cd33eaf47"
Jul 06 20:59:41.032157 ip-10-0-131-165 kubenswrapper[2566]: E0706 20:59:41.032140    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49b465582f5c7a7c4ecd8160527a5ff0885d486e6fffa8b9435e741cd33eaf47\": container with ID starting with 49b465582f5c7a7c4ecd8160527a5ff0885d486e6fffa8b9435e741cd33eaf47 not found: ID does not exist" containerID="49b465582f5c7a7c4ecd8160527a5ff0885d486e6fffa8b9435e741cd33eaf47"
Jul 06 20:59:41.032211 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:41.032165    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49b465582f5c7a7c4ecd8160527a5ff0885d486e6fffa8b9435e741cd33eaf47"} err="failed to get container status \"49b465582f5c7a7c4ecd8160527a5ff0885d486e6fffa8b9435e741cd33eaf47\": rpc error: code = NotFound desc = could not find container \"49b465582f5c7a7c4ecd8160527a5ff0885d486e6fffa8b9435e741cd33eaf47\": container with ID starting with 49b465582f5c7a7c4ecd8160527a5ff0885d486e6fffa8b9435e741cd33eaf47 not found: ID does not exist"
Jul 06 20:59:41.046544 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:41.046524    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-664cc55bd7-fjmx5"]
Jul 06 20:59:41.050448 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:41.050423    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-664cc55bd7-fjmx5"]
Jul 06 20:59:42.002409 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:42.002374    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5587ef3c-daa5-45e7-80f2-95004041d4d3" path="/var/lib/kubelet/pods/5587ef3c-daa5-45e7-80f2-95004041d4d3/volumes"
Jul 06 20:59:42.031241 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:42.031207    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-664cc55bd7-rblpn" event={"ID":"0cf9ea48-0611-4266-acfb-65a78d0d63d0","Type":"ContainerStarted","Data":"180c6d99b50fba631cbe54d632bdf3d1d484e6c7bb18c67d749aeb5d4ab69738"}
Jul 06 20:59:42.031397 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:42.031333    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-664cc55bd7-rblpn"
Jul 06 20:59:42.053605 ip-10-0-131-165 kubenswrapper[2566]: I0706 20:59:42.053557    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-664cc55bd7-rblpn" podStartSLOduration=1.679653399 podStartE2EDuration="2.053542845s" totalImagesPullingTime="373.889446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:59:42.051971201 +0000 UTC m=+660.518212455" watchObservedRunningTime="2026-07-06 20:59:42.053542845 +0000 UTC m=+660.519784098"
Jul 06 21:00:13.042652 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:13.042621    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-664cc55bd7-rblpn"
Jul 06 21:00:14.372105 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.372064    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-kwjqs"]
Jul 06 21:00:14.372540 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.372467    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5587ef3c-daa5-45e7-80f2-95004041d4d3" containerName="manager"
Jul 06 21:00:14.372540 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.372485    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5587ef3c-daa5-45e7-80f2-95004041d4d3" containerName="manager"
Jul 06 21:00:14.372641 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.372563    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5587ef3c-daa5-45e7-80f2-95004041d4d3" containerName="manager"
Jul 06 21:00:14.375666 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.375637    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-kwjqs"
Jul 06 21:00:14.378919 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.378892    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-7vtnv"]
Jul 06 21:00:14.382480 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.382080    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-gqfgb\""
Jul 06 21:00:14.382826 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.382807    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 06 21:00:14.383111 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.382834    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-7vtnv"
Jul 06 21:00:14.386481 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.386128    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Jul 06 21:00:14.387719 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.387697    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-k2hsf\""
Jul 06 21:00:14.401582 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.401556    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-kwjqs"]
Jul 06 21:00:14.404549 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.404522    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-7vtnv"]
Jul 06 21:00:14.446787 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.446758    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78h74\" (UniqueName: \"kubernetes.io/projected/9d31ef97-bdf0-4662-88e0-1e00f66ba388-kube-api-access-78h74\") pod \"odh-model-controller-84b54b7f97-7vtnv\" (UID: \"9d31ef97-bdf0-4662-88e0-1e00f66ba388\") " pod="kserve/odh-model-controller-84b54b7f97-7vtnv"
Jul 06 21:00:14.446978 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.446808    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nftk\" (UniqueName: \"kubernetes.io/projected/f6297f73-cf52-48ae-aeb7-959e9a06bf33-kube-api-access-8nftk\") pod \"model-serving-api-fd65d7d6-kwjqs\" (UID: \"f6297f73-cf52-48ae-aeb7-959e9a06bf33\") " pod="kserve/model-serving-api-fd65d7d6-kwjqs"
Jul 06 21:00:14.446978 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.446874    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d31ef97-bdf0-4662-88e0-1e00f66ba388-cert\") pod \"odh-model-controller-84b54b7f97-7vtnv\" (UID: \"9d31ef97-bdf0-4662-88e0-1e00f66ba388\") " pod="kserve/odh-model-controller-84b54b7f97-7vtnv"
Jul 06 21:00:14.446978 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.446910    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6297f73-cf52-48ae-aeb7-959e9a06bf33-tls-certs\") pod \"model-serving-api-fd65d7d6-kwjqs\" (UID: \"f6297f73-cf52-48ae-aeb7-959e9a06bf33\") " pod="kserve/model-serving-api-fd65d7d6-kwjqs"
Jul 06 21:00:14.547816 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.547781    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d31ef97-bdf0-4662-88e0-1e00f66ba388-cert\") pod \"odh-model-controller-84b54b7f97-7vtnv\" (UID: \"9d31ef97-bdf0-4662-88e0-1e00f66ba388\") " pod="kserve/odh-model-controller-84b54b7f97-7vtnv"
Jul 06 21:00:14.547984 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.547824    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6297f73-cf52-48ae-aeb7-959e9a06bf33-tls-certs\") pod \"model-serving-api-fd65d7d6-kwjqs\" (UID: \"f6297f73-cf52-48ae-aeb7-959e9a06bf33\") " pod="kserve/model-serving-api-fd65d7d6-kwjqs"
Jul 06 21:00:14.547984 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.547850    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78h74\" (UniqueName: \"kubernetes.io/projected/9d31ef97-bdf0-4662-88e0-1e00f66ba388-kube-api-access-78h74\") pod \"odh-model-controller-84b54b7f97-7vtnv\" (UID: \"9d31ef97-bdf0-4662-88e0-1e00f66ba388\") " pod="kserve/odh-model-controller-84b54b7f97-7vtnv"
Jul 06 21:00:14.547984 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.547896    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nftk\" (UniqueName: \"kubernetes.io/projected/f6297f73-cf52-48ae-aeb7-959e9a06bf33-kube-api-access-8nftk\") pod \"model-serving-api-fd65d7d6-kwjqs\" (UID: \"f6297f73-cf52-48ae-aeb7-959e9a06bf33\") " pod="kserve/model-serving-api-fd65d7d6-kwjqs"
Jul 06 21:00:14.547984 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:00:14.547946    2566 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 06 21:00:14.548125 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:00:14.548025    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d31ef97-bdf0-4662-88e0-1e00f66ba388-cert podName:9d31ef97-bdf0-4662-88e0-1e00f66ba388 nodeName:}" failed. No retries permitted until 2026-07-06 21:00:15.048008507 +0000 UTC m=+693.514249739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9d31ef97-bdf0-4662-88e0-1e00f66ba388-cert") pod "odh-model-controller-84b54b7f97-7vtnv" (UID: "9d31ef97-bdf0-4662-88e0-1e00f66ba388") : secret "odh-model-controller-webhook-cert" not found
Jul 06 21:00:14.548125 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:00:14.547946    2566 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 06 21:00:14.548125 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:00:14.548088    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6297f73-cf52-48ae-aeb7-959e9a06bf33-tls-certs podName:f6297f73-cf52-48ae-aeb7-959e9a06bf33 nodeName:}" failed. No retries permitted until 2026-07-06 21:00:15.048074195 +0000 UTC m=+693.514315436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f6297f73-cf52-48ae-aeb7-959e9a06bf33-tls-certs") pod "model-serving-api-fd65d7d6-kwjqs" (UID: "f6297f73-cf52-48ae-aeb7-959e9a06bf33") : secret "model-serving-api-tls" not found
Jul 06 21:00:14.558391 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.558361    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78h74\" (UniqueName: \"kubernetes.io/projected/9d31ef97-bdf0-4662-88e0-1e00f66ba388-kube-api-access-78h74\") pod \"odh-model-controller-84b54b7f97-7vtnv\" (UID: \"9d31ef97-bdf0-4662-88e0-1e00f66ba388\") " pod="kserve/odh-model-controller-84b54b7f97-7vtnv"
Jul 06 21:00:14.567348 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:14.567325    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nftk\" (UniqueName: \"kubernetes.io/projected/f6297f73-cf52-48ae-aeb7-959e9a06bf33-kube-api-access-8nftk\") pod \"model-serving-api-fd65d7d6-kwjqs\" (UID: \"f6297f73-cf52-48ae-aeb7-959e9a06bf33\") " pod="kserve/model-serving-api-fd65d7d6-kwjqs"
Jul 06 21:00:15.052751 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:15.052700    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d31ef97-bdf0-4662-88e0-1e00f66ba388-cert\") pod \"odh-model-controller-84b54b7f97-7vtnv\" (UID: \"9d31ef97-bdf0-4662-88e0-1e00f66ba388\") " pod="kserve/odh-model-controller-84b54b7f97-7vtnv"
Jul 06 21:00:15.052751 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:15.052750    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6297f73-cf52-48ae-aeb7-959e9a06bf33-tls-certs\") pod \"model-serving-api-fd65d7d6-kwjqs\" (UID: \"f6297f73-cf52-48ae-aeb7-959e9a06bf33\") " pod="kserve/model-serving-api-fd65d7d6-kwjqs"
Jul 06 21:00:15.055158 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:15.055131    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6297f73-cf52-48ae-aeb7-959e9a06bf33-tls-certs\") pod \"model-serving-api-fd65d7d6-kwjqs\" (UID: \"f6297f73-cf52-48ae-aeb7-959e9a06bf33\") " pod="kserve/model-serving-api-fd65d7d6-kwjqs"
Jul 06 21:00:15.055273 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:15.055178    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d31ef97-bdf0-4662-88e0-1e00f66ba388-cert\") pod \"odh-model-controller-84b54b7f97-7vtnv\" (UID: \"9d31ef97-bdf0-4662-88e0-1e00f66ba388\") " pod="kserve/odh-model-controller-84b54b7f97-7vtnv"
Jul 06 21:00:15.294391 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:15.294351    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-kwjqs"
Jul 06 21:00:15.302088 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:15.302052    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-7vtnv"
Jul 06 21:00:15.448014 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:15.447987    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-kwjqs"]
Jul 06 21:00:15.450550 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:00:15.450510    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6297f73_cf52_48ae_aeb7_959e9a06bf33.slice/crio-9cb5b222ac9edcdd13965eed1b21780446af9018a290f877a91eb49feeccc4f1 WatchSource:0}: Error finding container 9cb5b222ac9edcdd13965eed1b21780446af9018a290f877a91eb49feeccc4f1: Status 404 returned error can't find the container with id 9cb5b222ac9edcdd13965eed1b21780446af9018a290f877a91eb49feeccc4f1
Jul 06 21:00:15.467135 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:15.467112    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-7vtnv"]
Jul 06 21:00:15.468884 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:00:15.468858    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d31ef97_bdf0_4662_88e0_1e00f66ba388.slice/crio-cda030863211ff6b73c5e571bc0870955abd44e9ec2213f5951a381b0b1dcdfb WatchSource:0}: Error finding container cda030863211ff6b73c5e571bc0870955abd44e9ec2213f5951a381b0b1dcdfb: Status 404 returned error can't find the container with id cda030863211ff6b73c5e571bc0870955abd44e9ec2213f5951a381b0b1dcdfb
Jul 06 21:00:16.204973 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:16.204929    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-kwjqs" event={"ID":"f6297f73-cf52-48ae-aeb7-959e9a06bf33","Type":"ContainerStarted","Data":"9cb5b222ac9edcdd13965eed1b21780446af9018a290f877a91eb49feeccc4f1"}
Jul 06 21:00:16.206376 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:16.206331    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-7vtnv" event={"ID":"9d31ef97-bdf0-4662-88e0-1e00f66ba388","Type":"ContainerStarted","Data":"cda030863211ff6b73c5e571bc0870955abd44e9ec2213f5951a381b0b1dcdfb"}
Jul 06 21:00:19.223424 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:19.223387    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-kwjqs" event={"ID":"f6297f73-cf52-48ae-aeb7-959e9a06bf33","Type":"ContainerStarted","Data":"17ea960ab30d7ea06a6711e6ec9c5f1730cdcb7faa8af0296acf268bf3a67355"}
Jul 06 21:00:19.223901 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:19.223511    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-kwjqs"
Jul 06 21:00:19.225040 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:19.225010    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-7vtnv" event={"ID":"9d31ef97-bdf0-4662-88e0-1e00f66ba388","Type":"ContainerStarted","Data":"642f7f13322eedf150c535fd28d4e2027909f98f65b5b2c5e0daa6438fac68f1"}
Jul 06 21:00:19.225160 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:19.225102    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-7vtnv"
Jul 06 21:00:19.252494 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:19.252428    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-kwjqs" podStartSLOduration=2.285896771 podStartE2EDuration="5.252413497s" totalImagesPullingTime="2.966516726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:00:19.250162154 +0000 UTC m=+697.716403420" watchObservedRunningTime="2026-07-06 21:00:19.252413497 +0000 UTC m=+697.718654751"
Jul 06 21:00:19.279798 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:19.279744    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-7vtnv" podStartSLOduration=2.275488885 podStartE2EDuration="5.279726516s" totalImagesPullingTime="3.004237631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:00:19.27817609 +0000 UTC m=+697.744417344" watchObservedRunningTime="2026-07-06 21:00:19.279726516 +0000 UTC m=+697.745967795"
Jul 06 21:00:30.231422 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:30.231391    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-7vtnv"
Jul 06 21:00:30.233600 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:30.233576    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-kwjqs"
Jul 06 21:00:31.189195 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:31.189161    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-x9rxz"]
Jul 06 21:00:31.192684 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:31.192659    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-x9rxz"
Jul 06 21:00:31.196765 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:31.196740    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 06 21:00:31.196765 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:31.196746    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-hzhjx\""
Jul 06 21:00:31.205554 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:31.205532    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-x9rxz"]
Jul 06 21:00:31.296882 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:31.296848    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d457925-4653-4133-a036-b07ee89d4cd8-model-cache\") pod \"s3-init-x9rxz\" (UID: \"6d457925-4653-4133-a036-b07ee89d4cd8\") " pod="kserve/s3-init-x9rxz"
Jul 06 21:00:31.297263 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:31.296969    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl7rk\" (UniqueName: \"kubernetes.io/projected/6d457925-4653-4133-a036-b07ee89d4cd8-kube-api-access-bl7rk\") pod \"s3-init-x9rxz\" (UID: \"6d457925-4653-4133-a036-b07ee89d4cd8\") " pod="kserve/s3-init-x9rxz"
Jul 06 21:00:31.398367 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:31.398315    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl7rk\" (UniqueName: \"kubernetes.io/projected/6d457925-4653-4133-a036-b07ee89d4cd8-kube-api-access-bl7rk\") pod \"s3-init-x9rxz\" (UID: \"6d457925-4653-4133-a036-b07ee89d4cd8\") " pod="kserve/s3-init-x9rxz"
Jul 06 21:00:31.398581 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:31.398387    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d457925-4653-4133-a036-b07ee89d4cd8-model-cache\") pod \"s3-init-x9rxz\" (UID: \"6d457925-4653-4133-a036-b07ee89d4cd8\") " pod="kserve/s3-init-x9rxz"
Jul 06 21:00:31.398841 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:31.398820    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d457925-4653-4133-a036-b07ee89d4cd8-model-cache\") pod \"s3-init-x9rxz\" (UID: \"6d457925-4653-4133-a036-b07ee89d4cd8\") " pod="kserve/s3-init-x9rxz"
Jul 06 21:00:31.410395 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:31.410366    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl7rk\" (UniqueName: \"kubernetes.io/projected/6d457925-4653-4133-a036-b07ee89d4cd8-kube-api-access-bl7rk\") pod \"s3-init-x9rxz\" (UID: \"6d457925-4653-4133-a036-b07ee89d4cd8\") " pod="kserve/s3-init-x9rxz"
Jul 06 21:00:31.501686 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:31.501636    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-x9rxz"
Jul 06 21:00:31.644143 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:31.644119    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-x9rxz"]
Jul 06 21:00:31.646293 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:00:31.646266    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d457925_4653_4133_a036_b07ee89d4cd8.slice/crio-70eda9ec2a64f78aa7bc182433a23b572b7f29d10619e74e7638b27caafa28d2 WatchSource:0}: Error finding container 70eda9ec2a64f78aa7bc182433a23b572b7f29d10619e74e7638b27caafa28d2: Status 404 returned error can't find the container with id 70eda9ec2a64f78aa7bc182433a23b572b7f29d10619e74e7638b27caafa28d2
Jul 06 21:00:32.289761 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:32.289726    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-x9rxz" event={"ID":"6d457925-4653-4133-a036-b07ee89d4cd8","Type":"ContainerStarted","Data":"70eda9ec2a64f78aa7bc182433a23b572b7f29d10619e74e7638b27caafa28d2"}
Jul 06 21:00:36.314637 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:36.314595    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-x9rxz" event={"ID":"6d457925-4653-4133-a036-b07ee89d4cd8","Type":"ContainerStarted","Data":"9a3ef877bddcef109220ef9fba56e0560845b5cc36e8b0172bc8c271e25b854b"}
Jul 06 21:00:37.321160 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:37.321127    2566 generic.go:356] "Generic (PLEG): container finished" podID="6d457925-4653-4133-a036-b07ee89d4cd8" containerID="9a3ef877bddcef109220ef9fba56e0560845b5cc36e8b0172bc8c271e25b854b" exitCode=0
Jul 06 21:00:37.321584 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:37.321211    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-x9rxz" event={"ID":"6d457925-4653-4133-a036-b07ee89d4cd8","Type":"ContainerDied","Data":"9a3ef877bddcef109220ef9fba56e0560845b5cc36e8b0172bc8c271e25b854b"}
Jul 06 21:00:42.039623 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:42.039600    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 06 21:00:42.355924 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:42.355843    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-x9rxz" event={"ID":"6d457925-4653-4133-a036-b07ee89d4cd8","Type":"ContainerStarted","Data":"1a211eedb674e265ba1047618f8dc581356fe81a6c31c45d1fda76fb5a2e68e5"}
Jul 06 21:00:42.381682 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:00:42.381629    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-x9rxz" podStartSLOduration=1.196638289 podStartE2EDuration="11.381609176s" totalImagesPullingTime="8.332151243s" totalInitContainerRuntime="1.852819644s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:00:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:00:42.380188248 +0000 UTC m=+720.846429503" watchObservedRunningTime="2026-07-06 21:00:42.381609176 +0000 UTC m=+720.847850430"
Jul 06 21:01:47.700294 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:01:47.700259    2566 generic.go:356] "Generic (PLEG): container finished" podID="6d457925-4653-4133-a036-b07ee89d4cd8" containerID="1a211eedb674e265ba1047618f8dc581356fe81a6c31c45d1fda76fb5a2e68e5" exitCode=0
Jul 06 21:01:47.700743 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:01:47.700337    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-x9rxz" event={"ID":"6d457925-4653-4133-a036-b07ee89d4cd8","Type":"ContainerDied","Data":"1a211eedb674e265ba1047618f8dc581356fe81a6c31c45d1fda76fb5a2e68e5"}
Jul 06 21:01:48.838809 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:01:48.838783    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-x9rxz"
Jul 06 21:01:48.983362 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:01:48.983329    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d457925-4653-4133-a036-b07ee89d4cd8-model-cache\") pod \"6d457925-4653-4133-a036-b07ee89d4cd8\" (UID: \"6d457925-4653-4133-a036-b07ee89d4cd8\") "
Jul 06 21:01:48.983577 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:01:48.983407    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl7rk\" (UniqueName: \"kubernetes.io/projected/6d457925-4653-4133-a036-b07ee89d4cd8-kube-api-access-bl7rk\") pod \"6d457925-4653-4133-a036-b07ee89d4cd8\" (UID: \"6d457925-4653-4133-a036-b07ee89d4cd8\") "
Jul 06 21:01:48.985525 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:01:48.985487    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d457925-4653-4133-a036-b07ee89d4cd8-kube-api-access-bl7rk" (OuterVolumeSpecName: "kube-api-access-bl7rk") pod "6d457925-4653-4133-a036-b07ee89d4cd8" (UID: "6d457925-4653-4133-a036-b07ee89d4cd8"). InnerVolumeSpecName "kube-api-access-bl7rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:01:49.001955 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:01:49.001923    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d457925-4653-4133-a036-b07ee89d4cd8-model-cache" (OuterVolumeSpecName: "model-cache") pod "6d457925-4653-4133-a036-b07ee89d4cd8" (UID: "6d457925-4653-4133-a036-b07ee89d4cd8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:01:49.084540 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:01:49.084498    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl7rk\" (UniqueName: \"kubernetes.io/projected/6d457925-4653-4133-a036-b07ee89d4cd8-kube-api-access-bl7rk\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:01:49.084540 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:01:49.084538    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d457925-4653-4133-a036-b07ee89d4cd8-model-cache\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:01:49.711582 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:01:49.711548    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-x9rxz" event={"ID":"6d457925-4653-4133-a036-b07ee89d4cd8","Type":"ContainerDied","Data":"70eda9ec2a64f78aa7bc182433a23b572b7f29d10619e74e7638b27caafa28d2"}
Jul 06 21:01:49.711752 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:01:49.711590    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70eda9ec2a64f78aa7bc182433a23b572b7f29d10619e74e7638b27caafa28d2"
Jul 06 21:01:49.711752 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:01:49.711556    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-x9rxz"
Jul 06 21:02:23.939342 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:23.939304    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"]
Jul 06 21:02:23.939781 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:23.939691    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d457925-4653-4133-a036-b07ee89d4cd8" containerName="s3-init"
Jul 06 21:02:23.939781 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:23.939703    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d457925-4653-4133-a036-b07ee89d4cd8" containerName="s3-init"
Jul 06 21:02:23.939781 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:23.939713    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d457925-4653-4133-a036-b07ee89d4cd8" containerName="download-hf-model"
Jul 06 21:02:23.939781 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:23.939719    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d457925-4653-4133-a036-b07ee89d4cd8" containerName="download-hf-model"
Jul 06 21:02:23.939910 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:23.939785    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d457925-4653-4133-a036-b07ee89d4cd8" containerName="s3-init"
Jul 06 21:02:23.941914 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:23.941897    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:23.945942 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:23.945898    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g6qfk\""
Jul 06 21:02:23.945942 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:23.945929    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 06 21:02:23.946151 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:23.945958    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 06 21:02:23.946151 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:23.946014    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 06 21:02:23.946277 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:23.946263    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-kserve-self-signed-certs\""
Jul 06 21:02:23.961087 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:23.961058    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"]
Jul 06 21:02:24.081755 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.081725    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.081755 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.081759    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9h2l\" (UniqueName: \"kubernetes.io/projected/0bbcb9ae-5409-48cc-a194-562c839ab717-kube-api-access-q9h2l\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.081958 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.081787    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bbcb9ae-5409-48cc-a194-562c839ab717-tls-certs\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.081958 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.081834    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-dshm\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.081958 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.081873    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-model-cache\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.081958 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.081913    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-tmp-dir\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.081958 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.081957    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-home\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.182703 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.182667    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-tmp-dir\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.182886 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.182713    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-home\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.182886 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.182756    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.182886 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.182774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9h2l\" (UniqueName: \"kubernetes.io/projected/0bbcb9ae-5409-48cc-a194-562c839ab717-kube-api-access-q9h2l\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.182886 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.182797    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bbcb9ae-5409-48cc-a194-562c839ab717-tls-certs\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.182886 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.182816    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-dshm\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.182886 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.182847    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-model-cache\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.183202 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.183120    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-tmp-dir\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.183202 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.183159    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.183305 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.183202    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-model-cache\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.183305 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.183226    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-home\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.185056 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.185035    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-dshm\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.185225 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.185207    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bbcb9ae-5409-48cc-a194-562c839ab717-tls-certs\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.196686 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.196623    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9h2l\" (UniqueName: \"kubernetes.io/projected/0bbcb9ae-5409-48cc-a194-562c839ab717-kube-api-access-q9h2l\") pod \"precise-prefix-cache-test-kserve-6c69b69b5d-v4g66\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.252436 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.252398    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:24.370041 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.369992    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"]
Jul 06 21:02:24.374388 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.374365    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.379734 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.379709    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-epp-sa-dockercfg-wtdkn\""
Jul 06 21:02:24.390647 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.390622    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"]
Jul 06 21:02:24.397752 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.397731    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"]
Jul 06 21:02:24.399759 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:02:24.399735    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bbcb9ae_5409_48cc_a194_562c839ab717.slice/crio-0ca08a7912c6a4f574a0b97cfec2ff4cf5d7f1251116ebdeeec87a0cbb0ef2e9 WatchSource:0}: Error finding container 0ca08a7912c6a4f574a0b97cfec2ff4cf5d7f1251116ebdeeec87a0cbb0ef2e9: Status 404 returned error can't find the container with id 0ca08a7912c6a4f574a0b97cfec2ff4cf5d7f1251116ebdeeec87a0cbb0ef2e9
Jul 06 21:02:24.485933 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.485894    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.486111 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.485952    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nmzr\" (UniqueName: \"kubernetes.io/projected/d1640c2a-e75f-4f10-9801-700f1e11a562-kube-api-access-4nmzr\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.486111 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.485974    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1640c2a-e75f-4f10-9801-700f1e11a562-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.486111 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.486047    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.486111 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.486098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.486252 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.486168    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.587035 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.586996    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.587206 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.587056    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.587206 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.587091    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.587206 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.587125    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nmzr\" (UniqueName: \"kubernetes.io/projected/d1640c2a-e75f-4f10-9801-700f1e11a562-kube-api-access-4nmzr\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.587206 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.587149    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1640c2a-e75f-4f10-9801-700f1e11a562-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.587206 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.587170    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.587530 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.587503    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.587593 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.587528    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.587636 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.587592    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.587671 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.587649    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.589937 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.589912    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1640c2a-e75f-4f10-9801-700f1e11a562-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.597662 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.597631    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nmzr\" (UniqueName: \"kubernetes.io/projected/d1640c2a-e75f-4f10-9801-700f1e11a562-kube-api-access-4nmzr\") pod \"precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.688031 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.687985    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:24.838214 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.838182    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"]
Jul 06 21:02:24.838904 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:02:24.838880    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1640c2a_e75f_4f10_9801_700f1e11a562.slice/crio-df51cd32559221d210123ad60b8540c50213e930e1593a1723f36f0e674ddc5a WatchSource:0}: Error finding container df51cd32559221d210123ad60b8540c50213e930e1593a1723f36f0e674ddc5a: Status 404 returned error can't find the container with id df51cd32559221d210123ad60b8540c50213e930e1593a1723f36f0e674ddc5a
Jul 06 21:02:24.901142 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.901104    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" event={"ID":"d1640c2a-e75f-4f10-9801-700f1e11a562","Type":"ContainerStarted","Data":"df51cd32559221d210123ad60b8540c50213e930e1593a1723f36f0e674ddc5a"}
Jul 06 21:02:24.902791 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.902762    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66" event={"ID":"0bbcb9ae-5409-48cc-a194-562c839ab717","Type":"ContainerStarted","Data":"5417d7c9ad6104a1f5063e5053fcff5e08bb9c3b6677336790c6a5964076aa28"}
Jul 06 21:02:24.902935 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:24.902811    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66" event={"ID":"0bbcb9ae-5409-48cc-a194-562c839ab717","Type":"ContainerStarted","Data":"0ca08a7912c6a4f574a0b97cfec2ff4cf5d7f1251116ebdeeec87a0cbb0ef2e9"}
Jul 06 21:02:25.910183 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:25.910083    2566 generic.go:356] "Generic (PLEG): container finished" podID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerID="382fdc0ee1a09b18191eb95512a9e6af6a67f0bb5a47b9a6d7bc34342d04ccdd" exitCode=0
Jul 06 21:02:25.910183 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:25.910171    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" event={"ID":"d1640c2a-e75f-4f10-9801-700f1e11a562","Type":"ContainerDied","Data":"382fdc0ee1a09b18191eb95512a9e6af6a67f0bb5a47b9a6d7bc34342d04ccdd"}
Jul 06 21:02:27.929643 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:27.929601    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" event={"ID":"d1640c2a-e75f-4f10-9801-700f1e11a562","Type":"ContainerStarted","Data":"6696752d5b35b07d70c8029710aa93193f749c61693816996efa444c6757710b"}
Jul 06 21:02:28.939045 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:28.938865    2566 generic.go:356] "Generic (PLEG): container finished" podID="0bbcb9ae-5409-48cc-a194-562c839ab717" containerID="5417d7c9ad6104a1f5063e5053fcff5e08bb9c3b6677336790c6a5964076aa28" exitCode=0
Jul 06 21:02:28.939045 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:28.939009    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66" event={"ID":"0bbcb9ae-5409-48cc-a194-562c839ab717","Type":"ContainerDied","Data":"5417d7c9ad6104a1f5063e5053fcff5e08bb9c3b6677336790c6a5964076aa28"}
Jul 06 21:02:30.955407 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:30.955314    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66" event={"ID":"0bbcb9ae-5409-48cc-a194-562c839ab717","Type":"ContainerStarted","Data":"5bbac7f03b03bd6864795642ab3f38d07a352f0aab84ed66b0acf7edd16176f1"}
Jul 06 21:02:30.985177 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:30.985088    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66" podStartSLOduration=2.655671242 podStartE2EDuration="7.985068449s" totalImagesPullingTime="1.654449118s" totalInitContainerRuntime="3.674948089s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:02:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:02:30.984511723 +0000 UTC m=+829.450752971" watchObservedRunningTime="2026-07-06 21:02:30.985068449 +0000 UTC m=+829.451309704"
Jul 06 21:02:31.253021 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:31.252982    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:34.253447 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:34.253406    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:34.267928 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:34.267900    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:34.996238 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:34.996207    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:02:38.004755 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:38.004719    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr_d1640c2a-e75f-4f10-9801-700f1e11a562/main/0.log"
Jul 06 21:02:38.005230 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:38.005182    2566 generic.go:356] "Generic (PLEG): container finished" podID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerID="6696752d5b35b07d70c8029710aa93193f749c61693816996efa444c6757710b" exitCode=1
Jul 06 21:02:38.005302 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:38.005244    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" event={"ID":"d1640c2a-e75f-4f10-9801-700f1e11a562","Type":"ContainerDied","Data":"6696752d5b35b07d70c8029710aa93193f749c61693816996efa444c6757710b"}
Jul 06 21:02:58.133024 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:58.132938    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr_d1640c2a-e75f-4f10-9801-700f1e11a562/main/0.log"
Jul 06 21:02:58.133472 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:58.133326    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" event={"ID":"d1640c2a-e75f-4f10-9801-700f1e11a562","Type":"ContainerStarted","Data":"a1a6ddf25934b54ded395e45fef7f8385fcce2a50d047c5a4ce686f028b04bce"}
Jul 06 21:02:58.133673 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:58.133657    2566 scope.go:117] "RemoveContainer" containerID="6696752d5b35b07d70c8029710aa93193f749c61693816996efa444c6757710b"
Jul 06 21:02:59.143740 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:59.143711    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr_d1640c2a-e75f-4f10-9801-700f1e11a562/main/0.log"
Jul 06 21:02:59.144219 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:59.144035    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" event={"ID":"d1640c2a-e75f-4f10-9801-700f1e11a562","Type":"ContainerStarted","Data":"4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13"}
Jul 06 21:02:59.144961 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:59.144937    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:02:59.174341 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:02:59.174285    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" podStartSLOduration=2.6592216730000002 podStartE2EDuration="35.174269167s" totalImagesPullingTime="31.790222418s" totalInitContainerRuntime="724.825076ms" isStatefulPod=false podCreationTimestamp="2026-07-06 21:02:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:02:59.173119974 +0000 UTC m=+857.639361229" watchObservedRunningTime="2026-07-06 21:02:59.174269167 +0000 UTC m=+857.640510444"
Jul 06 21:03:04.688321 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:04.688271    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:03:04.688869 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:04.688372    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:03:04.689801 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:03:04.689776    2566 logging.go:55] [core] [Channel #17 SubChannel #18]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.45:9003", ServerName: "10.132.0.45:9003", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.45:9003: connect: connection refused"
Jul 06 21:03:04.689801 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:04.689775    2566 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="tokenizer" probeResult="failure" output="Get \"http://10.132.0.45:8082/healthz\": dial tcp 10.132.0.45:8082: connect: connection refused"
Jul 06 21:03:05.688667 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:05.688613    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.132.0.45:9003\" within 1s: context deadline exceeded"
Jul 06 21:03:14.688774 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:03:14.688734    2566 logging.go:55] [core] [Channel #19 SubChannel #20]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.45:9003", ServerName: "10.132.0.45:9003", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.45:9003: connect: connection refused"
Jul 06 21:03:14.690649 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:14.690626    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:03:14.691906 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:14.691888    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:03:15.688448 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:15.688400    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.132.0.45:9003\" within 1s: context deadline exceeded"
Jul 06 21:03:35.242260 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:35.242185    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:03:36.687822 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:36.687787    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"]
Jul 06 21:03:36.688244 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:36.688015    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="tokenizer" containerID="cri-o://a1a6ddf25934b54ded395e45fef7f8385fcce2a50d047c5a4ce686f028b04bce" gracePeriod=60
Jul 06 21:03:36.696068 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:36.696042    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"]
Jul 06 21:03:36.697431 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:03:36.697398    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:03:36.697431 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:03:36.697431 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:03:36.697431 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13" cmd=["/bin/sleep","15"]
Jul 06 21:03:36.697710 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:36.697474    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" message="" err=<
Jul 06 21:03:36.697710 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:03:36.697710 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:03:36.697710 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:03:36.697710 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:03:36.697492    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:03:36.697710 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:03:36.697710 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:03:36.697710 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="main" containerID="cri-o://4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13"
Jul 06 21:03:36.697710 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:36.697535    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="main" containerID="cri-o://4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13" gracePeriod=60
Jul 06 21:03:37.373928 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:37.373893    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr_d1640c2a-e75f-4f10-9801-700f1e11a562/main/0.log"
Jul 06 21:03:37.374306 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:37.374285    2566 generic.go:356] "Generic (PLEG): container finished" podID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerID="4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13" exitCode=0
Jul 06 21:03:37.374368 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:37.374336    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" event={"ID":"d1640c2a-e75f-4f10-9801-700f1e11a562","Type":"ContainerDied","Data":"4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13"}
Jul 06 21:03:37.374417 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:37.374406    2566 scope.go:117] "RemoveContainer" containerID="6696752d5b35b07d70c8029710aa93193f749c61693816996efa444c6757710b"
Jul 06 21:03:38.250305 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.250277    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:03:38.360774 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.360679    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1640c2a-e75f-4f10-9801-700f1e11a562-tls-certs\") pod \"d1640c2a-e75f-4f10-9801-700f1e11a562\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") "
Jul 06 21:03:38.360774 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.360726    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-kserve-provision-location\") pod \"d1640c2a-e75f-4f10-9801-700f1e11a562\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") "
Jul 06 21:03:38.360774 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.360771    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nmzr\" (UniqueName: \"kubernetes.io/projected/d1640c2a-e75f-4f10-9801-700f1e11a562-kube-api-access-4nmzr\") pod \"d1640c2a-e75f-4f10-9801-700f1e11a562\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") "
Jul 06 21:03:38.361016 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.360937    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-uds\") pod \"d1640c2a-e75f-4f10-9801-700f1e11a562\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") "
Jul 06 21:03:38.361076 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.361030    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-tmp\") pod \"d1640c2a-e75f-4f10-9801-700f1e11a562\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") "
Jul 06 21:03:38.361133 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.361084    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-cache\") pod \"d1640c2a-e75f-4f10-9801-700f1e11a562\" (UID: \"d1640c2a-e75f-4f10-9801-700f1e11a562\") "
Jul 06 21:03:38.361252 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.361228    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "d1640c2a-e75f-4f10-9801-700f1e11a562" (UID: "d1640c2a-e75f-4f10-9801-700f1e11a562"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:03:38.361372 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.361355    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:03:38.361485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.361434    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "d1640c2a-e75f-4f10-9801-700f1e11a562" (UID: "d1640c2a-e75f-4f10-9801-700f1e11a562"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:03:38.361556 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.361539    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "d1640c2a-e75f-4f10-9801-700f1e11a562" (UID: "d1640c2a-e75f-4f10-9801-700f1e11a562"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:03:38.361768 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.361743    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d1640c2a-e75f-4f10-9801-700f1e11a562" (UID: "d1640c2a-e75f-4f10-9801-700f1e11a562"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:03:38.363038 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.363017    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1640c2a-e75f-4f10-9801-700f1e11a562-kube-api-access-4nmzr" (OuterVolumeSpecName: "kube-api-access-4nmzr") pod "d1640c2a-e75f-4f10-9801-700f1e11a562" (UID: "d1640c2a-e75f-4f10-9801-700f1e11a562"). InnerVolumeSpecName "kube-api-access-4nmzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:03:38.363124 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.363044    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1640c2a-e75f-4f10-9801-700f1e11a562-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d1640c2a-e75f-4f10-9801-700f1e11a562" (UID: "d1640c2a-e75f-4f10-9801-700f1e11a562"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:03:38.380786 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.380756    2566 generic.go:356] "Generic (PLEG): container finished" podID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerID="a1a6ddf25934b54ded395e45fef7f8385fcce2a50d047c5a4ce686f028b04bce" exitCode=0
Jul 06 21:03:38.380909 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.380810    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"
Jul 06 21:03:38.380909 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.380816    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" event={"ID":"d1640c2a-e75f-4f10-9801-700f1e11a562","Type":"ContainerDied","Data":"a1a6ddf25934b54ded395e45fef7f8385fcce2a50d047c5a4ce686f028b04bce"}
Jul 06 21:03:38.380909 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.380860    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr" event={"ID":"d1640c2a-e75f-4f10-9801-700f1e11a562","Type":"ContainerDied","Data":"df51cd32559221d210123ad60b8540c50213e930e1593a1723f36f0e674ddc5a"}
Jul 06 21:03:38.380909 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.380882    2566 scope.go:117] "RemoveContainer" containerID="4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13"
Jul 06 21:03:38.390215 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.390198    2566 scope.go:117] "RemoveContainer" containerID="a1a6ddf25934b54ded395e45fef7f8385fcce2a50d047c5a4ce686f028b04bce"
Jul 06 21:03:38.397803 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.397784    2566 scope.go:117] "RemoveContainer" containerID="382fdc0ee1a09b18191eb95512a9e6af6a67f0bb5a47b9a6d7bc34342d04ccdd"
Jul 06 21:03:38.404752 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.404728    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"]
Jul 06 21:03:38.406050 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.406038    2566 scope.go:117] "RemoveContainer" containerID="4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13"
Jul 06 21:03:38.406301 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:03:38.406283    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13\": container with ID starting with 4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13 not found: ID does not exist" containerID="4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13"
Jul 06 21:03:38.406352 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.406309    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13"} err="failed to get container status \"4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13\": rpc error: code = NotFound desc = could not find container \"4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13\": container with ID starting with 4afc3e1f505d4f7b38aa611709e47542591b3e152ff4ef39169b8ea02a7b3c13 not found: ID does not exist"
Jul 06 21:03:38.406352 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.406325    2566 scope.go:117] "RemoveContainer" containerID="a1a6ddf25934b54ded395e45fef7f8385fcce2a50d047c5a4ce686f028b04bce"
Jul 06 21:03:38.406602 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:03:38.406560    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1a6ddf25934b54ded395e45fef7f8385fcce2a50d047c5a4ce686f028b04bce\": container with ID starting with a1a6ddf25934b54ded395e45fef7f8385fcce2a50d047c5a4ce686f028b04bce not found: ID does not exist" containerID="a1a6ddf25934b54ded395e45fef7f8385fcce2a50d047c5a4ce686f028b04bce"
Jul 06 21:03:38.406640 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.406618    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1a6ddf25934b54ded395e45fef7f8385fcce2a50d047c5a4ce686f028b04bce"} err="failed to get container status \"a1a6ddf25934b54ded395e45fef7f8385fcce2a50d047c5a4ce686f028b04bce\": rpc error: code = NotFound desc = could not find container \"a1a6ddf25934b54ded395e45fef7f8385fcce2a50d047c5a4ce686f028b04bce\": container with ID starting with a1a6ddf25934b54ded395e45fef7f8385fcce2a50d047c5a4ce686f028b04bce not found: ID does not exist"
Jul 06 21:03:38.406640 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.406634    2566 scope.go:117] "RemoveContainer" containerID="382fdc0ee1a09b18191eb95512a9e6af6a67f0bb5a47b9a6d7bc34342d04ccdd"
Jul 06 21:03:38.406859 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:03:38.406841    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"382fdc0ee1a09b18191eb95512a9e6af6a67f0bb5a47b9a6d7bc34342d04ccdd\": container with ID starting with 382fdc0ee1a09b18191eb95512a9e6af6a67f0bb5a47b9a6d7bc34342d04ccdd not found: ID does not exist" containerID="382fdc0ee1a09b18191eb95512a9e6af6a67f0bb5a47b9a6d7bc34342d04ccdd"
Jul 06 21:03:38.406921 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.406865    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"382fdc0ee1a09b18191eb95512a9e6af6a67f0bb5a47b9a6d7bc34342d04ccdd"} err="failed to get container status \"382fdc0ee1a09b18191eb95512a9e6af6a67f0bb5a47b9a6d7bc34342d04ccdd\": rpc error: code = NotFound desc = could not find container \"382fdc0ee1a09b18191eb95512a9e6af6a67f0bb5a47b9a6d7bc34342d04ccdd\": container with ID starting with 382fdc0ee1a09b18191eb95512a9e6af6a67f0bb5a47b9a6d7bc34342d04ccdd not found: ID does not exist"
Jul 06 21:03:38.411803 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.411782    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-66d4c9884dlzr"]
Jul 06 21:03:38.461917 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.461886    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-tmp\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:03:38.461917 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.461913    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-tokenizer-cache\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:03:38.461917 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.461923    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1640c2a-e75f-4f10-9801-700f1e11a562-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:03:38.462163 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.461935    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d1640c2a-e75f-4f10-9801-700f1e11a562-kserve-provision-location\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:03:38.462163 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:38.461944    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nmzr\" (UniqueName: \"kubernetes.io/projected/d1640c2a-e75f-4f10-9801-700f1e11a562-kube-api-access-4nmzr\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:03:40.002153 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:40.002122    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" path="/var/lib/kubelet/pods/d1640c2a-e75f-4f10-9801-700f1e11a562/volumes"
Jul 06 21:03:51.719068 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:51.719024    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66" podUID="0bbcb9ae-5409-48cc-a194-562c839ab717" containerName="main" containerID="cri-o://5bbac7f03b03bd6864795642ab3f38d07a352f0aab84ed66b0acf7edd16176f1" gracePeriod=45
Jul 06 21:03:51.978376 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:51.978317    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:03:52.084969 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.084933    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-model-cache\") pod \"0bbcb9ae-5409-48cc-a194-562c839ab717\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") "
Jul 06 21:03:52.085154 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.084995    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-home\") pod \"0bbcb9ae-5409-48cc-a194-562c839ab717\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") "
Jul 06 21:03:52.085154 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.085026    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-kserve-provision-location\") pod \"0bbcb9ae-5409-48cc-a194-562c839ab717\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") "
Jul 06 21:03:52.085154 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.085047    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-tmp-dir\") pod \"0bbcb9ae-5409-48cc-a194-562c839ab717\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") "
Jul 06 21:03:52.085154 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.085080    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bbcb9ae-5409-48cc-a194-562c839ab717-tls-certs\") pod \"0bbcb9ae-5409-48cc-a194-562c839ab717\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") "
Jul 06 21:03:52.085154 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.085100    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-dshm\") pod \"0bbcb9ae-5409-48cc-a194-562c839ab717\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") "
Jul 06 21:03:52.085154 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.085153    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9h2l\" (UniqueName: \"kubernetes.io/projected/0bbcb9ae-5409-48cc-a194-562c839ab717-kube-api-access-q9h2l\") pod \"0bbcb9ae-5409-48cc-a194-562c839ab717\" (UID: \"0bbcb9ae-5409-48cc-a194-562c839ab717\") "
Jul 06 21:03:52.085514 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.085278    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-model-cache" (OuterVolumeSpecName: "model-cache") pod "0bbcb9ae-5409-48cc-a194-562c839ab717" (UID: "0bbcb9ae-5409-48cc-a194-562c839ab717"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:03:52.085514 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.085345    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0bbcb9ae-5409-48cc-a194-562c839ab717" (UID: "0bbcb9ae-5409-48cc-a194-562c839ab717"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:03:52.085514 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.085395    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-home" (OuterVolumeSpecName: "home") pod "0bbcb9ae-5409-48cc-a194-562c839ab717" (UID: "0bbcb9ae-5409-48cc-a194-562c839ab717"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:03:52.085688 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.085522    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-model-cache\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:03:52.085688 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.085539    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-home\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:03:52.085688 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.085547    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-tmp-dir\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:03:52.087325 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.087298    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bbcb9ae-5409-48cc-a194-562c839ab717-kube-api-access-q9h2l" (OuterVolumeSpecName: "kube-api-access-q9h2l") pod "0bbcb9ae-5409-48cc-a194-562c839ab717" (UID: "0bbcb9ae-5409-48cc-a194-562c839ab717"). InnerVolumeSpecName "kube-api-access-q9h2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:03:52.087325 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.087317    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bbcb9ae-5409-48cc-a194-562c839ab717-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0bbcb9ae-5409-48cc-a194-562c839ab717" (UID: "0bbcb9ae-5409-48cc-a194-562c839ab717"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:03:52.087508 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.087362    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-dshm" (OuterVolumeSpecName: "dshm") pod "0bbcb9ae-5409-48cc-a194-562c839ab717" (UID: "0bbcb9ae-5409-48cc-a194-562c839ab717"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:03:52.147418 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.147374    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "0bbcb9ae-5409-48cc-a194-562c839ab717" (UID: "0bbcb9ae-5409-48cc-a194-562c839ab717"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:03:52.186437 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.186404    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-kserve-provision-location\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:03:52.186437 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.186431    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bbcb9ae-5409-48cc-a194-562c839ab717-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:03:52.186437 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.186441    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bbcb9ae-5409-48cc-a194-562c839ab717-dshm\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:03:52.186690 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.186450    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9h2l\" (UniqueName: \"kubernetes.io/projected/0bbcb9ae-5409-48cc-a194-562c839ab717-kube-api-access-q9h2l\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:03:52.456765 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.456727    2566 generic.go:356] "Generic (PLEG): container finished" podID="0bbcb9ae-5409-48cc-a194-562c839ab717" containerID="5bbac7f03b03bd6864795642ab3f38d07a352f0aab84ed66b0acf7edd16176f1" exitCode=0
Jul 06 21:03:52.456942 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.456784    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"
Jul 06 21:03:52.456942 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.456808    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66" event={"ID":"0bbcb9ae-5409-48cc-a194-562c839ab717","Type":"ContainerDied","Data":"5bbac7f03b03bd6864795642ab3f38d07a352f0aab84ed66b0acf7edd16176f1"}
Jul 06 21:03:52.456942 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.456859    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66" event={"ID":"0bbcb9ae-5409-48cc-a194-562c839ab717","Type":"ContainerDied","Data":"0ca08a7912c6a4f574a0b97cfec2ff4cf5d7f1251116ebdeeec87a0cbb0ef2e9"}
Jul 06 21:03:52.456942 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.456880    2566 scope.go:117] "RemoveContainer" containerID="5bbac7f03b03bd6864795642ab3f38d07a352f0aab84ed66b0acf7edd16176f1"
Jul 06 21:03:52.470008 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.469987    2566 scope.go:117] "RemoveContainer" containerID="5417d7c9ad6104a1f5063e5053fcff5e08bb9c3b6677336790c6a5964076aa28"
Jul 06 21:03:52.478328 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.478307    2566 scope.go:117] "RemoveContainer" containerID="5bbac7f03b03bd6864795642ab3f38d07a352f0aab84ed66b0acf7edd16176f1"
Jul 06 21:03:52.478619 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:03:52.478599    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bbac7f03b03bd6864795642ab3f38d07a352f0aab84ed66b0acf7edd16176f1\": container with ID starting with 5bbac7f03b03bd6864795642ab3f38d07a352f0aab84ed66b0acf7edd16176f1 not found: ID does not exist" containerID="5bbac7f03b03bd6864795642ab3f38d07a352f0aab84ed66b0acf7edd16176f1"
Jul 06 21:03:52.478673 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.478628    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bbac7f03b03bd6864795642ab3f38d07a352f0aab84ed66b0acf7edd16176f1"} err="failed to get container status \"5bbac7f03b03bd6864795642ab3f38d07a352f0aab84ed66b0acf7edd16176f1\": rpc error: code = NotFound desc = could not find container \"5bbac7f03b03bd6864795642ab3f38d07a352f0aab84ed66b0acf7edd16176f1\": container with ID starting with 5bbac7f03b03bd6864795642ab3f38d07a352f0aab84ed66b0acf7edd16176f1 not found: ID does not exist"
Jul 06 21:03:52.478673 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.478646    2566 scope.go:117] "RemoveContainer" containerID="5417d7c9ad6104a1f5063e5053fcff5e08bb9c3b6677336790c6a5964076aa28"
Jul 06 21:03:52.478855 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:03:52.478834    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5417d7c9ad6104a1f5063e5053fcff5e08bb9c3b6677336790c6a5964076aa28\": container with ID starting with 5417d7c9ad6104a1f5063e5053fcff5e08bb9c3b6677336790c6a5964076aa28 not found: ID does not exist" containerID="5417d7c9ad6104a1f5063e5053fcff5e08bb9c3b6677336790c6a5964076aa28"
Jul 06 21:03:52.478911 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.478864    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5417d7c9ad6104a1f5063e5053fcff5e08bb9c3b6677336790c6a5964076aa28"} err="failed to get container status \"5417d7c9ad6104a1f5063e5053fcff5e08bb9c3b6677336790c6a5964076aa28\": rpc error: code = NotFound desc = could not find container \"5417d7c9ad6104a1f5063e5053fcff5e08bb9c3b6677336790c6a5964076aa28\": container with ID starting with 5417d7c9ad6104a1f5063e5053fcff5e08bb9c3b6677336790c6a5964076aa28 not found: ID does not exist"
Jul 06 21:03:52.485079 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.485055    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"]
Jul 06 21:03:52.489339 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:52.489318    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6c69b69b5d-v4g66"]
Jul 06 21:03:54.003448 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:03:54.003412    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bbcb9ae-5409-48cc-a194-562c839ab717" path="/var/lib/kubelet/pods/0bbcb9ae-5409-48cc-a194-562c839ab717/volumes"
Jul 06 21:04:09.333106 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333068    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"]
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333423    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="main"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333434    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="main"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333448    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="tokenizer"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333466    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="tokenizer"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333473    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="main"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333478    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="main"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333488    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="storage-initializer"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333494    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="storage-initializer"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333502    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bbcb9ae-5409-48cc-a194-562c839ab717" containerName="storage-initializer"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333507    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bbcb9ae-5409-48cc-a194-562c839ab717" containerName="storage-initializer"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333518    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bbcb9ae-5409-48cc-a194-562c839ab717" containerName="main"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333523    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bbcb9ae-5409-48cc-a194-562c839ab717" containerName="main"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333595    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="main"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333606    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bbcb9ae-5409-48cc-a194-562c839ab717" containerName="main"
Jul 06 21:04:09.334485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.333611    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="tokenizer"
Jul 06 21:04:09.335533 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.335518    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.339793 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.339767    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs\""
Jul 06 21:04:09.339938 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.339804    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 06 21:04:09.339938 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.339767    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 06 21:04:09.339938 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.339863    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g6qfk\""
Jul 06 21:04:09.355052 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.355021    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"]
Jul 06 21:04:09.434387 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.434347    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.434387 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.434388    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.434689 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.434422    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.434689 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.434528    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.434689 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.434574    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwg9w\" (UniqueName: \"kubernetes.io/projected/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-kube-api-access-dwg9w\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.434689 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.434666    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.535385 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.535346    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwg9w\" (UniqueName: \"kubernetes.io/projected/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-kube-api-access-dwg9w\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.535579 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.535403    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.535579 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.535441    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.535579 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.535477    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.535579 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.535508    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.535579 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.535555    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.535908 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.535886    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.535962 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.535908    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.535962 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.535944    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.537621 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.537600    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.537942 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.537920    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.544356 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.544326    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwg9w\" (UniqueName: \"kubernetes.io/projected/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-kube-api-access-dwg9w\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.646585 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.646495    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:09.718599 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.718564    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"]
Jul 06 21:04:09.719168 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.719042    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1640c2a-e75f-4f10-9801-700f1e11a562" containerName="main"
Jul 06 21:04:09.721705 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.721687    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:09.728182 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.728160    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-router-managed-test-llm-e45d1f79-epp-sa-dockercfg-w29k8\""
Jul 06 21:04:09.752527 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.752491    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"]
Jul 06 21:04:09.810109 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.810079    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"]
Jul 06 21:04:09.813829 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:04:09.813799    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a7ea88_1fac_40b4_a316_a8bb2a47b0d9.slice/crio-79cd5dbc332f3986045e6a45c0eabe830b5dc924b6914d9de3c65b4aef7dadba WatchSource:0}: Error finding container 79cd5dbc332f3986045e6a45c0eabe830b5dc924b6914d9de3c65b4aef7dadba: Status 404 returned error can't find the container with id 79cd5dbc332f3986045e6a45c0eabe830b5dc924b6914d9de3c65b4aef7dadba
Jul 06 21:04:09.837240 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.837213    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a4b0fe0-7e88-496b-aaec-57de430de975-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5\" (UID: \"2a4b0fe0-7e88-496b-aaec-57de430de975\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:09.837343 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.837252    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2a4b0fe0-7e88-496b-aaec-57de430de975-tokenizer-uds\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5\" (UID: \"2a4b0fe0-7e88-496b-aaec-57de430de975\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:09.837343 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.837279    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h44m\" (UniqueName: \"kubernetes.io/projected/2a4b0fe0-7e88-496b-aaec-57de430de975-kube-api-access-8h44m\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5\" (UID: \"2a4b0fe0-7e88-496b-aaec-57de430de975\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:09.938545 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.938426    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a4b0fe0-7e88-496b-aaec-57de430de975-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5\" (UID: \"2a4b0fe0-7e88-496b-aaec-57de430de975\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:09.938545 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.938498    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2a4b0fe0-7e88-496b-aaec-57de430de975-tokenizer-uds\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5\" (UID: \"2a4b0fe0-7e88-496b-aaec-57de430de975\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:09.938770 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.938665    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h44m\" (UniqueName: \"kubernetes.io/projected/2a4b0fe0-7e88-496b-aaec-57de430de975-kube-api-access-8h44m\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5\" (UID: \"2a4b0fe0-7e88-496b-aaec-57de430de975\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:09.938937 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.938914    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2a4b0fe0-7e88-496b-aaec-57de430de975-tokenizer-uds\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5\" (UID: \"2a4b0fe0-7e88-496b-aaec-57de430de975\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:09.941387 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.941358    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a4b0fe0-7e88-496b-aaec-57de430de975-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5\" (UID: \"2a4b0fe0-7e88-496b-aaec-57de430de975\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:09.958805 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:09.958780    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h44m\" (UniqueName: \"kubernetes.io/projected/2a4b0fe0-7e88-496b-aaec-57de430de975-kube-api-access-8h44m\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5\" (UID: \"2a4b0fe0-7e88-496b-aaec-57de430de975\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:10.032487 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:10.032439    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:10.180566 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:10.180533    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"]
Jul 06 21:04:10.183669 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:04:10.183643    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a4b0fe0_7e88_496b_aaec_57de430de975.slice/crio-8fe7b10fcb0999ec165569241c61d084ceab1cef167211654ab08ae79fb9dcb6 WatchSource:0}: Error finding container 8fe7b10fcb0999ec165569241c61d084ceab1cef167211654ab08ae79fb9dcb6: Status 404 returned error can't find the container with id 8fe7b10fcb0999ec165569241c61d084ceab1cef167211654ab08ae79fb9dcb6
Jul 06 21:04:10.556062 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:10.556021    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc" event={"ID":"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9","Type":"ContainerStarted","Data":"84765cf6c090bcaa73c0e023dabc36e3d3ff04dd93d81fc6cab9c05511e4999a"}
Jul 06 21:04:10.556537 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:10.556107    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc" event={"ID":"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9","Type":"ContainerStarted","Data":"79cd5dbc332f3986045e6a45c0eabe830b5dc924b6914d9de3c65b4aef7dadba"}
Jul 06 21:04:10.557691 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:10.557663    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5" event={"ID":"2a4b0fe0-7e88-496b-aaec-57de430de975","Type":"ContainerStarted","Data":"e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68"}
Jul 06 21:04:10.557817 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:10.557713    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5" event={"ID":"2a4b0fe0-7e88-496b-aaec-57de430de975","Type":"ContainerStarted","Data":"8fe7b10fcb0999ec165569241c61d084ceab1cef167211654ab08ae79fb9dcb6"}
Jul 06 21:04:10.557817 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:10.557749    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:10.586887 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:10.586822    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc" podStartSLOduration=1.586806391 podStartE2EDuration="1.586806391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:04:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:04:10.584336583 +0000 UTC m=+929.050577836" watchObservedRunningTime="2026-07-06 21:04:10.586806391 +0000 UTC m=+929.053047642"
Jul 06 21:04:10.635333 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:10.635278    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5" podStartSLOduration=1.635261396 podStartE2EDuration="1.635261396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:04:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:04:10.628428783 +0000 UTC m=+929.094670037" watchObservedRunningTime="2026-07-06 21:04:10.635261396 +0000 UTC m=+929.101502653"
Jul 06 21:04:10.646962 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:10.646931    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:19.647001 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:19.646954    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:19.659581 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:19.659558    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:20.620733 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:20.620704    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:41.564543 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:41.564509    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:43.357025 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.356989    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"]
Jul 06 21:04:43.360378 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.360346    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"]
Jul 06 21:04:43.371270 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:04:43.371231    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:04:43.371270 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:04:43.371270 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:04:43.371270 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68" cmd=["/bin/sleep","15"]
Jul 06 21:04:43.371704 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.371293    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5" message="" err=<
Jul 06 21:04:43.371704 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:04:43.371704 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:04:43.371704 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:04:43.371704 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:04:43.371311    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:04:43.371704 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:04:43.371704 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:04:43.371704 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5" podUID="2a4b0fe0-7e88-496b-aaec-57de430de975" containerName="main" containerID="cri-o://e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68"
Jul 06 21:04:43.371704 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.371357    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5" podUID="2a4b0fe0-7e88-496b-aaec-57de430de975" containerName="main" containerID="cri-o://e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68" gracePeriod=60
Jul 06 21:04:43.632661 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.632633    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:43.730069 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.730032    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2a4b0fe0-7e88-496b-aaec-57de430de975-tokenizer-uds\") pod \"2a4b0fe0-7e88-496b-aaec-57de430de975\" (UID: \"2a4b0fe0-7e88-496b-aaec-57de430de975\") "
Jul 06 21:04:43.730261 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.730092    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a4b0fe0-7e88-496b-aaec-57de430de975-tls-certs\") pod \"2a4b0fe0-7e88-496b-aaec-57de430de975\" (UID: \"2a4b0fe0-7e88-496b-aaec-57de430de975\") "
Jul 06 21:04:43.730261 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.730166    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8h44m\" (UniqueName: \"kubernetes.io/projected/2a4b0fe0-7e88-496b-aaec-57de430de975-kube-api-access-8h44m\") pod \"2a4b0fe0-7e88-496b-aaec-57de430de975\" (UID: \"2a4b0fe0-7e88-496b-aaec-57de430de975\") "
Jul 06 21:04:43.730390 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.730365    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a4b0fe0-7e88-496b-aaec-57de430de975-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "2a4b0fe0-7e88-496b-aaec-57de430de975" (UID: "2a4b0fe0-7e88-496b-aaec-57de430de975"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:04:43.732354 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.732321    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a4b0fe0-7e88-496b-aaec-57de430de975-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2a4b0fe0-7e88-496b-aaec-57de430de975" (UID: "2a4b0fe0-7e88-496b-aaec-57de430de975"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:04:43.732484 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.732381    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a4b0fe0-7e88-496b-aaec-57de430de975-kube-api-access-8h44m" (OuterVolumeSpecName: "kube-api-access-8h44m") pod "2a4b0fe0-7e88-496b-aaec-57de430de975" (UID: "2a4b0fe0-7e88-496b-aaec-57de430de975"). InnerVolumeSpecName "kube-api-access-8h44m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:04:43.735726 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.735703    2566 generic.go:356] "Generic (PLEG): container finished" podID="2a4b0fe0-7e88-496b-aaec-57de430de975" containerID="e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68" exitCode=0
Jul 06 21:04:43.735822 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.735740    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"
Jul 06 21:04:43.735822 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.735786    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5" event={"ID":"2a4b0fe0-7e88-496b-aaec-57de430de975","Type":"ContainerDied","Data":"e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68"}
Jul 06 21:04:43.735943 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.735844    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5" event={"ID":"2a4b0fe0-7e88-496b-aaec-57de430de975","Type":"ContainerDied","Data":"8fe7b10fcb0999ec165569241c61d084ceab1cef167211654ab08ae79fb9dcb6"}
Jul 06 21:04:43.735943 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.735876    2566 scope.go:117] "RemoveContainer" containerID="e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68"
Jul 06 21:04:43.746335 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.746318    2566 scope.go:117] "RemoveContainer" containerID="e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68"
Jul 06 21:04:43.746600 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:04:43.746581    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68\": container with ID starting with e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68 not found: ID does not exist" containerID="e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68"
Jul 06 21:04:43.746653 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.746607    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68"} err="failed to get container status \"e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68\": rpc error: code = NotFound desc = could not find container \"e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68\": container with ID starting with e3334e99a8293eedbc1a70e2f54c9e0040fc0846b3afe83fd6c86561bbfc2f68 not found: ID does not exist"
Jul 06 21:04:43.770666 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.770631    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"]
Jul 06 21:04:43.772884 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.772859    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schez4nq5"]
Jul 06 21:04:43.830857 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.830827    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2a4b0fe0-7e88-496b-aaec-57de430de975-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:04:43.830857 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.830853    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a4b0fe0-7e88-496b-aaec-57de430de975-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:04:43.830857 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:43.830863    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8h44m\" (UniqueName: \"kubernetes.io/projected/2a4b0fe0-7e88-496b-aaec-57de430de975-kube-api-access-8h44m\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:04:44.004595 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:44.004553    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a4b0fe0-7e88-496b-aaec-57de430de975" path="/var/lib/kubelet/pods/2a4b0fe0-7e88-496b-aaec-57de430de975/volumes"
Jul 06 21:04:58.383436 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.383374    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc" podUID="38a7ea88-1fac-40b4-a316-a8bb2a47b0d9" containerName="main" containerID="cri-o://84765cf6c090bcaa73c0e023dabc36e3d3ff04dd93d81fc6cab9c05511e4999a" gracePeriod=45
Jul 06 21:04:58.624730 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.624705    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:58.769248 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.769210    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-home\") pod \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") "
Jul 06 21:04:58.769547 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.769270    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-tls-certs\") pod \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") "
Jul 06 21:04:58.769547 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.769293    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-dshm\") pod \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") "
Jul 06 21:04:58.769547 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.769421    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-model-cache\") pod \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") "
Jul 06 21:04:58.769547 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.769500    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-tmp-dir\") pod \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") "
Jul 06 21:04:58.769766 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.769563    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwg9w\" (UniqueName: \"kubernetes.io/projected/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-kube-api-access-dwg9w\") pod \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\" (UID: \"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9\") "
Jul 06 21:04:58.769766 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.769566    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-home" (OuterVolumeSpecName: "home") pod "38a7ea88-1fac-40b4-a316-a8bb2a47b0d9" (UID: "38a7ea88-1fac-40b4-a316-a8bb2a47b0d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:04:58.769766 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.769707    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "38a7ea88-1fac-40b4-a316-a8bb2a47b0d9" (UID: "38a7ea88-1fac-40b4-a316-a8bb2a47b0d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:04:58.769766 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.769711    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "38a7ea88-1fac-40b4-a316-a8bb2a47b0d9" (UID: "38a7ea88-1fac-40b4-a316-a8bb2a47b0d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:04:58.769932 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.769914    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-model-cache\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:04:58.769966 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.769938    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-tmp-dir\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:04:58.769966 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.769950    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-home\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:04:58.771556 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.771526    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "38a7ea88-1fac-40b4-a316-a8bb2a47b0d9" (UID: "38a7ea88-1fac-40b4-a316-a8bb2a47b0d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:04:58.771556 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.771528    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-dshm" (OuterVolumeSpecName: "dshm") pod "38a7ea88-1fac-40b4-a316-a8bb2a47b0d9" (UID: "38a7ea88-1fac-40b4-a316-a8bb2a47b0d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:04:58.771556 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.771528    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-kube-api-access-dwg9w" (OuterVolumeSpecName: "kube-api-access-dwg9w") pod "38a7ea88-1fac-40b4-a316-a8bb2a47b0d9" (UID: "38a7ea88-1fac-40b4-a316-a8bb2a47b0d9"). InnerVolumeSpecName "kube-api-access-dwg9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:04:58.816066 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.816028    2566 generic.go:356] "Generic (PLEG): container finished" podID="38a7ea88-1fac-40b4-a316-a8bb2a47b0d9" containerID="84765cf6c090bcaa73c0e023dabc36e3d3ff04dd93d81fc6cab9c05511e4999a" exitCode=0
Jul 06 21:04:58.816229 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.816078    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"
Jul 06 21:04:58.816229 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.816114    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc" event={"ID":"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9","Type":"ContainerDied","Data":"84765cf6c090bcaa73c0e023dabc36e3d3ff04dd93d81fc6cab9c05511e4999a"}
Jul 06 21:04:58.816229 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.816170    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc" event={"ID":"38a7ea88-1fac-40b4-a316-a8bb2a47b0d9","Type":"ContainerDied","Data":"79cd5dbc332f3986045e6a45c0eabe830b5dc924b6914d9de3c65b4aef7dadba"}
Jul 06 21:04:58.816229 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.816200    2566 scope.go:117] "RemoveContainer" containerID="84765cf6c090bcaa73c0e023dabc36e3d3ff04dd93d81fc6cab9c05511e4999a"
Jul 06 21:04:58.825845 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.825825    2566 scope.go:117] "RemoveContainer" containerID="84765cf6c090bcaa73c0e023dabc36e3d3ff04dd93d81fc6cab9c05511e4999a"
Jul 06 21:04:58.826098 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:04:58.826078    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84765cf6c090bcaa73c0e023dabc36e3d3ff04dd93d81fc6cab9c05511e4999a\": container with ID starting with 84765cf6c090bcaa73c0e023dabc36e3d3ff04dd93d81fc6cab9c05511e4999a not found: ID does not exist" containerID="84765cf6c090bcaa73c0e023dabc36e3d3ff04dd93d81fc6cab9c05511e4999a"
Jul 06 21:04:58.826153 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.826108    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84765cf6c090bcaa73c0e023dabc36e3d3ff04dd93d81fc6cab9c05511e4999a"} err="failed to get container status \"84765cf6c090bcaa73c0e023dabc36e3d3ff04dd93d81fc6cab9c05511e4999a\": rpc error: code = NotFound desc = could not find container \"84765cf6c090bcaa73c0e023dabc36e3d3ff04dd93d81fc6cab9c05511e4999a\": container with ID starting with 84765cf6c090bcaa73c0e023dabc36e3d3ff04dd93d81fc6cab9c05511e4999a not found: ID does not exist"
Jul 06 21:04:58.839266 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.839237    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"]
Jul 06 21:04:58.848122 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.848095    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc"]
Jul 06 21:04:58.871259 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.871232    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:04:58.871259 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.871255    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-dshm\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:04:58.871259 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:58.871264    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwg9w\" (UniqueName: \"kubernetes.io/projected/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9-kube-api-access-dwg9w\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:04:59.610515 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:04:59.610451    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879bc2cc" podUID="38a7ea88-1fac-40b4-a316-a8bb2a47b0d9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 06 21:05:00.002587 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:00.002554    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38a7ea88-1fac-40b4-a316-a8bb2a47b0d9" path="/var/lib/kubelet/pods/38a7ea88-1fac-40b4-a316-a8bb2a47b0d9/volumes"
Jul 06 21:05:17.330209 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.330129    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"]
Jul 06 21:05:17.330603 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.330497    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a4b0fe0-7e88-496b-aaec-57de430de975" containerName="main"
Jul 06 21:05:17.330603 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.330510    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a4b0fe0-7e88-496b-aaec-57de430de975" containerName="main"
Jul 06 21:05:17.330603 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.330523    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38a7ea88-1fac-40b4-a316-a8bb2a47b0d9" containerName="main"
Jul 06 21:05:17.330603 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.330528    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="38a7ea88-1fac-40b4-a316-a8bb2a47b0d9" containerName="main"
Jul 06 21:05:17.330738 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.330608    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a4b0fe0-7e88-496b-aaec-57de430de975" containerName="main"
Jul 06 21:05:17.330738 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.330616    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="38a7ea88-1fac-40b4-a316-a8bb2a47b0d9" containerName="main"
Jul 06 21:05:17.332579 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.332562    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.337031 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.337014    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g6qfk\""
Jul 06 21:05:17.338064 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.338041    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 06 21:05:17.338791 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.338770    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 06 21:05:17.338903 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.338834    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs\""
Jul 06 21:05:17.355673 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.355647    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"]
Jul 06 21:05:17.425268 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.425231    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzh4j\" (UniqueName: \"kubernetes.io/projected/20b30208-81f6-47fe-a4bd-bd062305f948-kube-api-access-rzh4j\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.425422 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.425272    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.425422 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.425342    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.425422 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.425372    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20b30208-81f6-47fe-a4bd-bd062305f948-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.425422 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.425408    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.425615 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.425425    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.526802 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.526760    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzh4j\" (UniqueName: \"kubernetes.io/projected/20b30208-81f6-47fe-a4bd-bd062305f948-kube-api-access-rzh4j\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.526802 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.526807    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.527085 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.526844    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.527085 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.526867    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20b30208-81f6-47fe-a4bd-bd062305f948-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.527085 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.526904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.527085 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.526922    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.527353 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.527328    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.527418 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.527357    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.527418 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.527397    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.529309 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.529286    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.529481 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.529443    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20b30208-81f6-47fe-a4bd-bd062305f948-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.536382 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.536344    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzh4j\" (UniqueName: \"kubernetes.io/projected/20b30208-81f6-47fe-a4bd-bd062305f948-kube-api-access-rzh4j\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.602919 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.602844    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"]
Jul 06 21:05:17.605942 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.605920    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:05:17.608448 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.608422    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-router-managed-test-llm-5b1e8f15-epp-sa-dockercfg-mlmxn\""
Jul 06 21:05:17.619215 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.619188    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"]
Jul 06 21:05:17.628359 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.628330    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gts4\" (UniqueName: \"kubernetes.io/projected/f9a7ba15-7245-4664-bede-d20c8bef1255-kube-api-access-4gts4\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb\" (UID: \"f9a7ba15-7245-4664-bede-d20c8bef1255\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:05:17.628496 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.628413    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9a7ba15-7245-4664-bede-d20c8bef1255-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb\" (UID: \"f9a7ba15-7245-4664-bede-d20c8bef1255\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:05:17.628496 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.628449    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f9a7ba15-7245-4664-bede-d20c8bef1255-tokenizer-uds\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb\" (UID: \"f9a7ba15-7245-4664-bede-d20c8bef1255\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:05:17.643059 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.643030    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:17.730626 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.729918    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gts4\" (UniqueName: \"kubernetes.io/projected/f9a7ba15-7245-4664-bede-d20c8bef1255-kube-api-access-4gts4\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb\" (UID: \"f9a7ba15-7245-4664-bede-d20c8bef1255\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:05:17.730626 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.729988    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9a7ba15-7245-4664-bede-d20c8bef1255-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb\" (UID: \"f9a7ba15-7245-4664-bede-d20c8bef1255\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:05:17.730626 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.730156    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f9a7ba15-7245-4664-bede-d20c8bef1255-tokenizer-uds\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb\" (UID: \"f9a7ba15-7245-4664-bede-d20c8bef1255\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:05:17.730626 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.730564    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f9a7ba15-7245-4664-bede-d20c8bef1255-tokenizer-uds\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb\" (UID: \"f9a7ba15-7245-4664-bede-d20c8bef1255\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:05:17.735103 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.733079    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9a7ba15-7245-4664-bede-d20c8bef1255-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb\" (UID: \"f9a7ba15-7245-4664-bede-d20c8bef1255\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:05:17.740300 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.740257    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gts4\" (UniqueName: \"kubernetes.io/projected/f9a7ba15-7245-4664-bede-d20c8bef1255-kube-api-access-4gts4\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb\" (UID: \"f9a7ba15-7245-4664-bede-d20c8bef1255\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:05:17.790402 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.790255    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"]
Jul 06 21:05:17.792636 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:05:17.792605    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20b30208_81f6_47fe_a4bd_bd062305f948.slice/crio-dc79e3ba88df56cc6b351c00bfa890e05151da026f0422eafa3fb0b72b559af6 WatchSource:0}: Error finding container dc79e3ba88df56cc6b351c00bfa890e05151da026f0422eafa3fb0b72b559af6: Status 404 returned error can't find the container with id dc79e3ba88df56cc6b351c00bfa890e05151da026f0422eafa3fb0b72b559af6
Jul 06 21:05:17.916684 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.916584    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:05:17.918523 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.918491    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn" event={"ID":"20b30208-81f6-47fe-a4bd-bd062305f948","Type":"ContainerStarted","Data":"764c5474a060622fcbc4bed2066d91ade009e0d6c9ce14b9228134827bfde718"}
Jul 06 21:05:17.918651 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.918546    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn" event={"ID":"20b30208-81f6-47fe-a4bd-bd062305f948","Type":"ContainerStarted","Data":"dc79e3ba88df56cc6b351c00bfa890e05151da026f0422eafa3fb0b72b559af6"}
Jul 06 21:05:17.949877 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:17.949714    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn" podStartSLOduration=0.949690488 podStartE2EDuration="949.690488ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:05:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:05:17.947881136 +0000 UTC m=+996.414122392" watchObservedRunningTime="2026-07-06 21:05:17.949690488 +0000 UTC m=+996.415931742"
Jul 06 21:05:18.058349 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:18.058324    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"]
Jul 06 21:05:18.060344 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:05:18.060320    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9a7ba15_7245_4664_bede_d20c8bef1255.slice/crio-bc151c0889582eb13cc7acd042decb8228319198222691de1c9f29670144fe49 WatchSource:0}: Error finding container bc151c0889582eb13cc7acd042decb8228319198222691de1c9f29670144fe49: Status 404 returned error can't find the container with id bc151c0889582eb13cc7acd042decb8228319198222691de1c9f29670144fe49
Jul 06 21:05:18.643785 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:18.643745    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:18.925531 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:18.925414    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb" event={"ID":"f9a7ba15-7245-4664-bede-d20c8bef1255","Type":"ContainerStarted","Data":"b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc"}
Jul 06 21:05:18.925531 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:18.925483    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb" event={"ID":"f9a7ba15-7245-4664-bede-d20c8bef1255","Type":"ContainerStarted","Data":"bc151c0889582eb13cc7acd042decb8228319198222691de1c9f29670144fe49"}
Jul 06 21:05:18.949438 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:18.949377    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb" podStartSLOduration=1.949360032 podStartE2EDuration="1.949360032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:05:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:05:18.945810166 +0000 UTC m=+997.412051419" watchObservedRunningTime="2026-07-06 21:05:18.949360032 +0000 UTC m=+997.415601519"
Jul 06 21:05:19.930042 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:19.930010    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:05:27.643399 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:27.643360    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:27.655607 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:27.655583    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:27.982185 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:27.982157    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:05:50.936270 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:05:50.936242    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:06:10.981829 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:10.981796    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"]
Jul 06 21:06:10.985414 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:10.985390    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:10.989387 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:10.989361    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-sa-dockercfg-pvqwq\""
Jul 06 21:06:10.989553 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:10.989362    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs\""
Jul 06 21:06:11.005060 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:11.005022    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-tokenizer-uds\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4\" (UID: \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:11.005219 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:11.005093    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4\" (UID: \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:11.005219 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:11.005117    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6wjz\" (UniqueName: \"kubernetes.io/projected/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-kube-api-access-w6wjz\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4\" (UID: \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:11.005300 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:11.005287    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"]
Jul 06 21:06:11.106379 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:11.106338    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4\" (UID: \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:11.106379 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:11.106375    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6wjz\" (UniqueName: \"kubernetes.io/projected/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-kube-api-access-w6wjz\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4\" (UID: \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:11.106660 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:11.106426    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-tokenizer-uds\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4\" (UID: \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:11.106867 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:11.106845    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-tokenizer-uds\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4\" (UID: \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:11.108970 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:11.108951    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4\" (UID: \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:11.122549 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:11.122516    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6wjz\" (UniqueName: \"kubernetes.io/projected/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-kube-api-access-w6wjz\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4\" (UID: \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:11.296083 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:11.296004    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:11.442415 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:11.442294    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"]
Jul 06 21:06:11.444739 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:06:11.444709    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51be5a4d_9c55_4e1c_b25c_f13b29b7f5b9.slice/crio-8bfd7af298cf328b59d5ef0c49b29a9628308d7b92ebdfa695e1a86ae274583c WatchSource:0}: Error finding container 8bfd7af298cf328b59d5ef0c49b29a9628308d7b92ebdfa695e1a86ae274583c: Status 404 returned error can't find the container with id 8bfd7af298cf328b59d5ef0c49b29a9628308d7b92ebdfa695e1a86ae274583c
Jul 06 21:06:12.215085 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:12.215049    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4" event={"ID":"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9","Type":"ContainerStarted","Data":"f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d"}
Jul 06 21:06:12.215484 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:12.215104    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4" event={"ID":"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9","Type":"ContainerStarted","Data":"8bfd7af298cf328b59d5ef0c49b29a9628308d7b92ebdfa695e1a86ae274583c"}
Jul 06 21:06:12.215484 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:12.215249    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:12.241474 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:12.241262    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4" podStartSLOduration=2.241243561 podStartE2EDuration="2.241243561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:06:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:06:12.237338728 +0000 UTC m=+1050.703579983" watchObservedRunningTime="2026-07-06 21:06:12.241243561 +0000 UTC m=+1050.707484815"
Jul 06 21:06:43.222334 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:43.222242    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:44.838818 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:44.838783    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"]
Jul 06 21:06:44.850730 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:06:44.850666    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:06:44.850730 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:06:44.850730 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:06:44.850730 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d" cmd=["/bin/sleep","15"]
Jul 06 21:06:44.851033 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:44.850740    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4" message="" err=<
Jul 06 21:06:44.851033 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:06:44.851033 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:06:44.851033 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:06:44.851033 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:06:44.850757    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:06:44.851033 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:06:44.851033 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:06:44.851033 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4" podUID="51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9" containerName="main" containerID="cri-o://f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d"
Jul 06 21:06:44.851033 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:44.850804    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4" podUID="51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9" containerName="main" containerID="cri-o://f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d" gracePeriod=60
Jul 06 21:06:45.106145 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.106117    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:45.213126 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.213084    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-tls-certs\") pod \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\" (UID: \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\") "
Jul 06 21:06:45.213126 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.213132    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-tokenizer-uds\") pod \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\" (UID: \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\") "
Jul 06 21:06:45.213390 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.213198    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6wjz\" (UniqueName: \"kubernetes.io/projected/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-kube-api-access-w6wjz\") pod \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\" (UID: \"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9\") "
Jul 06 21:06:45.213497 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.213438    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9" (UID: "51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:06:45.213582 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.213565    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:06:45.215267 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.215231    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9" (UID: "51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:06:45.215267 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.215243    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-kube-api-access-w6wjz" (OuterVolumeSpecName: "kube-api-access-w6wjz") pod "51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9" (UID: "51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9"). InnerVolumeSpecName "kube-api-access-w6wjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:06:45.314855 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.314820    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6wjz\" (UniqueName: \"kubernetes.io/projected/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-kube-api-access-w6wjz\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:06:45.314855 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.314850    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:06:45.396139 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.396041    2566 generic.go:356] "Generic (PLEG): container finished" podID="51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9" containerID="f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d" exitCode=0
Jul 06 21:06:45.396139 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.396084    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"
Jul 06 21:06:45.396139 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.396127    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4" event={"ID":"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9","Type":"ContainerDied","Data":"f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d"}
Jul 06 21:06:45.396378 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.396179    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4" event={"ID":"51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9","Type":"ContainerDied","Data":"8bfd7af298cf328b59d5ef0c49b29a9628308d7b92ebdfa695e1a86ae274583c"}
Jul 06 21:06:45.396378 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.396200    2566 scope.go:117] "RemoveContainer" containerID="f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d"
Jul 06 21:06:45.405820 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.405799    2566 scope.go:117] "RemoveContainer" containerID="f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d"
Jul 06 21:06:45.406116 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:06:45.406098    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d\": container with ID starting with f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d not found: ID does not exist" containerID="f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d"
Jul 06 21:06:45.406162 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.406123    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d"} err="failed to get container status \"f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d\": rpc error: code = NotFound desc = could not find container \"f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d\": container with ID starting with f31f2b5dc1977eccfbdca86c0a369769daebcc2994b2090e93b9cc3862c38a4d not found: ID does not exist"
Jul 06 21:06:45.428433 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.428397    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"]
Jul 06 21:06:45.430705 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:45.430683    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schevpfk4"]
Jul 06 21:06:46.002263 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:06:46.002229    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9" path="/var/lib/kubelet/pods/51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9/volumes"
Jul 06 21:07:04.591138 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.591106    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"]
Jul 06 21:07:04.597294 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.597264    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"]
Jul 06 21:07:04.600988 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:07:04.600948    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:07:04.600988 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:07:04.600988 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:07:04.600988 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc" cmd=["/bin/sleep","15"]
Jul 06 21:07:04.601237 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.601011    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb" message="" err=<
Jul 06 21:07:04.601237 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:07:04.601237 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:07:04.601237 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:07:04.601237 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:07:04.601032    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:07:04.601237 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:07:04.601237 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:07:04.601237 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb" podUID="f9a7ba15-7245-4664-bede-d20c8bef1255" containerName="main" containerID="cri-o://b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc"
Jul 06 21:07:04.601237 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.601081    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb" podUID="f9a7ba15-7245-4664-bede-d20c8bef1255" containerName="main" containerID="cri-o://b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc" gracePeriod=60
Jul 06 21:07:04.865527 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.865497    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:07:04.883281 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.883249    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f9a7ba15-7245-4664-bede-d20c8bef1255-tokenizer-uds\") pod \"f9a7ba15-7245-4664-bede-d20c8bef1255\" (UID: \"f9a7ba15-7245-4664-bede-d20c8bef1255\") "
Jul 06 21:07:04.883434 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.883351    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gts4\" (UniqueName: \"kubernetes.io/projected/f9a7ba15-7245-4664-bede-d20c8bef1255-kube-api-access-4gts4\") pod \"f9a7ba15-7245-4664-bede-d20c8bef1255\" (UID: \"f9a7ba15-7245-4664-bede-d20c8bef1255\") "
Jul 06 21:07:04.883434 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.883413    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9a7ba15-7245-4664-bede-d20c8bef1255-tls-certs\") pod \"f9a7ba15-7245-4664-bede-d20c8bef1255\" (UID: \"f9a7ba15-7245-4664-bede-d20c8bef1255\") "
Jul 06 21:07:04.883573 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.883504    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9a7ba15-7245-4664-bede-d20c8bef1255-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "f9a7ba15-7245-4664-bede-d20c8bef1255" (UID: "f9a7ba15-7245-4664-bede-d20c8bef1255"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:07:04.883661 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.883647    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f9a7ba15-7245-4664-bede-d20c8bef1255-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:07:04.885425 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.885360    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9a7ba15-7245-4664-bede-d20c8bef1255-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f9a7ba15-7245-4664-bede-d20c8bef1255" (UID: "f9a7ba15-7245-4664-bede-d20c8bef1255"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:07:04.885425 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.885404    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9a7ba15-7245-4664-bede-d20c8bef1255-kube-api-access-4gts4" (OuterVolumeSpecName: "kube-api-access-4gts4") pod "f9a7ba15-7245-4664-bede-d20c8bef1255" (UID: "f9a7ba15-7245-4664-bede-d20c8bef1255"). InnerVolumeSpecName "kube-api-access-4gts4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:07:04.984472 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.984420    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gts4\" (UniqueName: \"kubernetes.io/projected/f9a7ba15-7245-4664-bede-d20c8bef1255-kube-api-access-4gts4\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:07:04.984648 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:04.984483    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9a7ba15-7245-4664-bede-d20c8bef1255-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:07:05.509667 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:05.509630    2566 generic.go:356] "Generic (PLEG): container finished" podID="f9a7ba15-7245-4664-bede-d20c8bef1255" containerID="b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc" exitCode=0
Jul 06 21:07:05.509848 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:05.509684    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"
Jul 06 21:07:05.509848 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:05.509705    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb" event={"ID":"f9a7ba15-7245-4664-bede-d20c8bef1255","Type":"ContainerDied","Data":"b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc"}
Jul 06 21:07:05.509848 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:05.509749    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb" event={"ID":"f9a7ba15-7245-4664-bede-d20c8bef1255","Type":"ContainerDied","Data":"bc151c0889582eb13cc7acd042decb8228319198222691de1c9f29670144fe49"}
Jul 06 21:07:05.509848 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:05.509773    2566 scope.go:117] "RemoveContainer" containerID="b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc"
Jul 06 21:07:05.519416 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:05.519390    2566 scope.go:117] "RemoveContainer" containerID="b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc"
Jul 06 21:07:05.519735 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:07:05.519711    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc\": container with ID starting with b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc not found: ID does not exist" containerID="b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc"
Jul 06 21:07:05.519793 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:05.519748    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc"} err="failed to get container status \"b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc\": rpc error: code = NotFound desc = could not find container \"b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc\": container with ID starting with b8644b1f95ee07eb8e3896bdb3d4a1cfd2c4f4235446b8dd5d57a92639e584dc not found: ID does not exist"
Jul 06 21:07:05.535466 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:05.535420    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"]
Jul 06 21:07:05.538501 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:05.538478    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheffxrb"]
Jul 06 21:07:06.002863 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:06.002835    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9a7ba15-7245-4664-bede-d20c8bef1255" path="/var/lib/kubelet/pods/f9a7ba15-7245-4664-bede-d20c8bef1255/volumes"
Jul 06 21:07:19.620631 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.620556    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn" podUID="20b30208-81f6-47fe-a4bd-bd062305f948" containerName="main" containerID="cri-o://764c5474a060622fcbc4bed2066d91ade009e0d6c9ce14b9228134827bfde718" gracePeriod=45
Jul 06 21:07:19.876584 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.876518    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:07:19.915063 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.915029    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-dshm\") pod \"20b30208-81f6-47fe-a4bd-bd062305f948\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") "
Jul 06 21:07:19.915230 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.915088    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzh4j\" (UniqueName: \"kubernetes.io/projected/20b30208-81f6-47fe-a4bd-bd062305f948-kube-api-access-rzh4j\") pod \"20b30208-81f6-47fe-a4bd-bd062305f948\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") "
Jul 06 21:07:19.915230 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.915110    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20b30208-81f6-47fe-a4bd-bd062305f948-tls-certs\") pod \"20b30208-81f6-47fe-a4bd-bd062305f948\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") "
Jul 06 21:07:19.915230 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.915147    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-home\") pod \"20b30208-81f6-47fe-a4bd-bd062305f948\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") "
Jul 06 21:07:19.915230 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.915178    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-model-cache\") pod \"20b30208-81f6-47fe-a4bd-bd062305f948\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") "
Jul 06 21:07:19.915445 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.915247    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-tmp-dir\") pod \"20b30208-81f6-47fe-a4bd-bd062305f948\" (UID: \"20b30208-81f6-47fe-a4bd-bd062305f948\") "
Jul 06 21:07:19.915528 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.915448    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-home" (OuterVolumeSpecName: "home") pod "20b30208-81f6-47fe-a4bd-bd062305f948" (UID: "20b30208-81f6-47fe-a4bd-bd062305f948"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:07:19.915593 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.915553    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-model-cache" (OuterVolumeSpecName: "model-cache") pod "20b30208-81f6-47fe-a4bd-bd062305f948" (UID: "20b30208-81f6-47fe-a4bd-bd062305f948"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:07:19.915664 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.915645    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "20b30208-81f6-47fe-a4bd-bd062305f948" (UID: "20b30208-81f6-47fe-a4bd-bd062305f948"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:07:19.917367 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.917341    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20b30208-81f6-47fe-a4bd-bd062305f948-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "20b30208-81f6-47fe-a4bd-bd062305f948" (UID: "20b30208-81f6-47fe-a4bd-bd062305f948"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:07:19.917676 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.917656    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20b30208-81f6-47fe-a4bd-bd062305f948-kube-api-access-rzh4j" (OuterVolumeSpecName: "kube-api-access-rzh4j") pod "20b30208-81f6-47fe-a4bd-bd062305f948" (UID: "20b30208-81f6-47fe-a4bd-bd062305f948"). InnerVolumeSpecName "kube-api-access-rzh4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:07:19.917676 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:19.917666    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-dshm" (OuterVolumeSpecName: "dshm") pod "20b30208-81f6-47fe-a4bd-bd062305f948" (UID: "20b30208-81f6-47fe-a4bd-bd062305f948"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:07:20.016339 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.016317    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-model-cache\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:07:20.016339 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.016340    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-tmp-dir\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:07:20.016478 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.016348    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-dshm\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:07:20.016478 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.016357    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzh4j\" (UniqueName: \"kubernetes.io/projected/20b30208-81f6-47fe-a4bd-bd062305f948-kube-api-access-rzh4j\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:07:20.016478 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.016366    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20b30208-81f6-47fe-a4bd-bd062305f948-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:07:20.016478 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.016373    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20b30208-81f6-47fe-a4bd-bd062305f948-home\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:07:20.588694 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.588664    2566 generic.go:356] "Generic (PLEG): container finished" podID="20b30208-81f6-47fe-a4bd-bd062305f948" containerID="764c5474a060622fcbc4bed2066d91ade009e0d6c9ce14b9228134827bfde718" exitCode=0
Jul 06 21:07:20.588694 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.588700    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn" event={"ID":"20b30208-81f6-47fe-a4bd-bd062305f948","Type":"ContainerDied","Data":"764c5474a060622fcbc4bed2066d91ade009e0d6c9ce14b9228134827bfde718"}
Jul 06 21:07:20.588931 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.588714    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"
Jul 06 21:07:20.588931 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.588737    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn" event={"ID":"20b30208-81f6-47fe-a4bd-bd062305f948","Type":"ContainerDied","Data":"dc79e3ba88df56cc6b351c00bfa890e05151da026f0422eafa3fb0b72b559af6"}
Jul 06 21:07:20.588931 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.588758    2566 scope.go:117] "RemoveContainer" containerID="764c5474a060622fcbc4bed2066d91ade009e0d6c9ce14b9228134827bfde718"
Jul 06 21:07:20.597372 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.597352    2566 scope.go:117] "RemoveContainer" containerID="764c5474a060622fcbc4bed2066d91ade009e0d6c9ce14b9228134827bfde718"
Jul 06 21:07:20.597664 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:07:20.597643    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"764c5474a060622fcbc4bed2066d91ade009e0d6c9ce14b9228134827bfde718\": container with ID starting with 764c5474a060622fcbc4bed2066d91ade009e0d6c9ce14b9228134827bfde718 not found: ID does not exist" containerID="764c5474a060622fcbc4bed2066d91ade009e0d6c9ce14b9228134827bfde718"
Jul 06 21:07:20.597776 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.597671    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"764c5474a060622fcbc4bed2066d91ade009e0d6c9ce14b9228134827bfde718"} err="failed to get container status \"764c5474a060622fcbc4bed2066d91ade009e0d6c9ce14b9228134827bfde718\": rpc error: code = NotFound desc = could not find container \"764c5474a060622fcbc4bed2066d91ade009e0d6c9ce14b9228134827bfde718\": container with ID starting with 764c5474a060622fcbc4bed2066d91ade009e0d6c9ce14b9228134827bfde718 not found: ID does not exist"
Jul 06 21:07:20.626088 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.626058    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"]
Jul 06 21:07:20.632249 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:20.632220    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb456jdn"]
Jul 06 21:07:22.002229 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:22.002196    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20b30208-81f6-47fe-a4bd-bd062305f948" path="/var/lib/kubelet/pods/20b30208-81f6-47fe-a4bd-bd062305f948/volumes"
Jul 06 21:07:28.576370 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.576333    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"]
Jul 06 21:07:28.576893 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.576723    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20b30208-81f6-47fe-a4bd-bd062305f948" containerName="main"
Jul 06 21:07:28.576893 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.576736    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="20b30208-81f6-47fe-a4bd-bd062305f948" containerName="main"
Jul 06 21:07:28.576893 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.576745    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9a7ba15-7245-4664-bede-d20c8bef1255" containerName="main"
Jul 06 21:07:28.576893 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.576751    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9a7ba15-7245-4664-bede-d20c8bef1255" containerName="main"
Jul 06 21:07:28.576893 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.576767    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9" containerName="main"
Jul 06 21:07:28.576893 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.576773    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9" containerName="main"
Jul 06 21:07:28.576893 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.576830    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="20b30208-81f6-47fe-a4bd-bd062305f948" containerName="main"
Jul 06 21:07:28.576893 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.576836    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="51be5a4d-9c55-4e1c-b25c-f13b29b7f5b9" containerName="main"
Jul 06 21:07:28.576893 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.576842    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9a7ba15-7245-4664-bede-d20c8bef1255" containerName="main"
Jul 06 21:07:28.581521 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.581504    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.584318 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.584282    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 06 21:07:28.585092 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.585076    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g6qfk\""
Jul 06 21:07:28.585289 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.585270    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 06 21:07:28.585339 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.585326    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs\""
Jul 06 21:07:28.612632 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.612591    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"]
Jul 06 21:07:28.689952 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.689911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.690140 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.689974    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.690140 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.690047    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.690140 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.690072    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaa1e414-7418-4b42-b099-ef152df7c17b-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.690140 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.690122    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w977w\" (UniqueName: \"kubernetes.io/projected/aaa1e414-7418-4b42-b099-ef152df7c17b-kube-api-access-w977w\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.690282 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.690185    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.791599 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.791556    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.791599 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.791602    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaa1e414-7418-4b42-b099-ef152df7c17b-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.791869 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.791626    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w977w\" (UniqueName: \"kubernetes.io/projected/aaa1e414-7418-4b42-b099-ef152df7c17b-kube-api-access-w977w\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.791869 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.791657    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.791869 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.791691    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.791869 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.791766    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.792121 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.792027    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.792121 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.792048    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.792121 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.792094    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.793981 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.793955    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.794233 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.794215    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaa1e414-7418-4b42-b099-ef152df7c17b-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.821708 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.821671    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w977w\" (UniqueName: \"kubernetes.io/projected/aaa1e414-7418-4b42-b099-ef152df7c17b-kube-api-access-w977w\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.892013 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.891916    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:28.992246 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:28.992209    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"]
Jul 06 21:07:29.001075 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.001037    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:07:29.004665 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.004636    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-router-managed-test-llm-4b931143-epp-sa-dockercfg-p97cx\""
Jul 06 21:07:29.010087 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.010056    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"]
Jul 06 21:07:29.051042 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.051013    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"]
Jul 06 21:07:29.052821 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:07:29.052793    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa1e414_7418_4b42_b099_ef152df7c17b.slice/crio-570615236b3215cc3cb94b81586f31f001a84ef17153a78689997dcba2d975d8 WatchSource:0}: Error finding container 570615236b3215cc3cb94b81586f31f001a84ef17153a78689997dcba2d975d8: Status 404 returned error can't find the container with id 570615236b3215cc3cb94b81586f31f001a84ef17153a78689997dcba2d975d8
Jul 06 21:07:29.093760 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.093736    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-tokenizer-uds\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r\" (UID: \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:07:29.093884 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.093863    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r\" (UID: \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:07:29.093934 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.093902    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rw2j\" (UniqueName: \"kubernetes.io/projected/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-kube-api-access-7rw2j\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r\" (UID: \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:07:29.195056 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.194949    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r\" (UID: \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:07:29.195056 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.195000    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rw2j\" (UniqueName: \"kubernetes.io/projected/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-kube-api-access-7rw2j\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r\" (UID: \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:07:29.195056 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.195050    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-tokenizer-uds\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r\" (UID: \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:07:29.195545 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.195524    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-tokenizer-uds\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r\" (UID: \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:07:29.197504 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.197484    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r\" (UID: \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:07:29.205234 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.205215    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rw2j\" (UniqueName: \"kubernetes.io/projected/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-kube-api-access-7rw2j\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r\" (UID: \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:07:29.316355 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.316318    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:07:29.459488 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.459447    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"]
Jul 06 21:07:29.462097 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:07:29.462066    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb26270db_7c65_4d1c_8fde_1dbc2b5abef8.slice/crio-a14e5181d2082a3ec5e222e3887a012b40b9c0c83e820664d59cdb3c9824c3be WatchSource:0}: Error finding container a14e5181d2082a3ec5e222e3887a012b40b9c0c83e820664d59cdb3c9824c3be: Status 404 returned error can't find the container with id a14e5181d2082a3ec5e222e3887a012b40b9c0c83e820664d59cdb3c9824c3be
Jul 06 21:07:29.637632 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.637587    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r" event={"ID":"b26270db-7c65-4d1c-8fde-1dbc2b5abef8","Type":"ContainerStarted","Data":"066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413"}
Jul 06 21:07:29.638110 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.637653    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r" event={"ID":"b26270db-7c65-4d1c-8fde-1dbc2b5abef8","Type":"ContainerStarted","Data":"a14e5181d2082a3ec5e222e3887a012b40b9c0c83e820664d59cdb3c9824c3be"}
Jul 06 21:07:29.638110 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.637696    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:07:29.639663 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.639631    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8" event={"ID":"aaa1e414-7418-4b42-b099-ef152df7c17b","Type":"ContainerStarted","Data":"e869743c570f7c9e8ee8e47aa9041ab56bf2e95fba2e79c7fe47a42f8373d69e"}
Jul 06 21:07:29.639810 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.639683    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8" event={"ID":"aaa1e414-7418-4b42-b099-ef152df7c17b","Type":"ContainerStarted","Data":"570615236b3215cc3cb94b81586f31f001a84ef17153a78689997dcba2d975d8"}
Jul 06 21:07:29.659634 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.659578    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r" podStartSLOduration=1.659560089 podStartE2EDuration="1.659560089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:07:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:07:29.656599163 +0000 UTC m=+1128.122840430" watchObservedRunningTime="2026-07-06 21:07:29.659560089 +0000 UTC m=+1128.125801344"
Jul 06 21:07:29.682079 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.682018    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8" podStartSLOduration=1.6820008560000002 podStartE2EDuration="1.682000856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:07:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:07:29.678768142 +0000 UTC m=+1128.145009397" watchObservedRunningTime="2026-07-06 21:07:29.682000856 +0000 UTC m=+1128.148242109"
Jul 06 21:07:29.892371 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:29.892327    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:38.892436 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:38.892399    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:38.905055 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:38.905029    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:07:39.703872 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:07:39.703840    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:08:00.647070 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:08:00.647039    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:09:40.973657 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:40.973429    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"]
Jul 06 21:09:40.976791 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:40.976755    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"]
Jul 06 21:09:40.986654 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:09:40.986609    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:09:40.986654 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:09:40.986654 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:09:40.986654 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413" cmd=["/bin/sleep","15"]
Jul 06 21:09:40.986945 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:40.986675    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r" message="" err=<
Jul 06 21:09:40.986945 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:09:40.986945 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:09:40.986945 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:09:40.986945 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:09:40.986692    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:09:40.986945 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:09:40.986945 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:09:40.986945 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r" podUID="b26270db-7c65-4d1c-8fde-1dbc2b5abef8" containerName="main" containerID="cri-o://066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413"
Jul 06 21:09:40.986945 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:40.986739    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r" podUID="b26270db-7c65-4d1c-8fde-1dbc2b5abef8" containerName="main" containerID="cri-o://066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413" gracePeriod=60
Jul 06 21:09:41.236198 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.236171    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:09:41.315954 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.315913    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-tls-certs\") pod \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\" (UID: \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\") "
Jul 06 21:09:41.315954 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.315956    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rw2j\" (UniqueName: \"kubernetes.io/projected/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-kube-api-access-7rw2j\") pod \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\" (UID: \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\") "
Jul 06 21:09:41.316165 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.316001    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-tokenizer-uds\") pod \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\" (UID: \"b26270db-7c65-4d1c-8fde-1dbc2b5abef8\") "
Jul 06 21:09:41.316379 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.316344    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "b26270db-7c65-4d1c-8fde-1dbc2b5abef8" (UID: "b26270db-7c65-4d1c-8fde-1dbc2b5abef8"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:09:41.318108 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.318080    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-kube-api-access-7rw2j" (OuterVolumeSpecName: "kube-api-access-7rw2j") pod "b26270db-7c65-4d1c-8fde-1dbc2b5abef8" (UID: "b26270db-7c65-4d1c-8fde-1dbc2b5abef8"). InnerVolumeSpecName "kube-api-access-7rw2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:09:41.318190 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.318086    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b26270db-7c65-4d1c-8fde-1dbc2b5abef8" (UID: "b26270db-7c65-4d1c-8fde-1dbc2b5abef8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:09:41.339530 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.339499    2566 generic.go:356] "Generic (PLEG): container finished" podID="b26270db-7c65-4d1c-8fde-1dbc2b5abef8" containerID="066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413" exitCode=0
Jul 06 21:09:41.339707 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.339537    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r" event={"ID":"b26270db-7c65-4d1c-8fde-1dbc2b5abef8","Type":"ContainerDied","Data":"066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413"}
Jul 06 21:09:41.339707 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.339560    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"
Jul 06 21:09:41.339707 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.339582    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r" event={"ID":"b26270db-7c65-4d1c-8fde-1dbc2b5abef8","Type":"ContainerDied","Data":"a14e5181d2082a3ec5e222e3887a012b40b9c0c83e820664d59cdb3c9824c3be"}
Jul 06 21:09:41.339707 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.339606    2566 scope.go:117] "RemoveContainer" containerID="066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413"
Jul 06 21:09:41.357083 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.357061    2566 scope.go:117] "RemoveContainer" containerID="066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413"
Jul 06 21:09:41.357395 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:09:41.357372    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413\": container with ID starting with 066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413 not found: ID does not exist" containerID="066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413"
Jul 06 21:09:41.357501 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.357402    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413"} err="failed to get container status \"066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413\": rpc error: code = NotFound desc = could not find container \"066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413\": container with ID starting with 066378ed8cf8911ff5a7779b3a037786a30f49a7899cdf9375a9e936f69b4413 not found: ID does not exist"
Jul 06 21:09:41.366491 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.366464    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"]
Jul 06 21:09:41.372901 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.372875    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schebs46r"]
Jul 06 21:09:41.417093 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.417055    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:09:41.417093 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.417085    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rw2j\" (UniqueName: \"kubernetes.io/projected/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-kube-api-access-7rw2j\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:09:41.417313 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:41.417119    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b26270db-7c65-4d1c-8fde-1dbc2b5abef8-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:09:42.002829 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:42.002798    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b26270db-7c65-4d1c-8fde-1dbc2b5abef8" path="/var/lib/kubelet/pods/b26270db-7c65-4d1c-8fde-1dbc2b5abef8/volumes"
Jul 06 21:09:55.996278 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:55.996215    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8" podUID="aaa1e414-7418-4b42-b099-ef152df7c17b" containerName="main" containerID="cri-o://e869743c570f7c9e8ee8e47aa9041ab56bf2e95fba2e79c7fe47a42f8373d69e" gracePeriod=45
Jul 06 21:09:56.243877 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.243850    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:09:56.347263 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.347170    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-tmp-dir\") pod \"aaa1e414-7418-4b42-b099-ef152df7c17b\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") "
Jul 06 21:09:56.347263 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.347242    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-dshm\") pod \"aaa1e414-7418-4b42-b099-ef152df7c17b\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") "
Jul 06 21:09:56.347526 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.347269    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-model-cache\") pod \"aaa1e414-7418-4b42-b099-ef152df7c17b\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") "
Jul 06 21:09:56.347526 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.347312    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaa1e414-7418-4b42-b099-ef152df7c17b-tls-certs\") pod \"aaa1e414-7418-4b42-b099-ef152df7c17b\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") "
Jul 06 21:09:56.347526 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.347331    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-home\") pod \"aaa1e414-7418-4b42-b099-ef152df7c17b\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") "
Jul 06 21:09:56.347526 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.347367    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w977w\" (UniqueName: \"kubernetes.io/projected/aaa1e414-7418-4b42-b099-ef152df7c17b-kube-api-access-w977w\") pod \"aaa1e414-7418-4b42-b099-ef152df7c17b\" (UID: \"aaa1e414-7418-4b42-b099-ef152df7c17b\") "
Jul 06 21:09:56.347765 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.347547    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aaa1e414-7418-4b42-b099-ef152df7c17b" (UID: "aaa1e414-7418-4b42-b099-ef152df7c17b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:09:56.347765 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.347696    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-home" (OuterVolumeSpecName: "home") pod "aaa1e414-7418-4b42-b099-ef152df7c17b" (UID: "aaa1e414-7418-4b42-b099-ef152df7c17b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:09:56.347765 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.347706    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-model-cache" (OuterVolumeSpecName: "model-cache") pod "aaa1e414-7418-4b42-b099-ef152df7c17b" (UID: "aaa1e414-7418-4b42-b099-ef152df7c17b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:09:56.347765 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.347722    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-tmp-dir\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:09:56.349429 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.349400    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaa1e414-7418-4b42-b099-ef152df7c17b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aaa1e414-7418-4b42-b099-ef152df7c17b" (UID: "aaa1e414-7418-4b42-b099-ef152df7c17b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:09:56.349555 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.349492    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaa1e414-7418-4b42-b099-ef152df7c17b-kube-api-access-w977w" (OuterVolumeSpecName: "kube-api-access-w977w") pod "aaa1e414-7418-4b42-b099-ef152df7c17b" (UID: "aaa1e414-7418-4b42-b099-ef152df7c17b"). InnerVolumeSpecName "kube-api-access-w977w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:09:56.349555 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.349502    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-dshm" (OuterVolumeSpecName: "dshm") pod "aaa1e414-7418-4b42-b099-ef152df7c17b" (UID: "aaa1e414-7418-4b42-b099-ef152df7c17b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:09:56.428324 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.428286    2566 generic.go:356] "Generic (PLEG): container finished" podID="aaa1e414-7418-4b42-b099-ef152df7c17b" containerID="e869743c570f7c9e8ee8e47aa9041ab56bf2e95fba2e79c7fe47a42f8373d69e" exitCode=0
Jul 06 21:09:56.428545 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.428338    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"
Jul 06 21:09:56.428545 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.428370    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8" event={"ID":"aaa1e414-7418-4b42-b099-ef152df7c17b","Type":"ContainerDied","Data":"e869743c570f7c9e8ee8e47aa9041ab56bf2e95fba2e79c7fe47a42f8373d69e"}
Jul 06 21:09:56.428545 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.428421    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8" event={"ID":"aaa1e414-7418-4b42-b099-ef152df7c17b","Type":"ContainerDied","Data":"570615236b3215cc3cb94b81586f31f001a84ef17153a78689997dcba2d975d8"}
Jul 06 21:09:56.428545 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.428442    2566 scope.go:117] "RemoveContainer" containerID="e869743c570f7c9e8ee8e47aa9041ab56bf2e95fba2e79c7fe47a42f8373d69e"
Jul 06 21:09:56.437434 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.437416    2566 scope.go:117] "RemoveContainer" containerID="e869743c570f7c9e8ee8e47aa9041ab56bf2e95fba2e79c7fe47a42f8373d69e"
Jul 06 21:09:56.437751 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:09:56.437728    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e869743c570f7c9e8ee8e47aa9041ab56bf2e95fba2e79c7fe47a42f8373d69e\": container with ID starting with e869743c570f7c9e8ee8e47aa9041ab56bf2e95fba2e79c7fe47a42f8373d69e not found: ID does not exist" containerID="e869743c570f7c9e8ee8e47aa9041ab56bf2e95fba2e79c7fe47a42f8373d69e"
Jul 06 21:09:56.437826 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.437763    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e869743c570f7c9e8ee8e47aa9041ab56bf2e95fba2e79c7fe47a42f8373d69e"} err="failed to get container status \"e869743c570f7c9e8ee8e47aa9041ab56bf2e95fba2e79c7fe47a42f8373d69e\": rpc error: code = NotFound desc = could not find container \"e869743c570f7c9e8ee8e47aa9041ab56bf2e95fba2e79c7fe47a42f8373d69e\": container with ID starting with e869743c570f7c9e8ee8e47aa9041ab56bf2e95fba2e79c7fe47a42f8373d69e not found: ID does not exist"
Jul 06 21:09:56.448309 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.448276    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaa1e414-7418-4b42-b099-ef152df7c17b-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:09:56.448309 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.448297    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-home\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:09:56.448309 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.448307    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w977w\" (UniqueName: \"kubernetes.io/projected/aaa1e414-7418-4b42-b099-ef152df7c17b-kube-api-access-w977w\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:09:56.448567 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.448315    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-dshm\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:09:56.448567 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.448326    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaa1e414-7418-4b42-b099-ef152df7c17b-model-cache\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:09:56.459270 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.459235    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"]
Jul 06 21:09:56.469547 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:56.469520    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-vv8n8"]
Jul 06 21:09:58.002025 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:09:58.001991    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaa1e414-7418-4b42-b099-ef152df7c17b" path="/var/lib/kubelet/pods/aaa1e414-7418-4b42-b099-ef152df7c17b/volumes"
Jul 06 21:10:16.190576 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.190536    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"]
Jul 06 21:10:16.191034 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.190895    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaa1e414-7418-4b42-b099-ef152df7c17b" containerName="main"
Jul 06 21:10:16.191034 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.190907    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaa1e414-7418-4b42-b099-ef152df7c17b" containerName="main"
Jul 06 21:10:16.191034 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.190917    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b26270db-7c65-4d1c-8fde-1dbc2b5abef8" containerName="main"
Jul 06 21:10:16.191034 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.190923    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b26270db-7c65-4d1c-8fde-1dbc2b5abef8" containerName="main"
Jul 06 21:10:16.191034 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.190983    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b26270db-7c65-4d1c-8fde-1dbc2b5abef8" containerName="main"
Jul 06 21:10:16.191034 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.190992    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaa1e414-7418-4b42-b099-ef152df7c17b" containerName="main"
Jul 06 21:10:16.194188 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.194169    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.196434 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.196408    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 06 21:10:16.196577 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.196561    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 06 21:10:16.197501 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.197483    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs\""
Jul 06 21:10:16.197601 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.197521    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 06 21:10:16.197601 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.197537    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g6qfk\""
Jul 06 21:10:16.197601 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.197537    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-with-7ca60146-epp-sa-dockercfg-hm72n\""
Jul 06 21:10:16.203641 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.203611    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tv8d\" (UniqueName: \"kubernetes.io/projected/46161428-635c-4677-bbdd-89c92af7b5b7-kube-api-access-6tv8d\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.203770 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.203644    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/46161428-635c-4677-bbdd-89c92af7b5b7-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.203770 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.203671    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46161428-635c-4677-bbdd-89c92af7b5b7-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.203770 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.203720    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46161428-635c-4677-bbdd-89c92af7b5b7-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.213149 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.213124    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"]
Jul 06 21:10:16.304344 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.304310    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46161428-635c-4677-bbdd-89c92af7b5b7-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.304560 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.304386    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tv8d\" (UniqueName: \"kubernetes.io/projected/46161428-635c-4677-bbdd-89c92af7b5b7-kube-api-access-6tv8d\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.304560 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.304412    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/46161428-635c-4677-bbdd-89c92af7b5b7-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.304560 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.304516    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46161428-635c-4677-bbdd-89c92af7b5b7-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.304882 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.304856    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46161428-635c-4677-bbdd-89c92af7b5b7-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.304882 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.304866    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/46161428-635c-4677-bbdd-89c92af7b5b7-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.306966 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.306945    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46161428-635c-4677-bbdd-89c92af7b5b7-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.321865 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.321839    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tv8d\" (UniqueName: \"kubernetes.io/projected/46161428-635c-4677-bbdd-89c92af7b5b7-kube-api-access-6tv8d\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.504218 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.504187    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:16.648478 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:16.648434    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"]
Jul 06 21:10:16.650214 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:10:16.650187    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46161428_635c_4677_bbdd_89c92af7b5b7.slice/crio-974fd3270c371eb75a4004c1252243edc02a81f40582a8cf33ea3dba2f2e316f WatchSource:0}: Error finding container 974fd3270c371eb75a4004c1252243edc02a81f40582a8cf33ea3dba2f2e316f: Status 404 returned error can't find the container with id 974fd3270c371eb75a4004c1252243edc02a81f40582a8cf33ea3dba2f2e316f
Jul 06 21:10:17.546337 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:17.546300    2566 generic.go:356] "Generic (PLEG): container finished" podID="46161428-635c-4677-bbdd-89c92af7b5b7" containerID="58d63c8864a741d767343616c78bdb2f5e19832b91bffd9cc79468f27adc77c8" exitCode=0
Jul 06 21:10:17.546805 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:17.546384    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7" event={"ID":"46161428-635c-4677-bbdd-89c92af7b5b7","Type":"ContainerDied","Data":"58d63c8864a741d767343616c78bdb2f5e19832b91bffd9cc79468f27adc77c8"}
Jul 06 21:10:17.546805 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:17.546438    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7" event={"ID":"46161428-635c-4677-bbdd-89c92af7b5b7","Type":"ContainerStarted","Data":"974fd3270c371eb75a4004c1252243edc02a81f40582a8cf33ea3dba2f2e316f"}
Jul 06 21:10:18.555096 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:18.555054    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7" event={"ID":"46161428-635c-4677-bbdd-89c92af7b5b7","Type":"ContainerStarted","Data":"fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1"}
Jul 06 21:10:18.555614 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:18.555177    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:49.563735 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:49.563704    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:10:49.589449 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:10:49.589390    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7" podStartSLOduration=32.890948469 podStartE2EDuration="33.589376023s" totalImagesPullingTime="0s" totalInitContainerRuntime="698.427554ms" isStatefulPod=false podCreationTimestamp="2026-07-06 21:10:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:10:18.592832419 +0000 UTC m=+1297.059073673" watchObservedRunningTime="2026-07-06 21:10:49.589376023 +0000 UTC m=+1328.055617276"
Jul 06 21:12:12.315087 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.315048    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"]
Jul 06 21:12:12.324293 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:12:12.324251    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:12:12.324293 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:12:12.324293 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:12:12.324293 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1" cmd=["/bin/sleep","15"]
Jul 06 21:12:12.324653 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.324312    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7" message="" err=<
Jul 06 21:12:12.324653 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:12:12.324653 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:12:12.324653 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:12:12.324653 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:12:12.324326    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:12:12.324653 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:12:12.324653 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:12:12.324653 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7" podUID="46161428-635c-4677-bbdd-89c92af7b5b7" containerName="main" containerID="cri-o://fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1"
Jul 06 21:12:12.324653 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.324363    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7" podUID="46161428-635c-4677-bbdd-89c92af7b5b7" containerName="main" containerID="cri-o://fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1" gracePeriod=60
Jul 06 21:12:12.583320 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.583286    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:12:12.708965 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.708929    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46161428-635c-4677-bbdd-89c92af7b5b7-tls-certs\") pod \"46161428-635c-4677-bbdd-89c92af7b5b7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") "
Jul 06 21:12:12.709139 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.708972    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/46161428-635c-4677-bbdd-89c92af7b5b7-tokenizer-uds\") pod \"46161428-635c-4677-bbdd-89c92af7b5b7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") "
Jul 06 21:12:12.709139 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.709006    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tv8d\" (UniqueName: \"kubernetes.io/projected/46161428-635c-4677-bbdd-89c92af7b5b7-kube-api-access-6tv8d\") pod \"46161428-635c-4677-bbdd-89c92af7b5b7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") "
Jul 06 21:12:12.709139 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.709090    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46161428-635c-4677-bbdd-89c92af7b5b7-kserve-provision-location\") pod \"46161428-635c-4677-bbdd-89c92af7b5b7\" (UID: \"46161428-635c-4677-bbdd-89c92af7b5b7\") "
Jul 06 21:12:12.709265 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.709227    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46161428-635c-4677-bbdd-89c92af7b5b7-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "46161428-635c-4677-bbdd-89c92af7b5b7" (UID: "46161428-635c-4677-bbdd-89c92af7b5b7"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:12:12.709313 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.709299    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/46161428-635c-4677-bbdd-89c92af7b5b7-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:12:12.709830 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.709802    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46161428-635c-4677-bbdd-89c92af7b5b7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "46161428-635c-4677-bbdd-89c92af7b5b7" (UID: "46161428-635c-4677-bbdd-89c92af7b5b7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:12:12.711081 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.711059    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46161428-635c-4677-bbdd-89c92af7b5b7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "46161428-635c-4677-bbdd-89c92af7b5b7" (UID: "46161428-635c-4677-bbdd-89c92af7b5b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:12:12.711139 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.711124    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46161428-635c-4677-bbdd-89c92af7b5b7-kube-api-access-6tv8d" (OuterVolumeSpecName: "kube-api-access-6tv8d") pod "46161428-635c-4677-bbdd-89c92af7b5b7" (UID: "46161428-635c-4677-bbdd-89c92af7b5b7"). InnerVolumeSpecName "kube-api-access-6tv8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:12:12.809791 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.809754    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46161428-635c-4677-bbdd-89c92af7b5b7-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:12:12.809791 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.809784    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tv8d\" (UniqueName: \"kubernetes.io/projected/46161428-635c-4677-bbdd-89c92af7b5b7-kube-api-access-6tv8d\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:12:12.809791 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:12.809795    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46161428-635c-4677-bbdd-89c92af7b5b7-kserve-provision-location\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:12:13.159396 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:13.159359    2566 generic.go:356] "Generic (PLEG): container finished" podID="46161428-635c-4677-bbdd-89c92af7b5b7" containerID="fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1" exitCode=0
Jul 06 21:12:13.159572 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:13.159411    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"
Jul 06 21:12:13.159572 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:13.159445    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7" event={"ID":"46161428-635c-4677-bbdd-89c92af7b5b7","Type":"ContainerDied","Data":"fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1"}
Jul 06 21:12:13.159572 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:13.159525    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7" event={"ID":"46161428-635c-4677-bbdd-89c92af7b5b7","Type":"ContainerDied","Data":"974fd3270c371eb75a4004c1252243edc02a81f40582a8cf33ea3dba2f2e316f"}
Jul 06 21:12:13.159572 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:13.159548    2566 scope.go:117] "RemoveContainer" containerID="fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1"
Jul 06 21:12:13.168864 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:13.168848    2566 scope.go:117] "RemoveContainer" containerID="58d63c8864a741d767343616c78bdb2f5e19832b91bffd9cc79468f27adc77c8"
Jul 06 21:12:13.176656 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:13.176637    2566 scope.go:117] "RemoveContainer" containerID="fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1"
Jul 06 21:12:13.176901 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:12:13.176881    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1\": container with ID starting with fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1 not found: ID does not exist" containerID="fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1"
Jul 06 21:12:13.176964 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:13.176911    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1"} err="failed to get container status \"fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1\": rpc error: code = NotFound desc = could not find container \"fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1\": container with ID starting with fef120920adf8d61a4206c012ac6fc693f35a87178b5627a14187fa2345c72b1 not found: ID does not exist"
Jul 06 21:12:13.176964 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:13.176927    2566 scope.go:117] "RemoveContainer" containerID="58d63c8864a741d767343616c78bdb2f5e19832b91bffd9cc79468f27adc77c8"
Jul 06 21:12:13.177170 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:12:13.177148    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58d63c8864a741d767343616c78bdb2f5e19832b91bffd9cc79468f27adc77c8\": container with ID starting with 58d63c8864a741d767343616c78bdb2f5e19832b91bffd9cc79468f27adc77c8 not found: ID does not exist" containerID="58d63c8864a741d767343616c78bdb2f5e19832b91bffd9cc79468f27adc77c8"
Jul 06 21:12:13.177214 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:13.177178    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58d63c8864a741d767343616c78bdb2f5e19832b91bffd9cc79468f27adc77c8"} err="failed to get container status \"58d63c8864a741d767343616c78bdb2f5e19832b91bffd9cc79468f27adc77c8\": rpc error: code = NotFound desc = could not find container \"58d63c8864a741d767343616c78bdb2f5e19832b91bffd9cc79468f27adc77c8\": container with ID starting with 58d63c8864a741d767343616c78bdb2f5e19832b91bffd9cc79468f27adc77c8 not found: ID does not exist"
Jul 06 21:12:13.183313 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:13.183289    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"]
Jul 06 21:12:13.185548 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:13.185528    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schev6vd7"]
Jul 06 21:12:14.003507 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:14.003473    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46161428-635c-4677-bbdd-89c92af7b5b7" path="/var/lib/kubelet/pods/46161428-635c-4677-bbdd-89c92af7b5b7/volumes"
Jul 06 21:12:35.664317 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.664235    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"]
Jul 06 21:12:35.665128 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.664934    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46161428-635c-4677-bbdd-89c92af7b5b7" containerName="storage-initializer"
Jul 06 21:12:35.665128 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.664958    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="46161428-635c-4677-bbdd-89c92af7b5b7" containerName="storage-initializer"
Jul 06 21:12:35.665128 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.664989    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46161428-635c-4677-bbdd-89c92af7b5b7" containerName="main"
Jul 06 21:12:35.665128 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.664998    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="46161428-635c-4677-bbdd-89c92af7b5b7" containerName="main"
Jul 06 21:12:35.665128 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.665083    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="46161428-635c-4677-bbdd-89c92af7b5b7" containerName="main"
Jul 06 21:12:35.669867 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.669846    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:35.672115 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.672091    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 06 21:12:35.672233 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.672134    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 06 21:12:35.672301 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.672287    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g6qfk\""
Jul 06 21:12:35.673105 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.673083    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 06 21:12:35.673178 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.673147    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs\""
Jul 06 21:12:35.673241 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.673167    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-with-ba4d693a-epp-sa-dockercfg-f6cz9\""
Jul 06 21:12:35.677061 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.677043    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"]
Jul 06 21:12:35.686291 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.686261    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptlfs\" (UniqueName: \"kubernetes.io/projected/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-kube-api-access-ptlfs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:35.686394 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.686304    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:35.686394 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.686366    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:35.686394 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.686388    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:35.786847 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.786809    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:35.787030 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.786867    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:35.787030 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.786889    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:35.787030 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.786947    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptlfs\" (UniqueName: \"kubernetes.io/projected/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-kube-api-access-ptlfs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:35.787230 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.787209    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:35.787288 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.787238    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:35.789377 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.789353    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:35.794806 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.794783    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptlfs\" (UniqueName: \"kubernetes.io/projected/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-kube-api-access-ptlfs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:35.980896 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:35.980864    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:36.130488 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:36.130423    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"]
Jul 06 21:12:36.132772 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:12:36.132735    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fbdae15_4752_42c2_9828_cfec3d4d5ab2.slice/crio-178c9f41687cb37b3df8b20050a3b8672846e0e40b0de9876d53a94e73ff5ac2 WatchSource:0}: Error finding container 178c9f41687cb37b3df8b20050a3b8672846e0e40b0de9876d53a94e73ff5ac2: Status 404 returned error can't find the container with id 178c9f41687cb37b3df8b20050a3b8672846e0e40b0de9876d53a94e73ff5ac2
Jul 06 21:12:36.285133 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:36.285043    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr" event={"ID":"2fbdae15-4752-42c2-9828-cfec3d4d5ab2","Type":"ContainerStarted","Data":"bed077cc0cb5d025783dc2840c7c46b5b9c416d95267b9e75d6833c0f0c3afee"}
Jul 06 21:12:36.285133 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:36.285103    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr" event={"ID":"2fbdae15-4752-42c2-9828-cfec3d4d5ab2","Type":"ContainerStarted","Data":"178c9f41687cb37b3df8b20050a3b8672846e0e40b0de9876d53a94e73ff5ac2"}
Jul 06 21:12:37.291881 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:37.291845    2566 generic.go:356] "Generic (PLEG): container finished" podID="2fbdae15-4752-42c2-9828-cfec3d4d5ab2" containerID="bed077cc0cb5d025783dc2840c7c46b5b9c416d95267b9e75d6833c0f0c3afee" exitCode=0
Jul 06 21:12:37.292311 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:37.291892    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr" event={"ID":"2fbdae15-4752-42c2-9828-cfec3d4d5ab2","Type":"ContainerDied","Data":"bed077cc0cb5d025783dc2840c7c46b5b9c416d95267b9e75d6833c0f0c3afee"}
Jul 06 21:12:38.299659 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:38.299618    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr" event={"ID":"2fbdae15-4752-42c2-9828-cfec3d4d5ab2","Type":"ContainerStarted","Data":"ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214"}
Jul 06 21:12:38.300142 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:38.299707    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:12:38.320653 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:12:38.320584    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr" podStartSLOduration=2.6371772570000003 podStartE2EDuration="3.320567627s" totalImagesPullingTime="0s" totalInitContainerRuntime="683.39037ms" isStatefulPod=false podCreationTimestamp="2026-07-06 21:12:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:12:38.317985037 +0000 UTC m=+1436.784226290" watchObservedRunningTime="2026-07-06 21:12:38.320567627 +0000 UTC m=+1436.786808887"
Jul 06 21:13:09.306577 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:13:09.306547    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:14:07.083872 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.083780    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"]
Jul 06 21:14:07.093989 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:14:07.093943    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:14:07.093989 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:14:07.093989 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:14:07.093989 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214" cmd=["/bin/sleep","15"]
Jul 06 21:14:07.094280 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.094006    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr" message="" err=<
Jul 06 21:14:07.094280 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:14:07.094280 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:14:07.094280 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:14:07.094280 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:14:07.094019    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:14:07.094280 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:14:07.094280 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:14:07.094280 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr" podUID="2fbdae15-4752-42c2-9828-cfec3d4d5ab2" containerName="main" containerID="cri-o://ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214"
Jul 06 21:14:07.094280 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.094070    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr" podUID="2fbdae15-4752-42c2-9828-cfec3d4d5ab2" containerName="main" containerID="cri-o://ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214" gracePeriod=60
Jul 06 21:14:07.355241 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.355215    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:14:07.463650 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.463613    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-tokenizer-uds\") pod \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") "
Jul 06 21:14:07.463850 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.463677    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-tls-certs\") pod \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") "
Jul 06 21:14:07.463850 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.463727    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-kserve-provision-location\") pod \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") "
Jul 06 21:14:07.463850 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.463755    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptlfs\" (UniqueName: \"kubernetes.io/projected/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-kube-api-access-ptlfs\") pod \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\" (UID: \"2fbdae15-4752-42c2-9828-cfec3d4d5ab2\") "
Jul 06 21:14:07.464015 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.463948    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "2fbdae15-4752-42c2-9828-cfec3d4d5ab2" (UID: "2fbdae15-4752-42c2-9828-cfec3d4d5ab2"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:14:07.464056 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.464040    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:14:07.464490 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.464442    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "2fbdae15-4752-42c2-9828-cfec3d4d5ab2" (UID: "2fbdae15-4752-42c2-9828-cfec3d4d5ab2"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:14:07.466018 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.465989    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2fbdae15-4752-42c2-9828-cfec3d4d5ab2" (UID: "2fbdae15-4752-42c2-9828-cfec3d4d5ab2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:14:07.466146 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.466122    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-kube-api-access-ptlfs" (OuterVolumeSpecName: "kube-api-access-ptlfs") pod "2fbdae15-4752-42c2-9828-cfec3d4d5ab2" (UID: "2fbdae15-4752-42c2-9828-cfec3d4d5ab2"). InnerVolumeSpecName "kube-api-access-ptlfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:14:07.565040 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.564998    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:14:07.565040 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.565030    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-kserve-provision-location\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:14:07.565040 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.565040    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptlfs\" (UniqueName: \"kubernetes.io/projected/2fbdae15-4752-42c2-9828-cfec3d4d5ab2-kube-api-access-ptlfs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:14:07.780744 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.780707    2566 generic.go:356] "Generic (PLEG): container finished" podID="2fbdae15-4752-42c2-9828-cfec3d4d5ab2" containerID="ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214" exitCode=0
Jul 06 21:14:07.780926 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.780767    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"
Jul 06 21:14:07.780926 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.780771    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr" event={"ID":"2fbdae15-4752-42c2-9828-cfec3d4d5ab2","Type":"ContainerDied","Data":"ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214"}
Jul 06 21:14:07.780926 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.780880    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr" event={"ID":"2fbdae15-4752-42c2-9828-cfec3d4d5ab2","Type":"ContainerDied","Data":"178c9f41687cb37b3df8b20050a3b8672846e0e40b0de9876d53a94e73ff5ac2"}
Jul 06 21:14:07.780926 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.780901    2566 scope.go:117] "RemoveContainer" containerID="ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214"
Jul 06 21:14:07.790447 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.790429    2566 scope.go:117] "RemoveContainer" containerID="bed077cc0cb5d025783dc2840c7c46b5b9c416d95267b9e75d6833c0f0c3afee"
Jul 06 21:14:07.798803 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.798778    2566 scope.go:117] "RemoveContainer" containerID="ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214"
Jul 06 21:14:07.799092 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:14:07.799074    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214\": container with ID starting with ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214 not found: ID does not exist" containerID="ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214"
Jul 06 21:14:07.799141 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.799101    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214"} err="failed to get container status \"ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214\": rpc error: code = NotFound desc = could not find container \"ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214\": container with ID starting with ec911bfc2f31c4b102b2feaa7590e48dd6eb578af2d77b138e5ac80a50bb3214 not found: ID does not exist"
Jul 06 21:14:07.799141 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.799120    2566 scope.go:117] "RemoveContainer" containerID="bed077cc0cb5d025783dc2840c7c46b5b9c416d95267b9e75d6833c0f0c3afee"
Jul 06 21:14:07.799397 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:14:07.799380    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bed077cc0cb5d025783dc2840c7c46b5b9c416d95267b9e75d6833c0f0c3afee\": container with ID starting with bed077cc0cb5d025783dc2840c7c46b5b9c416d95267b9e75d6833c0f0c3afee not found: ID does not exist" containerID="bed077cc0cb5d025783dc2840c7c46b5b9c416d95267b9e75d6833c0f0c3afee"
Jul 06 21:14:07.799447 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.799403    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bed077cc0cb5d025783dc2840c7c46b5b9c416d95267b9e75d6833c0f0c3afee"} err="failed to get container status \"bed077cc0cb5d025783dc2840c7c46b5b9c416d95267b9e75d6833c0f0c3afee\": rpc error: code = NotFound desc = could not find container \"bed077cc0cb5d025783dc2840c7c46b5b9c416d95267b9e75d6833c0f0c3afee\": container with ID starting with bed077cc0cb5d025783dc2840c7c46b5b9c416d95267b9e75d6833c0f0c3afee not found: ID does not exist"
Jul 06 21:14:07.808861 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.808831    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"]
Jul 06 21:14:07.817686 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:07.817657    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schesstzr"]
Jul 06 21:14:08.002703 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:08.002660    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fbdae15-4752-42c2-9828-cfec3d4d5ab2" path="/var/lib/kubelet/pods/2fbdae15-4752-42c2-9828-cfec3d4d5ab2/volumes"
Jul 06 21:14:26.150110 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.150072    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"]
Jul 06 21:14:26.150560 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.150449    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fbdae15-4752-42c2-9828-cfec3d4d5ab2" containerName="storage-initializer"
Jul 06 21:14:26.150560 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.150476    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fbdae15-4752-42c2-9828-cfec3d4d5ab2" containerName="storage-initializer"
Jul 06 21:14:26.150560 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.150484    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fbdae15-4752-42c2-9828-cfec3d4d5ab2" containerName="main"
Jul 06 21:14:26.150560 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.150490    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fbdae15-4752-42c2-9828-cfec3d4d5ab2" containerName="main"
Jul 06 21:14:26.150716 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.150564    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fbdae15-4752-42c2-9828-cfec3d4d5ab2" containerName="main"
Jul 06 21:14:26.153810 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.153779    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.156319 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.156291    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs\""
Jul 06 21:14:26.156651 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.156627    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 06 21:14:26.157285 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.157260    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa-dockercfg-f9tmh\""
Jul 06 21:14:26.157414 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.157371    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 06 21:14:26.157547 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.157385    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g6qfk\""
Jul 06 21:14:26.157547 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.157532    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 06 21:14:26.165695 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.165673    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"]
Jul 06 21:14:26.340339 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.340297    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f2f5830-9375-4081-86fc-d7278d77c21f-kserve-provision-location\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.340339 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.340342    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzswp\" (UniqueName: \"kubernetes.io/projected/8f2f5830-9375-4081-86fc-d7278d77c21f-kube-api-access-pzswp\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.340616 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.340468    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f2f5830-9375-4081-86fc-d7278d77c21f-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.340616 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.340538    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8f2f5830-9375-4081-86fc-d7278d77c21f-tokenizer-uds\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.441445 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.441337    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f2f5830-9375-4081-86fc-d7278d77c21f-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.441445 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.441399    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8f2f5830-9375-4081-86fc-d7278d77c21f-tokenizer-uds\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.441445 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.441445    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f2f5830-9375-4081-86fc-d7278d77c21f-kserve-provision-location\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.441755 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.441481    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzswp\" (UniqueName: \"kubernetes.io/projected/8f2f5830-9375-4081-86fc-d7278d77c21f-kube-api-access-pzswp\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.441927 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.441904    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f2f5830-9375-4081-86fc-d7278d77c21f-kserve-provision-location\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.441984 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.441919    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8f2f5830-9375-4081-86fc-d7278d77c21f-tokenizer-uds\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.444012 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.443988    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f2f5830-9375-4081-86fc-d7278d77c21f-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.451368 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.451343    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzswp\" (UniqueName: \"kubernetes.io/projected/8f2f5830-9375-4081-86fc-d7278d77c21f-kube-api-access-pzswp\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.465214 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.465182    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:26.606564 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.606536    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"]
Jul 06 21:14:26.608268 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:14:26.608240    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f2f5830_9375_4081_86fc_d7278d77c21f.slice/crio-12d6ef952c6dff3d0b42070c4cf8130f7b903198798b80a0225dc695fb1396a4 WatchSource:0}: Error finding container 12d6ef952c6dff3d0b42070c4cf8130f7b903198798b80a0225dc695fb1396a4: Status 404 returned error can't find the container with id 12d6ef952c6dff3d0b42070c4cf8130f7b903198798b80a0225dc695fb1396a4
Jul 06 21:14:26.883633 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.883592    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz" event={"ID":"8f2f5830-9375-4081-86fc-d7278d77c21f","Type":"ContainerStarted","Data":"4479a4103e93dc1bed6de68ad6d21026c782284faa149924d9a2340b6be40d67"}
Jul 06 21:14:26.883810 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:26.883685    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz" event={"ID":"8f2f5830-9375-4081-86fc-d7278d77c21f","Type":"ContainerStarted","Data":"12d6ef952c6dff3d0b42070c4cf8130f7b903198798b80a0225dc695fb1396a4"}
Jul 06 21:14:27.891267 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:27.891230    2566 generic.go:356] "Generic (PLEG): container finished" podID="8f2f5830-9375-4081-86fc-d7278d77c21f" containerID="4479a4103e93dc1bed6de68ad6d21026c782284faa149924d9a2340b6be40d67" exitCode=0
Jul 06 21:14:27.891684 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:27.891308    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz" event={"ID":"8f2f5830-9375-4081-86fc-d7278d77c21f","Type":"ContainerDied","Data":"4479a4103e93dc1bed6de68ad6d21026c782284faa149924d9a2340b6be40d67"}
Jul 06 21:14:28.898540 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:28.898499    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz" event={"ID":"8f2f5830-9375-4081-86fc-d7278d77c21f","Type":"ContainerStarted","Data":"b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2"}
Jul 06 21:14:28.898990 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:28.898596    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:14:28.926817 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:28.926762    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz" podStartSLOduration=2.257940294 podStartE2EDuration="2.926746793s" totalImagesPullingTime="0s" totalInitContainerRuntime="668.806499ms" isStatefulPod=false podCreationTimestamp="2026-07-06 21:14:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:14:28.924419632 +0000 UTC m=+1547.390660881" watchObservedRunningTime="2026-07-06 21:14:28.926746793 +0000 UTC m=+1547.392988047"
Jul 06 21:14:59.906090 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:14:59.906051    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:16:54.194859 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.194815    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"]
Jul 06 21:16:54.205780 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:16:54.205735    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:16:54.205780 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:16:54.205780 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:16:54.205780 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2" cmd=["/bin/sleep","15"]
Jul 06 21:16:54.206111 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.205801    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz" message="" err=<
Jul 06 21:16:54.206111 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:16:54.206111 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:16:54.206111 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:16:54.206111 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:16:54.205821    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:16:54.206111 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:16:54.206111 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:16:54.206111 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz" podUID="8f2f5830-9375-4081-86fc-d7278d77c21f" containerName="main" containerID="cri-o://b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2"
Jul 06 21:16:54.206111 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.205863    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz" podUID="8f2f5830-9375-4081-86fc-d7278d77c21f" containerName="main" containerID="cri-o://b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2" gracePeriod=60
Jul 06 21:16:54.473610 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.473585    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:16:54.641671 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.641633    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzswp\" (UniqueName: \"kubernetes.io/projected/8f2f5830-9375-4081-86fc-d7278d77c21f-kube-api-access-pzswp\") pod \"8f2f5830-9375-4081-86fc-d7278d77c21f\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") "
Jul 06 21:16:54.641671 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.641676    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f2f5830-9375-4081-86fc-d7278d77c21f-kserve-provision-location\") pod \"8f2f5830-9375-4081-86fc-d7278d77c21f\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") "
Jul 06 21:16:54.641945 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.641719    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8f2f5830-9375-4081-86fc-d7278d77c21f-tokenizer-uds\") pod \"8f2f5830-9375-4081-86fc-d7278d77c21f\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") "
Jul 06 21:16:54.641945 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.641781    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f2f5830-9375-4081-86fc-d7278d77c21f-tls-certs\") pod \"8f2f5830-9375-4081-86fc-d7278d77c21f\" (UID: \"8f2f5830-9375-4081-86fc-d7278d77c21f\") "
Jul 06 21:16:54.642070 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.642053    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f2f5830-9375-4081-86fc-d7278d77c21f-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "8f2f5830-9375-4081-86fc-d7278d77c21f" (UID: "8f2f5830-9375-4081-86fc-d7278d77c21f"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:16:54.642557 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.642529    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f2f5830-9375-4081-86fc-d7278d77c21f-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8f2f5830-9375-4081-86fc-d7278d77c21f" (UID: "8f2f5830-9375-4081-86fc-d7278d77c21f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:16:54.644071 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.644041    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f2f5830-9375-4081-86fc-d7278d77c21f-kube-api-access-pzswp" (OuterVolumeSpecName: "kube-api-access-pzswp") pod "8f2f5830-9375-4081-86fc-d7278d77c21f" (UID: "8f2f5830-9375-4081-86fc-d7278d77c21f"). InnerVolumeSpecName "kube-api-access-pzswp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:16:54.644071 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.644049    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f2f5830-9375-4081-86fc-d7278d77c21f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8f2f5830-9375-4081-86fc-d7278d77c21f" (UID: "8f2f5830-9375-4081-86fc-d7278d77c21f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:16:54.667334 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.667300    2566 generic.go:356] "Generic (PLEG): container finished" podID="8f2f5830-9375-4081-86fc-d7278d77c21f" containerID="b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2" exitCode=0
Jul 06 21:16:54.667490 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.667344    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"
Jul 06 21:16:54.667490 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.667355    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz" event={"ID":"8f2f5830-9375-4081-86fc-d7278d77c21f","Type":"ContainerDied","Data":"b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2"}
Jul 06 21:16:54.667490 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.667391    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz" event={"ID":"8f2f5830-9375-4081-86fc-d7278d77c21f","Type":"ContainerDied","Data":"12d6ef952c6dff3d0b42070c4cf8130f7b903198798b80a0225dc695fb1396a4"}
Jul 06 21:16:54.667490 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.667410    2566 scope.go:117] "RemoveContainer" containerID="b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2"
Jul 06 21:16:54.677283 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.677262    2566 scope.go:117] "RemoveContainer" containerID="4479a4103e93dc1bed6de68ad6d21026c782284faa149924d9a2340b6be40d67"
Jul 06 21:16:54.685348 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.685328    2566 scope.go:117] "RemoveContainer" containerID="b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2"
Jul 06 21:16:54.685605 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:16:54.685586    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2\": container with ID starting with b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2 not found: ID does not exist" containerID="b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2"
Jul 06 21:16:54.685680 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.685612    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2"} err="failed to get container status \"b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2\": rpc error: code = NotFound desc = could not find container \"b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2\": container with ID starting with b4db277f9d8800a30f73ff8b59dfc128d31df583bbbcac1ef96a95feed35c8c2 not found: ID does not exist"
Jul 06 21:16:54.685680 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.685630    2566 scope.go:117] "RemoveContainer" containerID="4479a4103e93dc1bed6de68ad6d21026c782284faa149924d9a2340b6be40d67"
Jul 06 21:16:54.685857 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:16:54.685839    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4479a4103e93dc1bed6de68ad6d21026c782284faa149924d9a2340b6be40d67\": container with ID starting with 4479a4103e93dc1bed6de68ad6d21026c782284faa149924d9a2340b6be40d67 not found: ID does not exist" containerID="4479a4103e93dc1bed6de68ad6d21026c782284faa149924d9a2340b6be40d67"
Jul 06 21:16:54.685897 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.685863    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4479a4103e93dc1bed6de68ad6d21026c782284faa149924d9a2340b6be40d67"} err="failed to get container status \"4479a4103e93dc1bed6de68ad6d21026c782284faa149924d9a2340b6be40d67\": rpc error: code = NotFound desc = could not find container \"4479a4103e93dc1bed6de68ad6d21026c782284faa149924d9a2340b6be40d67\": container with ID starting with 4479a4103e93dc1bed6de68ad6d21026c782284faa149924d9a2340b6be40d67 not found: ID does not exist"
Jul 06 21:16:54.691662 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.691640    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"]
Jul 06 21:16:54.694664 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.694638    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schect2dz"]
Jul 06 21:16:54.743053 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.743017    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8f2f5830-9375-4081-86fc-d7278d77c21f-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:16:54.743053 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.743049    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f2f5830-9375-4081-86fc-d7278d77c21f-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:16:54.743053 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.743059    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzswp\" (UniqueName: \"kubernetes.io/projected/8f2f5830-9375-4081-86fc-d7278d77c21f-kube-api-access-pzswp\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:16:54.743282 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:54.743069    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f2f5830-9375-4081-86fc-d7278d77c21f-kserve-provision-location\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:16:56.004064 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:16:56.004026    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f2f5830-9375-4081-86fc-d7278d77c21f" path="/var/lib/kubelet/pods/8f2f5830-9375-4081-86fc-d7278d77c21f/volumes"
Jul 06 21:17:24.272248 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.272211    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"]
Jul 06 21:17:24.272799 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.272586    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f2f5830-9375-4081-86fc-d7278d77c21f" containerName="storage-initializer"
Jul 06 21:17:24.272799 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.272598    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f2f5830-9375-4081-86fc-d7278d77c21f" containerName="storage-initializer"
Jul 06 21:17:24.272799 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.272618    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f2f5830-9375-4081-86fc-d7278d77c21f" containerName="main"
Jul 06 21:17:24.272799 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.272631    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f2f5830-9375-4081-86fc-d7278d77c21f" containerName="main"
Jul 06 21:17:24.272799 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.272684    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f2f5830-9375-4081-86fc-d7278d77c21f" containerName="main"
Jul 06 21:17:24.275952 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.275929    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.278541 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.278514    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Jul 06 21:17:24.279016 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.278996    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 06 21:17:24.279779 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.279548    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 06 21:17:24.279779 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.279560    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-epp-sa-dockercfg-nn7cd\""
Jul 06 21:17:24.279779 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.279639    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 06 21:17:24.280001 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.279832    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g6qfk\""
Jul 06 21:17:24.291106 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.291082    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"]
Jul 06 21:17:24.400845 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.400801    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0dc43f78-75a1-4277-ab40-75aa30eebc56-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.401080 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.400860    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0dc43f78-75a1-4277-ab40-75aa30eebc56-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.401080 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.400923    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0dc43f78-75a1-4277-ab40-75aa30eebc56-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.401080 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.401010    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74crv\" (UniqueName: \"kubernetes.io/projected/0dc43f78-75a1-4277-ab40-75aa30eebc56-kube-api-access-74crv\") pod \"custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.502402 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.502361    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74crv\" (UniqueName: \"kubernetes.io/projected/0dc43f78-75a1-4277-ab40-75aa30eebc56-kube-api-access-74crv\") pod \"custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.502641 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.502418    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0dc43f78-75a1-4277-ab40-75aa30eebc56-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.502641 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.502500    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0dc43f78-75a1-4277-ab40-75aa30eebc56-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.502641 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.502543    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0dc43f78-75a1-4277-ab40-75aa30eebc56-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.502917 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.502895    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0dc43f78-75a1-4277-ab40-75aa30eebc56-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.502988 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.502932    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0dc43f78-75a1-4277-ab40-75aa30eebc56-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.504809 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.504787    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0dc43f78-75a1-4277-ab40-75aa30eebc56-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.512197 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.512166    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74crv\" (UniqueName: \"kubernetes.io/projected/0dc43f78-75a1-4277-ab40-75aa30eebc56-kube-api-access-74crv\") pod \"custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.586759 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.586669    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:24.731002 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.730976    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"]
Jul 06 21:17:24.731989 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:17:24.731964    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dc43f78_75a1_4277_ab40_75aa30eebc56.slice/crio-3a2a2836c464dd087a3e5ac09d52a54505c4b2f594dec8d553e30dfe66f1c82d WatchSource:0}: Error finding container 3a2a2836c464dd087a3e5ac09d52a54505c4b2f594dec8d553e30dfe66f1c82d: Status 404 returned error can't find the container with id 3a2a2836c464dd087a3e5ac09d52a54505c4b2f594dec8d553e30dfe66f1c82d
Jul 06 21:17:24.827342 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.827305    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr" event={"ID":"0dc43f78-75a1-4277-ab40-75aa30eebc56","Type":"ContainerStarted","Data":"0b6f2f438ea4fca4b30f9ddd211c7ed18c4e3f5140b98113718e43907f6b97e9"}
Jul 06 21:17:24.827641 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:24.827357    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr" event={"ID":"0dc43f78-75a1-4277-ab40-75aa30eebc56","Type":"ContainerStarted","Data":"3a2a2836c464dd087a3e5ac09d52a54505c4b2f594dec8d553e30dfe66f1c82d"}
Jul 06 21:17:25.835028 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:25.834938    2566 generic.go:356] "Generic (PLEG): container finished" podID="0dc43f78-75a1-4277-ab40-75aa30eebc56" containerID="0b6f2f438ea4fca4b30f9ddd211c7ed18c4e3f5140b98113718e43907f6b97e9" exitCode=0
Jul 06 21:17:25.835465 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:25.835032    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr" event={"ID":"0dc43f78-75a1-4277-ab40-75aa30eebc56","Type":"ContainerDied","Data":"0b6f2f438ea4fca4b30f9ddd211c7ed18c4e3f5140b98113718e43907f6b97e9"}
Jul 06 21:17:26.842661 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:26.842627    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr" event={"ID":"0dc43f78-75a1-4277-ab40-75aa30eebc56","Type":"ContainerStarted","Data":"69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21"}
Jul 06 21:17:26.843146 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:26.842746    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:17:26.865710 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:26.865659    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr" podStartSLOduration=2.170512301 podStartE2EDuration="2.865641769s" totalImagesPullingTime="0s" totalInitContainerRuntime="695.129468ms" isStatefulPod=false podCreationTimestamp="2026-07-06 21:17:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:17:26.863534572 +0000 UTC m=+1725.329775825" watchObservedRunningTime="2026-07-06 21:17:26.865641769 +0000 UTC m=+1725.331883022"
Jul 06 21:17:57.850093 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:17:57.850064    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:19:39.653063 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:39.653026    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"]
Jul 06 21:19:39.661959 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:19:39.661924    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:19:39.661959 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:19:39.661959 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:19:39.661959 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21" cmd=["/bin/sleep","15"]
Jul 06 21:19:39.662195 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:39.661979    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr" message="" err=<
Jul 06 21:19:39.662195 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:19:39.662195 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:19:39.662195 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:19:39.662195 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:19:39.661997    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:19:39.662195 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:19:39.662195 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:19:39.662195 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr" podUID="0dc43f78-75a1-4277-ab40-75aa30eebc56" containerName="main" containerID="cri-o://69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21"
Jul 06 21:19:39.662195 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:39.662040    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr" podUID="0dc43f78-75a1-4277-ab40-75aa30eebc56" containerName="main" containerID="cri-o://69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21" gracePeriod=60
Jul 06 21:19:39.917131 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:39.917067    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:19:40.042087 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.042051    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0dc43f78-75a1-4277-ab40-75aa30eebc56-tokenizer-uds\") pod \"0dc43f78-75a1-4277-ab40-75aa30eebc56\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") "
Jul 06 21:19:40.042281 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.042119    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0dc43f78-75a1-4277-ab40-75aa30eebc56-tls-certs\") pod \"0dc43f78-75a1-4277-ab40-75aa30eebc56\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") "
Jul 06 21:19:40.042281 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.042165    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74crv\" (UniqueName: \"kubernetes.io/projected/0dc43f78-75a1-4277-ab40-75aa30eebc56-kube-api-access-74crv\") pod \"0dc43f78-75a1-4277-ab40-75aa30eebc56\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") "
Jul 06 21:19:40.042281 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.042216    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0dc43f78-75a1-4277-ab40-75aa30eebc56-kserve-provision-location\") pod \"0dc43f78-75a1-4277-ab40-75aa30eebc56\" (UID: \"0dc43f78-75a1-4277-ab40-75aa30eebc56\") "
Jul 06 21:19:40.042444 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.042381    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dc43f78-75a1-4277-ab40-75aa30eebc56-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "0dc43f78-75a1-4277-ab40-75aa30eebc56" (UID: "0dc43f78-75a1-4277-ab40-75aa30eebc56"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:19:40.042944 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.042909    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dc43f78-75a1-4277-ab40-75aa30eebc56-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "0dc43f78-75a1-4277-ab40-75aa30eebc56" (UID: "0dc43f78-75a1-4277-ab40-75aa30eebc56"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:19:40.044431 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.044403    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dc43f78-75a1-4277-ab40-75aa30eebc56-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0dc43f78-75a1-4277-ab40-75aa30eebc56" (UID: "0dc43f78-75a1-4277-ab40-75aa30eebc56"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:19:40.044542 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.044438    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dc43f78-75a1-4277-ab40-75aa30eebc56-kube-api-access-74crv" (OuterVolumeSpecName: "kube-api-access-74crv") pod "0dc43f78-75a1-4277-ab40-75aa30eebc56" (UID: "0dc43f78-75a1-4277-ab40-75aa30eebc56"). InnerVolumeSpecName "kube-api-access-74crv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:19:40.142782 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.142749    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0dc43f78-75a1-4277-ab40-75aa30eebc56-kserve-provision-location\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:19:40.142782 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.142781    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0dc43f78-75a1-4277-ab40-75aa30eebc56-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:19:40.142997 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.142794    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0dc43f78-75a1-4277-ab40-75aa30eebc56-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:19:40.142997 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.142806    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74crv\" (UniqueName: \"kubernetes.io/projected/0dc43f78-75a1-4277-ab40-75aa30eebc56-kube-api-access-74crv\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:19:40.567644 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.567602    2566 generic.go:356] "Generic (PLEG): container finished" podID="0dc43f78-75a1-4277-ab40-75aa30eebc56" containerID="69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21" exitCode=0
Jul 06 21:19:40.567856 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.567679    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"
Jul 06 21:19:40.567856 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.567686    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr" event={"ID":"0dc43f78-75a1-4277-ab40-75aa30eebc56","Type":"ContainerDied","Data":"69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21"}
Jul 06 21:19:40.567856 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.567772    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr" event={"ID":"0dc43f78-75a1-4277-ab40-75aa30eebc56","Type":"ContainerDied","Data":"3a2a2836c464dd087a3e5ac09d52a54505c4b2f594dec8d553e30dfe66f1c82d"}
Jul 06 21:19:40.567856 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.567794    2566 scope.go:117] "RemoveContainer" containerID="69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21"
Jul 06 21:19:40.577301 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.577280    2566 scope.go:117] "RemoveContainer" containerID="0b6f2f438ea4fca4b30f9ddd211c7ed18c4e3f5140b98113718e43907f6b97e9"
Jul 06 21:19:40.585848 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.585829    2566 scope.go:117] "RemoveContainer" containerID="69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21"
Jul 06 21:19:40.586122 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:19:40.586101    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21\": container with ID starting with 69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21 not found: ID does not exist" containerID="69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21"
Jul 06 21:19:40.586189 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.586130    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21"} err="failed to get container status \"69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21\": rpc error: code = NotFound desc = could not find container \"69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21\": container with ID starting with 69d7cb7de15c4523cc79e190252c69bb524f658c35b47c81c8299158369faa21 not found: ID does not exist"
Jul 06 21:19:40.586189 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.586149    2566 scope.go:117] "RemoveContainer" containerID="0b6f2f438ea4fca4b30f9ddd211c7ed18c4e3f5140b98113718e43907f6b97e9"
Jul 06 21:19:40.586376 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:19:40.586357    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b6f2f438ea4fca4b30f9ddd211c7ed18c4e3f5140b98113718e43907f6b97e9\": container with ID starting with 0b6f2f438ea4fca4b30f9ddd211c7ed18c4e3f5140b98113718e43907f6b97e9 not found: ID does not exist" containerID="0b6f2f438ea4fca4b30f9ddd211c7ed18c4e3f5140b98113718e43907f6b97e9"
Jul 06 21:19:40.586416 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.586383    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b6f2f438ea4fca4b30f9ddd211c7ed18c4e3f5140b98113718e43907f6b97e9"} err="failed to get container status \"0b6f2f438ea4fca4b30f9ddd211c7ed18c4e3f5140b98113718e43907f6b97e9\": rpc error: code = NotFound desc = could not find container \"0b6f2f438ea4fca4b30f9ddd211c7ed18c4e3f5140b98113718e43907f6b97e9\": container with ID starting with 0b6f2f438ea4fca4b30f9ddd211c7ed18c4e3f5140b98113718e43907f6b97e9 not found: ID does not exist"
Jul 06 21:19:40.590766 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.590744    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"]
Jul 06 21:19:40.594947 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:40.594926    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-5f76b9b8j8dlr"]
Jul 06 21:19:42.003011 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:19:42.002970    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dc43f78-75a1-4277-ab40-75aa30eebc56" path="/var/lib/kubelet/pods/0dc43f78-75a1-4277-ab40-75aa30eebc56/volumes"
Jul 06 21:20:13.802008 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.801834    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"]
Jul 06 21:20:13.802573 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.802408    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dc43f78-75a1-4277-ab40-75aa30eebc56" containerName="main"
Jul 06 21:20:13.802573 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.802425    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dc43f78-75a1-4277-ab40-75aa30eebc56" containerName="main"
Jul 06 21:20:13.802573 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.802439    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dc43f78-75a1-4277-ab40-75aa30eebc56" containerName="storage-initializer"
Jul 06 21:20:13.802573 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.802446    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dc43f78-75a1-4277-ab40-75aa30eebc56" containerName="storage-initializer"
Jul 06 21:20:13.802573 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.802531    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dc43f78-75a1-4277-ab40-75aa30eebc56" containerName="main"
Jul 06 21:20:13.805719 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.805701    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:13.808872 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.808843    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g6qfk\""
Jul 06 21:20:13.809340 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.809321    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-test-kserve-self-signed-certs\""
Jul 06 21:20:13.809422 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.809385    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 06 21:20:13.809636 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.809620    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 06 21:20:13.809728 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.809703    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-test-epp-sa-dockercfg-sf2lj\""
Jul 06 21:20:13.809925 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.809910    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 06 21:20:13.818548 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.818527    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"]
Jul 06 21:20:13.823447 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.823424    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e250ddd8-38ee-48fe-892d-be4b36f83434-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:13.823560 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.823480    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9ccp\" (UniqueName: \"kubernetes.io/projected/e250ddd8-38ee-48fe-892d-be4b36f83434-kube-api-access-q9ccp\") pod \"router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:13.823560 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.823526    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e250ddd8-38ee-48fe-892d-be4b36f83434-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:13.823647 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.823561    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e250ddd8-38ee-48fe-892d-be4b36f83434-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:13.924654 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.924604    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e250ddd8-38ee-48fe-892d-be4b36f83434-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:13.924849 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.924668    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9ccp\" (UniqueName: \"kubernetes.io/projected/e250ddd8-38ee-48fe-892d-be4b36f83434-kube-api-access-q9ccp\") pod \"router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:13.924849 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.924711    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e250ddd8-38ee-48fe-892d-be4b36f83434-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:13.924849 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.924773    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e250ddd8-38ee-48fe-892d-be4b36f83434-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:13.925025 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.924990    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e250ddd8-38ee-48fe-892d-be4b36f83434-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:13.925121 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.925097    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e250ddd8-38ee-48fe-892d-be4b36f83434-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:13.927404 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.927376    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e250ddd8-38ee-48fe-892d-be4b36f83434-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:13.935941 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:13.935920    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9ccp\" (UniqueName: \"kubernetes.io/projected/e250ddd8-38ee-48fe-892d-be4b36f83434-kube-api-access-q9ccp\") pod \"router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:14.114969 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:14.114887    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:14.255691 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:14.255663    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"]
Jul 06 21:20:14.257937 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:20:14.257906    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode250ddd8_38ee_48fe_892d_be4b36f83434.slice/crio-dea79b9d97bc1798c360a60fb7a97b5750ab605b3f460cb3039b82ca303cd7c6 WatchSource:0}: Error finding container dea79b9d97bc1798c360a60fb7a97b5750ab605b3f460cb3039b82ca303cd7c6: Status 404 returned error can't find the container with id dea79b9d97bc1798c360a60fb7a97b5750ab605b3f460cb3039b82ca303cd7c6
Jul 06 21:20:14.753629 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:14.753587    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg" event={"ID":"e250ddd8-38ee-48fe-892d-be4b36f83434","Type":"ContainerStarted","Data":"daf820c975ef9d0f7fbd98dfa1ef2316dc67c2d29a335a1a9c3176fdc7696482"}
Jul 06 21:20:14.753829 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:14.753645    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg" event={"ID":"e250ddd8-38ee-48fe-892d-be4b36f83434","Type":"ContainerStarted","Data":"dea79b9d97bc1798c360a60fb7a97b5750ab605b3f460cb3039b82ca303cd7c6"}
Jul 06 21:20:15.760149 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:15.760108    2566 generic.go:356] "Generic (PLEG): container finished" podID="e250ddd8-38ee-48fe-892d-be4b36f83434" containerID="daf820c975ef9d0f7fbd98dfa1ef2316dc67c2d29a335a1a9c3176fdc7696482" exitCode=0
Jul 06 21:20:15.760630 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:15.760189    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg" event={"ID":"e250ddd8-38ee-48fe-892d-be4b36f83434","Type":"ContainerDied","Data":"daf820c975ef9d0f7fbd98dfa1ef2316dc67c2d29a335a1a9c3176fdc7696482"}
Jul 06 21:20:16.766606 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:16.766570    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg" event={"ID":"e250ddd8-38ee-48fe-892d-be4b36f83434","Type":"ContainerStarted","Data":"cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc"}
Jul 06 21:20:16.767063 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:16.766667    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:20:16.792101 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:16.792048    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg" podStartSLOduration=3.037405042 podStartE2EDuration="3.79203078s" totalImagesPullingTime="0s" totalInitContainerRuntime="754.625738ms" isStatefulPod=false podCreationTimestamp="2026-07-06 21:20:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:20:16.790617911 +0000 UTC m=+1895.256859168" watchObservedRunningTime="2026-07-06 21:20:16.79203078 +0000 UTC m=+1895.258272183"
Jul 06 21:20:47.773743 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:20:47.773711    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:23:03.270826 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.270788    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"]
Jul 06 21:23:03.280201 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:23:03.280158    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:23:03.280201 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:23:03.280201 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:23:03.280201 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc" cmd=["/bin/sleep","15"]
Jul 06 21:23:03.280550 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.280217    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg" message="" err=<
Jul 06 21:23:03.280550 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:23:03.280550 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:23:03.280550 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:23:03.280550 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:23:03.280230    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:23:03.280550 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:23:03.280550 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:23:03.280550 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg" podUID="e250ddd8-38ee-48fe-892d-be4b36f83434" containerName="main" containerID="cri-o://cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc"
Jul 06 21:23:03.280550 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.280283    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg" podUID="e250ddd8-38ee-48fe-892d-be4b36f83434" containerName="main" containerID="cri-o://cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc" gracePeriod=60
Jul 06 21:23:03.535549 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.535484    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:23:03.592696 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.592658    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9ccp\" (UniqueName: \"kubernetes.io/projected/e250ddd8-38ee-48fe-892d-be4b36f83434-kube-api-access-q9ccp\") pod \"e250ddd8-38ee-48fe-892d-be4b36f83434\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") "
Jul 06 21:23:03.592898 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.592761    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e250ddd8-38ee-48fe-892d-be4b36f83434-kserve-provision-location\") pod \"e250ddd8-38ee-48fe-892d-be4b36f83434\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") "
Jul 06 21:23:03.592898 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.592792    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e250ddd8-38ee-48fe-892d-be4b36f83434-tls-certs\") pod \"e250ddd8-38ee-48fe-892d-be4b36f83434\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") "
Jul 06 21:23:03.592898 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.592845    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e250ddd8-38ee-48fe-892d-be4b36f83434-tokenizer-uds\") pod \"e250ddd8-38ee-48fe-892d-be4b36f83434\" (UID: \"e250ddd8-38ee-48fe-892d-be4b36f83434\") "
Jul 06 21:23:03.593169 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.593136    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e250ddd8-38ee-48fe-892d-be4b36f83434-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "e250ddd8-38ee-48fe-892d-be4b36f83434" (UID: "e250ddd8-38ee-48fe-892d-be4b36f83434"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:23:03.593498 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.593471    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e250ddd8-38ee-48fe-892d-be4b36f83434-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "e250ddd8-38ee-48fe-892d-be4b36f83434" (UID: "e250ddd8-38ee-48fe-892d-be4b36f83434"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:23:03.594837 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.594817    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e250ddd8-38ee-48fe-892d-be4b36f83434-kube-api-access-q9ccp" (OuterVolumeSpecName: "kube-api-access-q9ccp") pod "e250ddd8-38ee-48fe-892d-be4b36f83434" (UID: "e250ddd8-38ee-48fe-892d-be4b36f83434"). InnerVolumeSpecName "kube-api-access-q9ccp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:23:03.594957 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.594937    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e250ddd8-38ee-48fe-892d-be4b36f83434-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e250ddd8-38ee-48fe-892d-be4b36f83434" (UID: "e250ddd8-38ee-48fe-892d-be4b36f83434"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:23:03.646403 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.646353    2566 generic.go:356] "Generic (PLEG): container finished" podID="e250ddd8-38ee-48fe-892d-be4b36f83434" containerID="cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc" exitCode=0
Jul 06 21:23:03.646635 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.646428    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"
Jul 06 21:23:03.646635 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.646422    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg" event={"ID":"e250ddd8-38ee-48fe-892d-be4b36f83434","Type":"ContainerDied","Data":"cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc"}
Jul 06 21:23:03.646635 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.646616    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg" event={"ID":"e250ddd8-38ee-48fe-892d-be4b36f83434","Type":"ContainerDied","Data":"dea79b9d97bc1798c360a60fb7a97b5750ab605b3f460cb3039b82ca303cd7c6"}
Jul 06 21:23:03.646797 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.646651    2566 scope.go:117] "RemoveContainer" containerID="cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc"
Jul 06 21:23:03.656440 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.656422    2566 scope.go:117] "RemoveContainer" containerID="daf820c975ef9d0f7fbd98dfa1ef2316dc67c2d29a335a1a9c3176fdc7696482"
Jul 06 21:23:03.664754 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.664734    2566 scope.go:117] "RemoveContainer" containerID="cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc"
Jul 06 21:23:03.664998 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:23:03.664980    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc\": container with ID starting with cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc not found: ID does not exist" containerID="cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc"
Jul 06 21:23:03.665045 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.665010    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc"} err="failed to get container status \"cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc\": rpc error: code = NotFound desc = could not find container \"cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc\": container with ID starting with cc473f975bf327c3beb8b382833f79d28eb5e995592fdc6474d59e45272275dc not found: ID does not exist"
Jul 06 21:23:03.665045 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.665028    2566 scope.go:117] "RemoveContainer" containerID="daf820c975ef9d0f7fbd98dfa1ef2316dc67c2d29a335a1a9c3176fdc7696482"
Jul 06 21:23:03.665247 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:23:03.665229    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daf820c975ef9d0f7fbd98dfa1ef2316dc67c2d29a335a1a9c3176fdc7696482\": container with ID starting with daf820c975ef9d0f7fbd98dfa1ef2316dc67c2d29a335a1a9c3176fdc7696482 not found: ID does not exist" containerID="daf820c975ef9d0f7fbd98dfa1ef2316dc67c2d29a335a1a9c3176fdc7696482"
Jul 06 21:23:03.665294 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.665255    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daf820c975ef9d0f7fbd98dfa1ef2316dc67c2d29a335a1a9c3176fdc7696482"} err="failed to get container status \"daf820c975ef9d0f7fbd98dfa1ef2316dc67c2d29a335a1a9c3176fdc7696482\": rpc error: code = NotFound desc = could not find container \"daf820c975ef9d0f7fbd98dfa1ef2316dc67c2d29a335a1a9c3176fdc7696482\": container with ID starting with daf820c975ef9d0f7fbd98dfa1ef2316dc67c2d29a335a1a9c3176fdc7696482 not found: ID does not exist"
Jul 06 21:23:03.670750 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.670729    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"]
Jul 06 21:23:03.678975 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.678950    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-65d75b5c87-vkrdg"]
Jul 06 21:23:03.693886 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.693854    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e250ddd8-38ee-48fe-892d-be4b36f83434-kserve-provision-location\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:23:03.694053 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.693890    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e250ddd8-38ee-48fe-892d-be4b36f83434-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:23:03.694053 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.693905    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e250ddd8-38ee-48fe-892d-be4b36f83434-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:23:03.694053 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:03.693918    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9ccp\" (UniqueName: \"kubernetes.io/projected/e250ddd8-38ee-48fe-892d-be4b36f83434-kube-api-access-q9ccp\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:23:04.002557 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:04.002523    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e250ddd8-38ee-48fe-892d-be4b36f83434" path="/var/lib/kubelet/pods/e250ddd8-38ee-48fe-892d-be4b36f83434/volumes"
Jul 06 21:23:40.007518 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.007483    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"]
Jul 06 21:23:40.008619 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.008585    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e250ddd8-38ee-48fe-892d-be4b36f83434" containerName="storage-initializer"
Jul 06 21:23:40.008619 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.008620    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e250ddd8-38ee-48fe-892d-be4b36f83434" containerName="storage-initializer"
Jul 06 21:23:40.008813 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.008630    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e250ddd8-38ee-48fe-892d-be4b36f83434" containerName="main"
Jul 06 21:23:40.008813 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.008637    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e250ddd8-38ee-48fe-892d-be4b36f83434" containerName="main"
Jul 06 21:23:40.008813 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.008725    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e250ddd8-38ee-48fe-892d-be4b36f83434" containerName="main"
Jul 06 21:23:40.012179 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.012162    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.018082 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.018056    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa-dockercfg-mnv4r\""
Jul 06 21:23:40.018341 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.018318    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g6qfk\""
Jul 06 21:23:40.018629 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.018609    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 06 21:23:40.019274 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.019119    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 06 21:23:40.019274 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.019145    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs\""
Jul 06 21:23:40.019430 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.019406    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 06 21:23:40.025754 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.025730    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"]
Jul 06 21:23:40.100302 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.100259    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d07508-d03d-45de-8985-4cc97b8a1f5a-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.100302 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.100307    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b4d07508-d03d-45de-8985-4cc97b8a1f5a-tokenizer-uds\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.100593 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.100353    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhn4p\" (UniqueName: \"kubernetes.io/projected/b4d07508-d03d-45de-8985-4cc97b8a1f5a-kube-api-access-jhn4p\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.100593 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.100423    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b4d07508-d03d-45de-8985-4cc97b8a1f5a-kserve-provision-location\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.201649 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.201611    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b4d07508-d03d-45de-8985-4cc97b8a1f5a-kserve-provision-location\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.201832 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.201673    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d07508-d03d-45de-8985-4cc97b8a1f5a-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.201832 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.201696    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b4d07508-d03d-45de-8985-4cc97b8a1f5a-tokenizer-uds\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.201832 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.201718    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhn4p\" (UniqueName: \"kubernetes.io/projected/b4d07508-d03d-45de-8985-4cc97b8a1f5a-kube-api-access-jhn4p\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.202100 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.202077    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b4d07508-d03d-45de-8985-4cc97b8a1f5a-kserve-provision-location\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.202100 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.202091    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b4d07508-d03d-45de-8985-4cc97b8a1f5a-tokenizer-uds\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.204120 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.204099    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d07508-d03d-45de-8985-4cc97b8a1f5a-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.212735 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.212712    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhn4p\" (UniqueName: \"kubernetes.io/projected/b4d07508-d03d-45de-8985-4cc97b8a1f5a-kube-api-access-jhn4p\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.322556 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.322467    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:40.472554 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.472522    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"]
Jul 06 21:23:40.474046 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:23:40.474018    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4d07508_d03d_45de_8985_4cc97b8a1f5a.slice/crio-76cb3ec3d2a0f9ec167f805754fcf51d860d7699e2e6468c764be210eefe8f43 WatchSource:0}: Error finding container 76cb3ec3d2a0f9ec167f805754fcf51d860d7699e2e6468c764be210eefe8f43: Status 404 returned error can't find the container with id 76cb3ec3d2a0f9ec167f805754fcf51d860d7699e2e6468c764be210eefe8f43
Jul 06 21:23:40.852401 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.852363    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p" event={"ID":"b4d07508-d03d-45de-8985-4cc97b8a1f5a","Type":"ContainerStarted","Data":"cce7b0b26bfc269e1e0db2ea8d5ab96aa8239e7cd14df6570b87a04e903a58ef"}
Jul 06 21:23:40.852594 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:40.852417    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p" event={"ID":"b4d07508-d03d-45de-8985-4cc97b8a1f5a","Type":"ContainerStarted","Data":"76cb3ec3d2a0f9ec167f805754fcf51d860d7699e2e6468c764be210eefe8f43"}
Jul 06 21:23:41.859509 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:41.859449    2566 generic.go:356] "Generic (PLEG): container finished" podID="b4d07508-d03d-45de-8985-4cc97b8a1f5a" containerID="cce7b0b26bfc269e1e0db2ea8d5ab96aa8239e7cd14df6570b87a04e903a58ef" exitCode=0
Jul 06 21:23:41.859993 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:41.859535    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p" event={"ID":"b4d07508-d03d-45de-8985-4cc97b8a1f5a","Type":"ContainerDied","Data":"cce7b0b26bfc269e1e0db2ea8d5ab96aa8239e7cd14df6570b87a04e903a58ef"}
Jul 06 21:23:42.866749 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:42.866708    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p" event={"ID":"b4d07508-d03d-45de-8985-4cc97b8a1f5a","Type":"ContainerStarted","Data":"2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4"}
Jul 06 21:23:42.867114 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:42.866843    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:23:42.898252 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:23:42.898195    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p" podStartSLOduration=3.196403166 podStartE2EDuration="3.898179009s" totalImagesPullingTime="0s" totalInitContainerRuntime="701.775843ms" isStatefulPod=false podCreationTimestamp="2026-07-06 21:23:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:23:42.894975614 +0000 UTC m=+2101.361216867" watchObservedRunningTime="2026-07-06 21:23:42.898179009 +0000 UTC m=+2101.364420264"
Jul 06 21:24:13.874087 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:24:13.874054    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:27:03.133387 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.133346    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"]
Jul 06 21:27:03.149040 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:27:03.148991    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:27:03.149040 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:27:03.149040 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:27:03.149040 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4" cmd=["/bin/sleep","15"]
Jul 06 21:27:03.149374 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.149060    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p" message="" err=<
Jul 06 21:27:03.149374 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:27:03.149374 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:27:03.149374 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:27:03.149374 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:27:03.149078    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:27:03.149374 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:27:03.149374 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:27:03.149374 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p" podUID="b4d07508-d03d-45de-8985-4cc97b8a1f5a" containerName="main" containerID="cri-o://2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4"
Jul 06 21:27:03.149374 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.149124    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p" podUID="b4d07508-d03d-45de-8985-4cc97b8a1f5a" containerName="main" containerID="cri-o://2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4" gracePeriod=60
Jul 06 21:27:03.409485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.409442    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:27:03.597395 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.597357    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b4d07508-d03d-45de-8985-4cc97b8a1f5a-tokenizer-uds\") pod \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") "
Jul 06 21:27:03.597586 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.597433    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhn4p\" (UniqueName: \"kubernetes.io/projected/b4d07508-d03d-45de-8985-4cc97b8a1f5a-kube-api-access-jhn4p\") pod \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") "
Jul 06 21:27:03.597586 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.597504    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b4d07508-d03d-45de-8985-4cc97b8a1f5a-kserve-provision-location\") pod \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") "
Jul 06 21:27:03.597586 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.597533    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d07508-d03d-45de-8985-4cc97b8a1f5a-tls-certs\") pod \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\" (UID: \"b4d07508-d03d-45de-8985-4cc97b8a1f5a\") "
Jul 06 21:27:03.597702 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.597674    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4d07508-d03d-45de-8985-4cc97b8a1f5a-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "b4d07508-d03d-45de-8985-4cc97b8a1f5a" (UID: "b4d07508-d03d-45de-8985-4cc97b8a1f5a"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:27:03.597800 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.597786    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b4d07508-d03d-45de-8985-4cc97b8a1f5a-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:27:03.598187 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.598156    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4d07508-d03d-45de-8985-4cc97b8a1f5a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b4d07508-d03d-45de-8985-4cc97b8a1f5a" (UID: "b4d07508-d03d-45de-8985-4cc97b8a1f5a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:27:03.599587 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.599558    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4d07508-d03d-45de-8985-4cc97b8a1f5a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b4d07508-d03d-45de-8985-4cc97b8a1f5a" (UID: "b4d07508-d03d-45de-8985-4cc97b8a1f5a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:27:03.599677 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.599659    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4d07508-d03d-45de-8985-4cc97b8a1f5a-kube-api-access-jhn4p" (OuterVolumeSpecName: "kube-api-access-jhn4p") pod "b4d07508-d03d-45de-8985-4cc97b8a1f5a" (UID: "b4d07508-d03d-45de-8985-4cc97b8a1f5a"). InnerVolumeSpecName "kube-api-access-jhn4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:27:03.699121 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.699031    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b4d07508-d03d-45de-8985-4cc97b8a1f5a-kserve-provision-location\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:27:03.699121 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.699062    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d07508-d03d-45de-8985-4cc97b8a1f5a-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:27:03.699121 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.699071    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhn4p\" (UniqueName: \"kubernetes.io/projected/b4d07508-d03d-45de-8985-4cc97b8a1f5a-kube-api-access-jhn4p\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:27:03.916741 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.916702    2566 generic.go:356] "Generic (PLEG): container finished" podID="b4d07508-d03d-45de-8985-4cc97b8a1f5a" containerID="2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4" exitCode=0
Jul 06 21:27:03.916914 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.916756    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"
Jul 06 21:27:03.916914 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.916786    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p" event={"ID":"b4d07508-d03d-45de-8985-4cc97b8a1f5a","Type":"ContainerDied","Data":"2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4"}
Jul 06 21:27:03.916914 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.916878    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p" event={"ID":"b4d07508-d03d-45de-8985-4cc97b8a1f5a","Type":"ContainerDied","Data":"76cb3ec3d2a0f9ec167f805754fcf51d860d7699e2e6468c764be210eefe8f43"}
Jul 06 21:27:03.916914 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.916901    2566 scope.go:117] "RemoveContainer" containerID="2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4"
Jul 06 21:27:03.926336 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.926319    2566 scope.go:117] "RemoveContainer" containerID="cce7b0b26bfc269e1e0db2ea8d5ab96aa8239e7cd14df6570b87a04e903a58ef"
Jul 06 21:27:03.934760 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.934740    2566 scope.go:117] "RemoveContainer" containerID="2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4"
Jul 06 21:27:03.935016 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:27:03.934998    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4\": container with ID starting with 2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4 not found: ID does not exist" containerID="2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4"
Jul 06 21:27:03.935056 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.935028    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4"} err="failed to get container status \"2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4\": rpc error: code = NotFound desc = could not find container \"2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4\": container with ID starting with 2aa508d06f6f8662c809e54f9b0f117a771db97e650d960cd36f1ac23aee5ba4 not found: ID does not exist"
Jul 06 21:27:03.935056 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.935045    2566 scope.go:117] "RemoveContainer" containerID="cce7b0b26bfc269e1e0db2ea8d5ab96aa8239e7cd14df6570b87a04e903a58ef"
Jul 06 21:27:03.935266 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:27:03.935245    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cce7b0b26bfc269e1e0db2ea8d5ab96aa8239e7cd14df6570b87a04e903a58ef\": container with ID starting with cce7b0b26bfc269e1e0db2ea8d5ab96aa8239e7cd14df6570b87a04e903a58ef not found: ID does not exist" containerID="cce7b0b26bfc269e1e0db2ea8d5ab96aa8239e7cd14df6570b87a04e903a58ef"
Jul 06 21:27:03.935369 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.935270    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cce7b0b26bfc269e1e0db2ea8d5ab96aa8239e7cd14df6570b87a04e903a58ef"} err="failed to get container status \"cce7b0b26bfc269e1e0db2ea8d5ab96aa8239e7cd14df6570b87a04e903a58ef\": rpc error: code = NotFound desc = could not find container \"cce7b0b26bfc269e1e0db2ea8d5ab96aa8239e7cd14df6570b87a04e903a58ef\": container with ID starting with cce7b0b26bfc269e1e0db2ea8d5ab96aa8239e7cd14df6570b87a04e903a58ef not found: ID does not exist"
Jul 06 21:27:03.959612 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.959523    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"]
Jul 06 21:27:03.967893 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:03.967864    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scherhk8p"]
Jul 06 21:27:04.003613 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:04.003581    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4d07508-d03d-45de-8985-4cc97b8a1f5a" path="/var/lib/kubelet/pods/b4d07508-d03d-45de-8985-4cc97b8a1f5a/volumes"
Jul 06 21:27:35.253577 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.253495    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"]
Jul 06 21:27:35.254148 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.254029    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4d07508-d03d-45de-8985-4cc97b8a1f5a" containerName="storage-initializer"
Jul 06 21:27:35.254148 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.254049    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4d07508-d03d-45de-8985-4cc97b8a1f5a" containerName="storage-initializer"
Jul 06 21:27:35.254148 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.254100    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4d07508-d03d-45de-8985-4cc97b8a1f5a" containerName="main"
Jul 06 21:27:35.254148 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.254110    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4d07508-d03d-45de-8985-4cc97b8a1f5a" containerName="main"
Jul 06 21:27:35.254380 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.254196    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4d07508-d03d-45de-8985-4cc97b8a1f5a" containerName="main"
Jul 06 21:27:35.257688 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.257666    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.260596 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.260559    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 06 21:27:35.260596 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.260583    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-epp-sa-dockercfg-rprwt\""
Jul 06 21:27:35.261528 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.261507    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 06 21:27:35.261658 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.261534    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g6qfk\""
Jul 06 21:27:35.261658 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.261553    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Jul 06 21:27:35.261658 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.261613    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 06 21:27:35.270568 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.270548    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"]
Jul 06 21:27:35.372785 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.372744    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2d47c271-3abc-41a7-a051-59f52fbac1a5-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.372785 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.372786    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4xtt\" (UniqueName: \"kubernetes.io/projected/2d47c271-3abc-41a7-a051-59f52fbac1a5-kube-api-access-v4xtt\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.372995 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.372905    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d47c271-3abc-41a7-a051-59f52fbac1a5-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.372995 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.372937    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2d47c271-3abc-41a7-a051-59f52fbac1a5-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.474403 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.474369    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d47c271-3abc-41a7-a051-59f52fbac1a5-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.474403 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.474405    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2d47c271-3abc-41a7-a051-59f52fbac1a5-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.474701 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.474505    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2d47c271-3abc-41a7-a051-59f52fbac1a5-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.474701 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.474543    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4xtt\" (UniqueName: \"kubernetes.io/projected/2d47c271-3abc-41a7-a051-59f52fbac1a5-kube-api-access-v4xtt\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.474885 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.474865    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2d47c271-3abc-41a7-a051-59f52fbac1a5-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.474964 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.474869    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2d47c271-3abc-41a7-a051-59f52fbac1a5-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.476980 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.476957    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d47c271-3abc-41a7-a051-59f52fbac1a5-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.483655 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.483627    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4xtt\" (UniqueName: \"kubernetes.io/projected/2d47c271-3abc-41a7-a051-59f52fbac1a5-kube-api-access-v4xtt\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.568481 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.568391    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:35.709974 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:35.709949    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"]
Jul 06 21:27:35.711983 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:27:35.711955    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d47c271_3abc_41a7_a051_59f52fbac1a5.slice/crio-7ab448354a8242aadd6e47f2439546388ee306a1c955ecc76ea22ef591563b5f WatchSource:0}: Error finding container 7ab448354a8242aadd6e47f2439546388ee306a1c955ecc76ea22ef591563b5f: Status 404 returned error can't find the container with id 7ab448354a8242aadd6e47f2439546388ee306a1c955ecc76ea22ef591563b5f
Jul 06 21:27:36.087220 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:36.087179    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc" event={"ID":"2d47c271-3abc-41a7-a051-59f52fbac1a5","Type":"ContainerStarted","Data":"368e2f3b201cb626b1eb2e7420077c19a8391b74752229cdf390e4a8038403b3"}
Jul 06 21:27:36.087395 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:36.087263    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc" event={"ID":"2d47c271-3abc-41a7-a051-59f52fbac1a5","Type":"ContainerStarted","Data":"7ab448354a8242aadd6e47f2439546388ee306a1c955ecc76ea22ef591563b5f"}
Jul 06 21:27:37.094010 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:37.093973    2566 generic.go:356] "Generic (PLEG): container finished" podID="2d47c271-3abc-41a7-a051-59f52fbac1a5" containerID="368e2f3b201cb626b1eb2e7420077c19a8391b74752229cdf390e4a8038403b3" exitCode=0
Jul 06 21:27:37.094436 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:37.094058    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc" event={"ID":"2d47c271-3abc-41a7-a051-59f52fbac1a5","Type":"ContainerDied","Data":"368e2f3b201cb626b1eb2e7420077c19a8391b74752229cdf390e4a8038403b3"}
Jul 06 21:27:38.101037 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:38.101005    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc" event={"ID":"2d47c271-3abc-41a7-a051-59f52fbac1a5","Type":"ContainerStarted","Data":"d7ba051030b3fbc7eb8bc3ee3bf5a9c598fb049e5f9a5ff8cd0a86ce02b155d5"}
Jul 06 21:27:38.101423 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:38.101126    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:27:38.125322 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:27:38.125268    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc" podStartSLOduration=2.438398063 podStartE2EDuration="3.125251538s" totalImagesPullingTime="0s" totalInitContainerRuntime="686.853475ms" isStatefulPod=false podCreationTimestamp="2026-07-06 21:27:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:27:38.123185101 +0000 UTC m=+2336.589426357" watchObservedRunningTime="2026-07-06 21:27:38.125251538 +0000 UTC m=+2336.591492792"
Jul 06 21:28:09.107911 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:28:09.107881    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:30:40.815700 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:40.815614    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"]
Jul 06 21:30:40.825278 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:30:40.825236    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:30:40.825278 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:30:40.825278 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:30:40.825278 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="d7ba051030b3fbc7eb8bc3ee3bf5a9c598fb049e5f9a5ff8cd0a86ce02b155d5" cmd=["/bin/sleep","15"]
Jul 06 21:30:40.825646 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:40.825302    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc" message="" err=<
Jul 06 21:30:40.825646 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:30:40.825646 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:30:40.825646 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:30:40.825646 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:30:40.825320    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:30:40.825646 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:30:40.825646 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:30:40.825646 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc" podUID="2d47c271-3abc-41a7-a051-59f52fbac1a5" containerName="main" containerID="cri-o://d7ba051030b3fbc7eb8bc3ee3bf5a9c598fb049e5f9a5ff8cd0a86ce02b155d5"
Jul 06 21:30:40.825646 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:40.825365    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc" podUID="2d47c271-3abc-41a7-a051-59f52fbac1a5" containerName="main" containerID="cri-o://d7ba051030b3fbc7eb8bc3ee3bf5a9c598fb049e5f9a5ff8cd0a86ce02b155d5" gracePeriod=60
Jul 06 21:30:41.063793 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.063756    2566 generic.go:356] "Generic (PLEG): container finished" podID="2d47c271-3abc-41a7-a051-59f52fbac1a5" containerID="d7ba051030b3fbc7eb8bc3ee3bf5a9c598fb049e5f9a5ff8cd0a86ce02b155d5" exitCode=0
Jul 06 21:30:41.063974 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.063831    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc" event={"ID":"2d47c271-3abc-41a7-a051-59f52fbac1a5","Type":"ContainerDied","Data":"d7ba051030b3fbc7eb8bc3ee3bf5a9c598fb049e5f9a5ff8cd0a86ce02b155d5"}
Jul 06 21:30:41.090162 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.090129    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:30:41.180830 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.180788    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d47c271-3abc-41a7-a051-59f52fbac1a5-tls-certs\") pod \"2d47c271-3abc-41a7-a051-59f52fbac1a5\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") "
Jul 06 21:30:41.181035 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.180854    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2d47c271-3abc-41a7-a051-59f52fbac1a5-tokenizer-uds\") pod \"2d47c271-3abc-41a7-a051-59f52fbac1a5\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") "
Jul 06 21:30:41.181035 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.180888    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2d47c271-3abc-41a7-a051-59f52fbac1a5-kserve-provision-location\") pod \"2d47c271-3abc-41a7-a051-59f52fbac1a5\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") "
Jul 06 21:30:41.181035 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.180910    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4xtt\" (UniqueName: \"kubernetes.io/projected/2d47c271-3abc-41a7-a051-59f52fbac1a5-kube-api-access-v4xtt\") pod \"2d47c271-3abc-41a7-a051-59f52fbac1a5\" (UID: \"2d47c271-3abc-41a7-a051-59f52fbac1a5\") "
Jul 06 21:30:41.181198 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.181130    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d47c271-3abc-41a7-a051-59f52fbac1a5-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "2d47c271-3abc-41a7-a051-59f52fbac1a5" (UID: "2d47c271-3abc-41a7-a051-59f52fbac1a5"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:30:41.181620 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.181590    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d47c271-3abc-41a7-a051-59f52fbac1a5-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "2d47c271-3abc-41a7-a051-59f52fbac1a5" (UID: "2d47c271-3abc-41a7-a051-59f52fbac1a5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:30:41.183020 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.182993    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d47c271-3abc-41a7-a051-59f52fbac1a5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2d47c271-3abc-41a7-a051-59f52fbac1a5" (UID: "2d47c271-3abc-41a7-a051-59f52fbac1a5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:30:41.183020 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.183014    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d47c271-3abc-41a7-a051-59f52fbac1a5-kube-api-access-v4xtt" (OuterVolumeSpecName: "kube-api-access-v4xtt") pod "2d47c271-3abc-41a7-a051-59f52fbac1a5" (UID: "2d47c271-3abc-41a7-a051-59f52fbac1a5"). InnerVolumeSpecName "kube-api-access-v4xtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:30:41.281711 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.281677    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d47c271-3abc-41a7-a051-59f52fbac1a5-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:30:41.281711 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.281706    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2d47c271-3abc-41a7-a051-59f52fbac1a5-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:30:41.281711 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.281716    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2d47c271-3abc-41a7-a051-59f52fbac1a5-kserve-provision-location\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:30:41.281958 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:41.281725    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4xtt\" (UniqueName: \"kubernetes.io/projected/2d47c271-3abc-41a7-a051-59f52fbac1a5-kube-api-access-v4xtt\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:30:42.071954 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:42.071924    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"
Jul 06 21:30:42.072355 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:42.071962    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc" event={"ID":"2d47c271-3abc-41a7-a051-59f52fbac1a5","Type":"ContainerDied","Data":"7ab448354a8242aadd6e47f2439546388ee306a1c955ecc76ea22ef591563b5f"}
Jul 06 21:30:42.072355 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:42.072025    2566 scope.go:117] "RemoveContainer" containerID="d7ba051030b3fbc7eb8bc3ee3bf5a9c598fb049e5f9a5ff8cd0a86ce02b155d5"
Jul 06 21:30:42.081048 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:42.081023    2566 scope.go:117] "RemoveContainer" containerID="368e2f3b201cb626b1eb2e7420077c19a8391b74752229cdf390e4a8038403b3"
Jul 06 21:30:42.091857 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:42.091832    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"]
Jul 06 21:30:42.095117 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:42.095095    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-569482q2nc"]
Jul 06 21:30:44.003322 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:30:44.003286    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d47c271-3abc-41a7-a051-59f52fbac1a5" path="/var/lib/kubelet/pods/2d47c271-3abc-41a7-a051-59f52fbac1a5/volumes"
Jul 06 21:31:40.183598 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.183563    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"]
Jul 06 21:31:40.185896 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.183937    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d47c271-3abc-41a7-a051-59f52fbac1a5" containerName="storage-initializer"
Jul 06 21:31:40.185896 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.183950    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d47c271-3abc-41a7-a051-59f52fbac1a5" containerName="storage-initializer"
Jul 06 21:31:40.185896 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.183960    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d47c271-3abc-41a7-a051-59f52fbac1a5" containerName="main"
Jul 06 21:31:40.185896 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.183965    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d47c271-3abc-41a7-a051-59f52fbac1a5" containerName="main"
Jul 06 21:31:40.185896 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.184047    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d47c271-3abc-41a7-a051-59f52fbac1a5" containerName="main"
Jul 06 21:31:40.187242 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.187221    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.189697 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.189670    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 06 21:31:40.190030 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.190013    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 06 21:31:40.190238 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.190223    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-epp-sa-dockercfg-bkkf5\""
Jul 06 21:31:40.191056 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.190991    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 06 21:31:40.191204 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.191001    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Jul 06 21:31:40.191204 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.191001    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g6qfk\""
Jul 06 21:31:40.203398 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.203369    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"]
Jul 06 21:31:40.290242 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.290204    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbac5dc1-839c-487a-bfab-57f30b0050f8-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.290415 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.290330    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/dbac5dc1-839c-487a-bfab-57f30b0050f8-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.290415 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.290367    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dbac5dc1-839c-487a-bfab-57f30b0050f8-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.290528 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.290415    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6pb6\" (UniqueName: \"kubernetes.io/projected/dbac5dc1-839c-487a-bfab-57f30b0050f8-kube-api-access-r6pb6\") pod \"router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.391711 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.391675    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbac5dc1-839c-487a-bfab-57f30b0050f8-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.391884 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.391755    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/dbac5dc1-839c-487a-bfab-57f30b0050f8-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.391884 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.391791    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dbac5dc1-839c-487a-bfab-57f30b0050f8-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.391884 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.391847    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6pb6\" (UniqueName: \"kubernetes.io/projected/dbac5dc1-839c-487a-bfab-57f30b0050f8-kube-api-access-r6pb6\") pod \"router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.392157 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.392130    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/dbac5dc1-839c-487a-bfab-57f30b0050f8-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.392232 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.392176    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dbac5dc1-839c-487a-bfab-57f30b0050f8-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.394468 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.394431    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbac5dc1-839c-487a-bfab-57f30b0050f8-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.402078 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.402054    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6pb6\" (UniqueName: \"kubernetes.io/projected/dbac5dc1-839c-487a-bfab-57f30b0050f8-kube-api-access-r6pb6\") pod \"router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.497691 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.497654    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:40.654700 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:40.654664    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"]
Jul 06 21:31:40.658067 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:31:40.658032    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbac5dc1_839c_487a_bfab_57f30b0050f8.slice/crio-f94bf74f926d5d1ed7c061dbfd73cfbc22c047469be707cb42227c9c0ca493fe WatchSource:0}: Error finding container f94bf74f926d5d1ed7c061dbfd73cfbc22c047469be707cb42227c9c0ca493fe: Status 404 returned error can't find the container with id f94bf74f926d5d1ed7c061dbfd73cfbc22c047469be707cb42227c9c0ca493fe
Jul 06 21:31:41.378689 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:41.378652    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm" event={"ID":"dbac5dc1-839c-487a-bfab-57f30b0050f8","Type":"ContainerStarted","Data":"51f7649542e53a8fc07b1ab1a46e6b671df230b54c2f6cbd4a8cf3dfed0d12bd"}
Jul 06 21:31:41.379183 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:41.378745    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm" event={"ID":"dbac5dc1-839c-487a-bfab-57f30b0050f8","Type":"ContainerStarted","Data":"f94bf74f926d5d1ed7c061dbfd73cfbc22c047469be707cb42227c9c0ca493fe"}
Jul 06 21:31:42.384902 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:42.384859    2566 generic.go:356] "Generic (PLEG): container finished" podID="dbac5dc1-839c-487a-bfab-57f30b0050f8" containerID="51f7649542e53a8fc07b1ab1a46e6b671df230b54c2f6cbd4a8cf3dfed0d12bd" exitCode=0
Jul 06 21:31:42.385358 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:42.384949    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm" event={"ID":"dbac5dc1-839c-487a-bfab-57f30b0050f8","Type":"ContainerDied","Data":"51f7649542e53a8fc07b1ab1a46e6b671df230b54c2f6cbd4a8cf3dfed0d12bd"}
Jul 06 21:31:43.392602 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:43.392553    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm" event={"ID":"dbac5dc1-839c-487a-bfab-57f30b0050f8","Type":"ContainerStarted","Data":"ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499"}
Jul 06 21:31:43.393093 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:43.392668    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:31:43.420561 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:31:43.420504    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm" podStartSLOduration=2.73441264 podStartE2EDuration="3.420487227s" totalImagesPullingTime="0s" totalInitContainerRuntime="686.074587ms" isStatefulPod=false podCreationTimestamp="2026-07-06 21:31:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:31:43.418576989 +0000 UTC m=+2581.884818241" watchObservedRunningTime="2026-07-06 21:31:43.420487227 +0000 UTC m=+2581.886728475"
Jul 06 21:32:14.400074 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:32:14.399995    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:35:08.063108 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.063026    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"]
Jul 06 21:35:08.072319 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:35:08.072284    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:35:08.072319 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:35:08.072319 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:35:08.072319 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499" cmd=["/bin/sleep","15"]
Jul 06 21:35:08.072665 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.072333    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm" message="" err=<
Jul 06 21:35:08.072665 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:35:08.072665 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:35:08.072665 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:35:08.072665 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:35:08.072346    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:35:08.072665 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:35:08.072665 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:35:08.072665 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm" podUID="dbac5dc1-839c-487a-bfab-57f30b0050f8" containerName="main" containerID="cri-o://ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499"
Jul 06 21:35:08.072665 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.072384    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm" podUID="dbac5dc1-839c-487a-bfab-57f30b0050f8" containerName="main" containerID="cri-o://ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499" gracePeriod=60
Jul 06 21:35:08.334632 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.334606    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:35:08.445005 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.444967    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbac5dc1-839c-487a-bfab-57f30b0050f8-tls-certs\") pod \"dbac5dc1-839c-487a-bfab-57f30b0050f8\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") "
Jul 06 21:35:08.445206 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.445058    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dbac5dc1-839c-487a-bfab-57f30b0050f8-kserve-provision-location\") pod \"dbac5dc1-839c-487a-bfab-57f30b0050f8\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") "
Jul 06 21:35:08.445206 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.445149    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/dbac5dc1-839c-487a-bfab-57f30b0050f8-tokenizer-uds\") pod \"dbac5dc1-839c-487a-bfab-57f30b0050f8\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") "
Jul 06 21:35:08.445332 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.445225    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6pb6\" (UniqueName: \"kubernetes.io/projected/dbac5dc1-839c-487a-bfab-57f30b0050f8-kube-api-access-r6pb6\") pod \"dbac5dc1-839c-487a-bfab-57f30b0050f8\" (UID: \"dbac5dc1-839c-487a-bfab-57f30b0050f8\") "
Jul 06 21:35:08.445432 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.445405    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbac5dc1-839c-487a-bfab-57f30b0050f8-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "dbac5dc1-839c-487a-bfab-57f30b0050f8" (UID: "dbac5dc1-839c-487a-bfab-57f30b0050f8"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:35:08.445556 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.445539    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/dbac5dc1-839c-487a-bfab-57f30b0050f8-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:35:08.445865 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.445841    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbac5dc1-839c-487a-bfab-57f30b0050f8-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "dbac5dc1-839c-487a-bfab-57f30b0050f8" (UID: "dbac5dc1-839c-487a-bfab-57f30b0050f8"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:35:08.447223 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.447198    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbac5dc1-839c-487a-bfab-57f30b0050f8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dbac5dc1-839c-487a-bfab-57f30b0050f8" (UID: "dbac5dc1-839c-487a-bfab-57f30b0050f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:35:08.447477 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.447435    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbac5dc1-839c-487a-bfab-57f30b0050f8-kube-api-access-r6pb6" (OuterVolumeSpecName: "kube-api-access-r6pb6") pod "dbac5dc1-839c-487a-bfab-57f30b0050f8" (UID: "dbac5dc1-839c-487a-bfab-57f30b0050f8"). InnerVolumeSpecName "kube-api-access-r6pb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:35:08.485201 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.485172    2566 generic.go:356] "Generic (PLEG): container finished" podID="dbac5dc1-839c-487a-bfab-57f30b0050f8" containerID="ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499" exitCode=0
Jul 06 21:35:08.485354 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.485217    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"
Jul 06 21:35:08.485354 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.485262    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm" event={"ID":"dbac5dc1-839c-487a-bfab-57f30b0050f8","Type":"ContainerDied","Data":"ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499"}
Jul 06 21:35:08.485354 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.485308    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm" event={"ID":"dbac5dc1-839c-487a-bfab-57f30b0050f8","Type":"ContainerDied","Data":"f94bf74f926d5d1ed7c061dbfd73cfbc22c047469be707cb42227c9c0ca493fe"}
Jul 06 21:35:08.485354 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.485328    2566 scope.go:117] "RemoveContainer" containerID="ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499"
Jul 06 21:35:08.496102 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.496081    2566 scope.go:117] "RemoveContainer" containerID="51f7649542e53a8fc07b1ab1a46e6b671df230b54c2f6cbd4a8cf3dfed0d12bd"
Jul 06 21:35:08.504282 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.504262    2566 scope.go:117] "RemoveContainer" containerID="ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499"
Jul 06 21:35:08.504581 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:35:08.504560    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499\": container with ID starting with ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499 not found: ID does not exist" containerID="ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499"
Jul 06 21:35:08.504674 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.504594    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499"} err="failed to get container status \"ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499\": rpc error: code = NotFound desc = could not find container \"ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499\": container with ID starting with ac343dbb7ac7f475ad9960d203fa7962deafc7a1da4d1445a55e4ec2bb74a499 not found: ID does not exist"
Jul 06 21:35:08.504674 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.504618    2566 scope.go:117] "RemoveContainer" containerID="51f7649542e53a8fc07b1ab1a46e6b671df230b54c2f6cbd4a8cf3dfed0d12bd"
Jul 06 21:35:08.504884 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:35:08.504866    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51f7649542e53a8fc07b1ab1a46e6b671df230b54c2f6cbd4a8cf3dfed0d12bd\": container with ID starting with 51f7649542e53a8fc07b1ab1a46e6b671df230b54c2f6cbd4a8cf3dfed0d12bd not found: ID does not exist" containerID="51f7649542e53a8fc07b1ab1a46e6b671df230b54c2f6cbd4a8cf3dfed0d12bd"
Jul 06 21:35:08.504926 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.504891    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f7649542e53a8fc07b1ab1a46e6b671df230b54c2f6cbd4a8cf3dfed0d12bd"} err="failed to get container status \"51f7649542e53a8fc07b1ab1a46e6b671df230b54c2f6cbd4a8cf3dfed0d12bd\": rpc error: code = NotFound desc = could not find container \"51f7649542e53a8fc07b1ab1a46e6b671df230b54c2f6cbd4a8cf3dfed0d12bd\": container with ID starting with 51f7649542e53a8fc07b1ab1a46e6b671df230b54c2f6cbd4a8cf3dfed0d12bd not found: ID does not exist"
Jul 06 21:35:08.512378 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.512355    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"]
Jul 06 21:35:08.515587 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.515567    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-555b5c7c6jmtxm"]
Jul 06 21:35:08.546661 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.546628    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6pb6\" (UniqueName: \"kubernetes.io/projected/dbac5dc1-839c-487a-bfab-57f30b0050f8-kube-api-access-r6pb6\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:35:08.546661 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.546658    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbac5dc1-839c-487a-bfab-57f30b0050f8-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:35:08.546835 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:08.546678    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dbac5dc1-839c-487a-bfab-57f30b0050f8-kserve-provision-location\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:35:10.002539 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:35:10.002497    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbac5dc1-839c-487a-bfab-57f30b0050f8" path="/var/lib/kubelet/pods/dbac5dc1-839c-487a-bfab-57f30b0050f8/volumes"
Jul 06 21:39:06.592938 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.592900    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Jul 06 21:39:06.593363 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.593245    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbac5dc1-839c-487a-bfab-57f30b0050f8" containerName="storage-initializer"
Jul 06 21:39:06.593363 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.593256    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbac5dc1-839c-487a-bfab-57f30b0050f8" containerName="storage-initializer"
Jul 06 21:39:06.593363 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.593278    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbac5dc1-839c-487a-bfab-57f30b0050f8" containerName="main"
Jul 06 21:39:06.593363 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.593284    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbac5dc1-839c-487a-bfab-57f30b0050f8" containerName="main"
Jul 06 21:39:06.593363 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.593338    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbac5dc1-839c-487a-bfab-57f30b0050f8" containerName="main"
Jul 06 21:39:06.596443 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.596426    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.600134 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.600107    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 06 21:39:06.601137 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.601118    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-dc21cb-57cf7736-dockercfg-gkgq6\""
Jul 06 21:39:06.601137 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.601128    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 06 21:39:06.601311 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.601124    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs\""
Jul 06 21:39:06.601311 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.601148    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 06 21:39:06.601311 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.601126    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g6qfk\""
Jul 06 21:39:06.615105 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.615079    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Jul 06 21:39:06.682752 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.682721    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.682924 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.682770    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.682924 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.682827    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.682924 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.682846    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfcbl\" (UniqueName: \"kubernetes.io/projected/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-kube-api-access-rfcbl\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.682924 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.682901    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.682924 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.682916    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.683113 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.682932    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.783758 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.783725    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.783758 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.783762    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.784009 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.783784    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.784009 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.783810    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.784009 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.783865    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.784009 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.783934    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.784009 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.783959    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfcbl\" (UniqueName: \"kubernetes.io/projected/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-kube-api-access-rfcbl\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.784212 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.784147    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.784326 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.784301    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.784406 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.784309    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.784406 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.784354    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.786345 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.786324    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.786978 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.786960    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.788277 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.788251    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"]
Jul 06 21:39:06.792042 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.792023    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:06.794443 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.794414    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-dc21cb14-epp-sa-dockercfg-p6mw4\""
Jul 06 21:39:06.796041 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.796019    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfcbl\" (UniqueName: \"kubernetes.io/projected/796ba5a7-c7bc-4027-bc2c-0050ed9f9d65-kube-api-access-rfcbl\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.808889 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.808864    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"]
Jul 06 21:39:06.885117 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.885020    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c2c3c0d-74e2-4c8a-b483-533601eca911-tls-certs\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm\" (UID: \"2c2c3c0d-74e2-4c8a-b483-533601eca911\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:06.885117 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.885090    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbtnc\" (UniqueName: \"kubernetes.io/projected/2c2c3c0d-74e2-4c8a-b483-533601eca911-kube-api-access-tbtnc\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm\" (UID: \"2c2c3c0d-74e2-4c8a-b483-533601eca911\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:06.885346 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.885123    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2c2c3c0d-74e2-4c8a-b483-533601eca911-tokenizer-uds\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm\" (UID: \"2c2c3c0d-74e2-4c8a-b483-533601eca911\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:06.885346 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.885239    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2c2c3c0d-74e2-4c8a-b483-533601eca911-kserve-provision-location\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm\" (UID: \"2c2c3c0d-74e2-4c8a-b483-533601eca911\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:06.906990 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.906964    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 06 21:39:06.985891 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.985856    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2c2c3c0d-74e2-4c8a-b483-533601eca911-kserve-provision-location\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm\" (UID: \"2c2c3c0d-74e2-4c8a-b483-533601eca911\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:06.986043 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.985912    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c2c3c0d-74e2-4c8a-b483-533601eca911-tls-certs\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm\" (UID: \"2c2c3c0d-74e2-4c8a-b483-533601eca911\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:06.986043 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.985966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbtnc\" (UniqueName: \"kubernetes.io/projected/2c2c3c0d-74e2-4c8a-b483-533601eca911-kube-api-access-tbtnc\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm\" (UID: \"2c2c3c0d-74e2-4c8a-b483-533601eca911\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:06.986043 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.985990    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2c2c3c0d-74e2-4c8a-b483-533601eca911-tokenizer-uds\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm\" (UID: \"2c2c3c0d-74e2-4c8a-b483-533601eca911\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:06.986373 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.986339    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2c2c3c0d-74e2-4c8a-b483-533601eca911-tokenizer-uds\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm\" (UID: \"2c2c3c0d-74e2-4c8a-b483-533601eca911\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:06.986498 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.986395    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2c2c3c0d-74e2-4c8a-b483-533601eca911-kserve-provision-location\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm\" (UID: \"2c2c3c0d-74e2-4c8a-b483-533601eca911\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:06.988330 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:06.988307    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c2c3c0d-74e2-4c8a-b483-533601eca911-tls-certs\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm\" (UID: \"2c2c3c0d-74e2-4c8a-b483-533601eca911\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:07.019391 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:07.019352    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbtnc\" (UniqueName: \"kubernetes.io/projected/2c2c3c0d-74e2-4c8a-b483-533601eca911-kube-api-access-tbtnc\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm\" (UID: \"2c2c3c0d-74e2-4c8a-b483-533601eca911\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:07.060678 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:07.060651    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Jul 06 21:39:07.061348 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:39:07.061317    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod796ba5a7_c7bc_4027_bc2c_0050ed9f9d65.slice/crio-a04ff39e311cb01fe5a47ccb733a1c9a147ed35b2a7e24ff5470dc58af42df9a WatchSource:0}: Error finding container a04ff39e311cb01fe5a47ccb733a1c9a147ed35b2a7e24ff5470dc58af42df9a: Status 404 returned error can't find the container with id a04ff39e311cb01fe5a47ccb733a1c9a147ed35b2a7e24ff5470dc58af42df9a
Jul 06 21:39:07.116622 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:07.116598    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:07.271841 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:07.266540    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"]
Jul 06 21:39:07.271841 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:39:07.268942    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c2c3c0d_74e2_4c8a_b483_533601eca911.slice/crio-26e6934147172d2a0ce434828c4f898d286674d287a787352f10a5a15becc4de WatchSource:0}: Error finding container 26e6934147172d2a0ce434828c4f898d286674d287a787352f10a5a15becc4de: Status 404 returned error can't find the container with id 26e6934147172d2a0ce434828c4f898d286674d287a787352f10a5a15becc4de
Jul 06 21:39:07.735949 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:07.735910    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm" event={"ID":"2c2c3c0d-74e2-4c8a-b483-533601eca911","Type":"ContainerStarted","Data":"4b2b14f9d45690d755a3b301d782fc40cc3c318ed9805f89df6246add3f631d8"}
Jul 06 21:39:07.736395 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:07.735974    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm" event={"ID":"2c2c3c0d-74e2-4c8a-b483-533601eca911","Type":"ContainerStarted","Data":"26e6934147172d2a0ce434828c4f898d286674d287a787352f10a5a15becc4de"}
Jul 06 21:39:07.737648 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:07.737603    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65","Type":"ContainerStarted","Data":"3d14509089249766738cc9800b8cb924cc5a2c0bc457435b8d2985089699223b"}
Jul 06 21:39:07.737805 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:07.737670    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65","Type":"ContainerStarted","Data":"a04ff39e311cb01fe5a47ccb733a1c9a147ed35b2a7e24ff5470dc58af42df9a"}
Jul 06 21:39:08.746224 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:08.746185    2566 generic.go:356] "Generic (PLEG): container finished" podID="2c2c3c0d-74e2-4c8a-b483-533601eca911" containerID="4b2b14f9d45690d755a3b301d782fc40cc3c318ed9805f89df6246add3f631d8" exitCode=0
Jul 06 21:39:08.746743 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:08.746264    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm" event={"ID":"2c2c3c0d-74e2-4c8a-b483-533601eca911","Type":"ContainerDied","Data":"4b2b14f9d45690d755a3b301d782fc40cc3c318ed9805f89df6246add3f631d8"}
Jul 06 21:39:09.753538 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:09.753496    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm" event={"ID":"2c2c3c0d-74e2-4c8a-b483-533601eca911","Type":"ContainerStarted","Data":"a21fe26382b3c5b08d40cbaa56829f146ed378d4696d9f9fb6e12e5eb92fe2b8"}
Jul 06 21:39:09.753930 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:09.753649    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:09.778514 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:09.778443    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm" podStartSLOduration=3.0870626740000002 podStartE2EDuration="3.778428606s" totalImagesPullingTime="0s" totalInitContainerRuntime="691.365932ms" isStatefulPod=false podCreationTimestamp="2026-07-06 21:39:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:39:09.77607449 +0000 UTC m=+3028.242315737" watchObservedRunningTime="2026-07-06 21:39:09.778428606 +0000 UTC m=+3028.244670090"
Jul 06 21:39:12.778653 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:12.778620    2566 generic.go:356] "Generic (PLEG): container finished" podID="796ba5a7-c7bc-4027-bc2c-0050ed9f9d65" containerID="3d14509089249766738cc9800b8cb924cc5a2c0bc457435b8d2985089699223b" exitCode=0
Jul 06 21:39:12.779052 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:12.778674    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65","Type":"ContainerDied","Data":"3d14509089249766738cc9800b8cb924cc5a2c0bc457435b8d2985089699223b"}
Jul 06 21:39:12.780001 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:12.779983    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 21:39:40.764952 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:40.764843    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm"
Jul 06 21:39:59.066894 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:59.066862    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"796ba5a7-c7bc-4027-bc2c-0050ed9f9d65","Type":"ContainerStarted","Data":"cde71696b487dff112b86cce14742ab5e54a28a2e786ec6ea92a005680651803"}
Jul 06 21:39:59.091995 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:39:59.091928    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" podStartSLOduration=1.974140257 podStartE2EDuration="53.091905778s" totalImagesPullingTime="46.13427949s" totalInitContainerRuntime="4.983486031s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:39:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:39:59.088905217 +0000 UTC m=+3077.555146470" watchObservedRunningTime="2026-07-06 21:39:59.091905778 +0000 UTC m=+3077.558147034"
Jul 06 21:53:42.373769 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:53:42.373736    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 21:53:42.382485 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:53:42.382444    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 21:53:42.389795 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:53:42.389776    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 21:53:42.397637 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:53:42.397619    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 21:54:16.717748 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.717711    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"]
Jul 06 21:54:16.720581 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.720561    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:16.723841 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.723814    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-epp-sa-dockercfg-mw5qq\""
Jul 06 21:54:16.723947 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.723850    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Jul 06 21:54:16.733371 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.733347    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"]
Jul 06 21:54:16.768658 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.768617    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38797575-08fe-44bf-a641-0f948888b916-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l\" (UID: \"38797575-08fe-44bf-a641-0f948888b916\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:16.768856 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.768683    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs96l\" (UniqueName: \"kubernetes.io/projected/38797575-08fe-44bf-a641-0f948888b916-kube-api-access-fs96l\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l\" (UID: \"38797575-08fe-44bf-a641-0f948888b916\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:16.768856 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.768758    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/38797575-08fe-44bf-a641-0f948888b916-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l\" (UID: \"38797575-08fe-44bf-a641-0f948888b916\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:16.870054 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.870009    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38797575-08fe-44bf-a641-0f948888b916-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l\" (UID: \"38797575-08fe-44bf-a641-0f948888b916\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:16.870229 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.870068    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs96l\" (UniqueName: \"kubernetes.io/projected/38797575-08fe-44bf-a641-0f948888b916-kube-api-access-fs96l\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l\" (UID: \"38797575-08fe-44bf-a641-0f948888b916\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:16.870229 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.870147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/38797575-08fe-44bf-a641-0f948888b916-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l\" (UID: \"38797575-08fe-44bf-a641-0f948888b916\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:16.870627 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.870606    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/38797575-08fe-44bf-a641-0f948888b916-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l\" (UID: \"38797575-08fe-44bf-a641-0f948888b916\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:16.872665 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.872645    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38797575-08fe-44bf-a641-0f948888b916-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l\" (UID: \"38797575-08fe-44bf-a641-0f948888b916\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:16.878500 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:16.878448    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs96l\" (UniqueName: \"kubernetes.io/projected/38797575-08fe-44bf-a641-0f948888b916-kube-api-access-fs96l\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l\" (UID: \"38797575-08fe-44bf-a641-0f948888b916\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:17.030498 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:17.030439    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:17.172850 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:17.172827    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"]
Jul 06 21:54:17.175128 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:54:17.175103    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38797575_08fe_44bf_a641_0f948888b916.slice/crio-a60daf474b1574e6ebcdaf238b088661c1b025c257244c59579ef0b17c4b86b2 WatchSource:0}: Error finding container a60daf474b1574e6ebcdaf238b088661c1b025c257244c59579ef0b17c4b86b2: Status 404 returned error can't find the container with id a60daf474b1574e6ebcdaf238b088661c1b025c257244c59579ef0b17c4b86b2
Jul 06 21:54:17.672928 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:17.672889    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l" event={"ID":"38797575-08fe-44bf-a641-0f948888b916","Type":"ContainerStarted","Data":"eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e"}
Jul 06 21:54:17.673114 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:17.672942    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l" event={"ID":"38797575-08fe-44bf-a641-0f948888b916","Type":"ContainerStarted","Data":"a60daf474b1574e6ebcdaf238b088661c1b025c257244c59579ef0b17c4b86b2"}
Jul 06 21:54:17.673114 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:17.672962    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:17.707133 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:17.707066    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l" podStartSLOduration=1.707043349 podStartE2EDuration="1.707043349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:54:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:54:17.702040868 +0000 UTC m=+3936.168282122" watchObservedRunningTime="2026-07-06 21:54:17.707043349 +0000 UTC m=+3936.173284605"
Jul 06 21:54:48.679521 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:48.679428    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:58.018100 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.018060    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"]
Jul 06 21:54:58.028056 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:54:58.028017    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:54:58.028056 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:54:58.028056 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:54:58.028056 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e" cmd=["/bin/sleep","15"]
Jul 06 21:54:58.028479 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.028070    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l" message="" err=<
Jul 06 21:54:58.028479 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:54:58.028479 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:54:58.028479 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:54:58.028479 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:54:58.028082    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:54:58.028479 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:54:58.028479 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:54:58.028479 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l" podUID="38797575-08fe-44bf-a641-0f948888b916" containerName="main" containerID="cri-o://eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e"
Jul 06 21:54:58.028479 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.028136    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l" podUID="38797575-08fe-44bf-a641-0f948888b916" containerName="main" containerID="cri-o://eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e" gracePeriod=60
Jul 06 21:54:58.293589 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.293564    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:58.349519 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.349485    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38797575-08fe-44bf-a641-0f948888b916-tls-certs\") pod \"38797575-08fe-44bf-a641-0f948888b916\" (UID: \"38797575-08fe-44bf-a641-0f948888b916\") "
Jul 06 21:54:58.349720 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.349530    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/38797575-08fe-44bf-a641-0f948888b916-tokenizer-uds\") pod \"38797575-08fe-44bf-a641-0f948888b916\" (UID: \"38797575-08fe-44bf-a641-0f948888b916\") "
Jul 06 21:54:58.349720 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.349626    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs96l\" (UniqueName: \"kubernetes.io/projected/38797575-08fe-44bf-a641-0f948888b916-kube-api-access-fs96l\") pod \"38797575-08fe-44bf-a641-0f948888b916\" (UID: \"38797575-08fe-44bf-a641-0f948888b916\") "
Jul 06 21:54:58.349874 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.349850    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38797575-08fe-44bf-a641-0f948888b916-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "38797575-08fe-44bf-a641-0f948888b916" (UID: "38797575-08fe-44bf-a641-0f948888b916"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:54:58.351744 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.351713    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38797575-08fe-44bf-a641-0f948888b916-kube-api-access-fs96l" (OuterVolumeSpecName: "kube-api-access-fs96l") pod "38797575-08fe-44bf-a641-0f948888b916" (UID: "38797575-08fe-44bf-a641-0f948888b916"). InnerVolumeSpecName "kube-api-access-fs96l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:54:58.351744 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.351723    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38797575-08fe-44bf-a641-0f948888b916-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "38797575-08fe-44bf-a641-0f948888b916" (UID: "38797575-08fe-44bf-a641-0f948888b916"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:54:58.450832 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.450797    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs96l\" (UniqueName: \"kubernetes.io/projected/38797575-08fe-44bf-a641-0f948888b916-kube-api-access-fs96l\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:54:58.450832 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.450827    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38797575-08fe-44bf-a641-0f948888b916-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:54:58.450832 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.450837    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/38797575-08fe-44bf-a641-0f948888b916-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:54:58.891244 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.891208    2566 generic.go:356] "Generic (PLEG): container finished" podID="38797575-08fe-44bf-a641-0f948888b916" containerID="eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e" exitCode=0
Jul 06 21:54:58.891429 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.891259    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"
Jul 06 21:54:58.891429 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.891258    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l" event={"ID":"38797575-08fe-44bf-a641-0f948888b916","Type":"ContainerDied","Data":"eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e"}
Jul 06 21:54:58.891429 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.891392    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l" event={"ID":"38797575-08fe-44bf-a641-0f948888b916","Type":"ContainerDied","Data":"a60daf474b1574e6ebcdaf238b088661c1b025c257244c59579ef0b17c4b86b2"}
Jul 06 21:54:58.891429 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.891426    2566 scope.go:117] "RemoveContainer" containerID="eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e"
Jul 06 21:54:58.901579 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.901550    2566 scope.go:117] "RemoveContainer" containerID="eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e"
Jul 06 21:54:58.901897 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:54:58.901873    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e\": container with ID starting with eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e not found: ID does not exist" containerID="eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e"
Jul 06 21:54:58.901971 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.901908    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e"} err="failed to get container status \"eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e\": rpc error: code = NotFound desc = could not find container \"eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e\": container with ID starting with eca184b1b9b55c5fd2cf97c2c7c0724d4331bf3937d6d530f1716f7eef7bdb3e not found: ID does not exist"
Jul 06 21:54:58.914751 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.914723    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"]
Jul 06 21:54:58.918254 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:54:58.918229    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfvmh8l"]
Jul 06 21:55:00.002233 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:00.002187    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38797575-08fe-44bf-a641-0f948888b916" path="/var/lib/kubelet/pods/38797575-08fe-44bf-a641-0f948888b916/volumes"
Jul 06 21:55:35.652082 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.652045    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"]
Jul 06 21:55:35.652552 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.652479    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38797575-08fe-44bf-a641-0f948888b916" containerName="main"
Jul 06 21:55:35.652552 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.652491    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="38797575-08fe-44bf-a641-0f948888b916" containerName="main"
Jul 06 21:55:35.652645 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.652582    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="38797575-08fe-44bf-a641-0f948888b916" containerName="main"
Jul 06 21:55:35.655565 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.655545    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:55:35.658758 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.658728    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-sa-dockercfg-q4w6b\""
Jul 06 21:55:35.658885 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.658759    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs\""
Jul 06 21:55:35.667201 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.667175    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"]
Jul 06 21:55:35.780623 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.780568    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbd9v\" (UniqueName: \"kubernetes.io/projected/fc66db48-94e5-48b1-b14c-3392cac60c26-kube-api-access-tbd9v\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d\" (UID: \"fc66db48-94e5-48b1-b14c-3392cac60c26\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:55:35.780827 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.780643    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/fc66db48-94e5-48b1-b14c-3392cac60c26-tokenizer-uds\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d\" (UID: \"fc66db48-94e5-48b1-b14c-3392cac60c26\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:55:35.780827 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.780704    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc66db48-94e5-48b1-b14c-3392cac60c26-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d\" (UID: \"fc66db48-94e5-48b1-b14c-3392cac60c26\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:55:35.882265 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.882229    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbd9v\" (UniqueName: \"kubernetes.io/projected/fc66db48-94e5-48b1-b14c-3392cac60c26-kube-api-access-tbd9v\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d\" (UID: \"fc66db48-94e5-48b1-b14c-3392cac60c26\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:55:35.882441 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.882278    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/fc66db48-94e5-48b1-b14c-3392cac60c26-tokenizer-uds\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d\" (UID: \"fc66db48-94e5-48b1-b14c-3392cac60c26\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:55:35.882441 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.882307    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc66db48-94e5-48b1-b14c-3392cac60c26-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d\" (UID: \"fc66db48-94e5-48b1-b14c-3392cac60c26\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:55:35.882709 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.882690    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/fc66db48-94e5-48b1-b14c-3392cac60c26-tokenizer-uds\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d\" (UID: \"fc66db48-94e5-48b1-b14c-3392cac60c26\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:55:35.884832 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.884809    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc66db48-94e5-48b1-b14c-3392cac60c26-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d\" (UID: \"fc66db48-94e5-48b1-b14c-3392cac60c26\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:55:35.890098 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.890075    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbd9v\" (UniqueName: \"kubernetes.io/projected/fc66db48-94e5-48b1-b14c-3392cac60c26-kube-api-access-tbd9v\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d\" (UID: \"fc66db48-94e5-48b1-b14c-3392cac60c26\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:55:35.965409 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:35.965309    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:55:36.106311 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:36.106274    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"]
Jul 06 21:55:36.107951 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:55:36.107916    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc66db48_94e5_48b1_b14c_3392cac60c26.slice/crio-11cbae50498c04bf87499482f83bbdceb7953afdcf49da27af72be6226689050 WatchSource:0}: Error finding container 11cbae50498c04bf87499482f83bbdceb7953afdcf49da27af72be6226689050: Status 404 returned error can't find the container with id 11cbae50498c04bf87499482f83bbdceb7953afdcf49da27af72be6226689050
Jul 06 21:55:37.099847 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:37.099786    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d" event={"ID":"fc66db48-94e5-48b1-b14c-3392cac60c26","Type":"ContainerStarted","Data":"aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318"}
Jul 06 21:55:37.099847 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:37.099853    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d" event={"ID":"fc66db48-94e5-48b1-b14c-3392cac60c26","Type":"ContainerStarted","Data":"11cbae50498c04bf87499482f83bbdceb7953afdcf49da27af72be6226689050"}
Jul 06 21:55:37.100265 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:37.099970    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:55:37.120813 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:55:37.120757    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d" podStartSLOduration=2.120742903 podStartE2EDuration="2.120742903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:55:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:55:37.118248984 +0000 UTC m=+4015.584490240" watchObservedRunningTime="2026-07-06 21:55:37.120742903 +0000 UTC m=+4015.586984201"
Jul 06 21:56:08.108116 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:08.108037    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:56:34.966958 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:34.966922    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"]
Jul 06 21:56:34.977897 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:56:34.977854    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:56:34.977897 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:56:34.977897 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:56:34.977897 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318" cmd=["/bin/sleep","15"]
Jul 06 21:56:34.978228 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:34.977921    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d" message="" err=<
Jul 06 21:56:34.978228 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:56:34.978228 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:56:34.978228 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:56:34.978228 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:56:34.977938    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:56:34.978228 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:56:34.978228 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:56:34.978228 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d" podUID="fc66db48-94e5-48b1-b14c-3392cac60c26" containerName="main" containerID="cri-o://aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318"
Jul 06 21:56:34.978228 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:34.977998    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d" podUID="fc66db48-94e5-48b1-b14c-3392cac60c26" containerName="main" containerID="cri-o://aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318" gracePeriod=60
Jul 06 21:56:35.235330 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.235306    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:56:35.320590 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.320551    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbd9v\" (UniqueName: \"kubernetes.io/projected/fc66db48-94e5-48b1-b14c-3392cac60c26-kube-api-access-tbd9v\") pod \"fc66db48-94e5-48b1-b14c-3392cac60c26\" (UID: \"fc66db48-94e5-48b1-b14c-3392cac60c26\") "
Jul 06 21:56:35.320767 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.320611    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/fc66db48-94e5-48b1-b14c-3392cac60c26-tokenizer-uds\") pod \"fc66db48-94e5-48b1-b14c-3392cac60c26\" (UID: \"fc66db48-94e5-48b1-b14c-3392cac60c26\") "
Jul 06 21:56:35.320767 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.320712    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc66db48-94e5-48b1-b14c-3392cac60c26-tls-certs\") pod \"fc66db48-94e5-48b1-b14c-3392cac60c26\" (UID: \"fc66db48-94e5-48b1-b14c-3392cac60c26\") "
Jul 06 21:56:35.320970 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.320945    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc66db48-94e5-48b1-b14c-3392cac60c26-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "fc66db48-94e5-48b1-b14c-3392cac60c26" (UID: "fc66db48-94e5-48b1-b14c-3392cac60c26"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:56:35.322792 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.322763    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc66db48-94e5-48b1-b14c-3392cac60c26-kube-api-access-tbd9v" (OuterVolumeSpecName: "kube-api-access-tbd9v") pod "fc66db48-94e5-48b1-b14c-3392cac60c26" (UID: "fc66db48-94e5-48b1-b14c-3392cac60c26"). InnerVolumeSpecName "kube-api-access-tbd9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:56:35.322905 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.322806    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc66db48-94e5-48b1-b14c-3392cac60c26-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fc66db48-94e5-48b1-b14c-3392cac60c26" (UID: "fc66db48-94e5-48b1-b14c-3392cac60c26"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:56:35.414352 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.414322    2566 generic.go:356] "Generic (PLEG): container finished" podID="fc66db48-94e5-48b1-b14c-3392cac60c26" containerID="aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318" exitCode=0
Jul 06 21:56:35.414527 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.414360    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"
Jul 06 21:56:35.414527 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.414401    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d" event={"ID":"fc66db48-94e5-48b1-b14c-3392cac60c26","Type":"ContainerDied","Data":"aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318"}
Jul 06 21:56:35.414527 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.414465    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d" event={"ID":"fc66db48-94e5-48b1-b14c-3392cac60c26","Type":"ContainerDied","Data":"11cbae50498c04bf87499482f83bbdceb7953afdcf49da27af72be6226689050"}
Jul 06 21:56:35.414527 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.414496    2566 scope.go:117] "RemoveContainer" containerID="aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318"
Jul 06 21:56:35.421734 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.421711    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbd9v\" (UniqueName: \"kubernetes.io/projected/fc66db48-94e5-48b1-b14c-3392cac60c26-kube-api-access-tbd9v\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:56:35.421854 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.421740    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/fc66db48-94e5-48b1-b14c-3392cac60c26-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:56:35.421854 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.421757    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc66db48-94e5-48b1-b14c-3392cac60c26-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:56:35.424328 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.424312    2566 scope.go:117] "RemoveContainer" containerID="aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318"
Jul 06 21:56:35.424600 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:56:35.424582    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318\": container with ID starting with aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318 not found: ID does not exist" containerID="aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318"
Jul 06 21:56:35.424661 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.424607    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318"} err="failed to get container status \"aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318\": rpc error: code = NotFound desc = could not find container \"aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318\": container with ID starting with aac233eb68ed1820f980dac8eb89ae248cf669e55d4532de53d44cb4de71c318 not found: ID does not exist"
Jul 06 21:56:35.436418 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.436395    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"]
Jul 06 21:56:35.442417 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:35.442393    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schec5q9d"]
Jul 06 21:56:36.002079 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:56:36.002046    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc66db48-94e5-48b1-b14c-3392cac60c26" path="/var/lib/kubelet/pods/fc66db48-94e5-48b1-b14c-3392cac60c26/volumes"
Jul 06 21:57:16.523113 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.523081    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"]
Jul 06 21:57:16.523598 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.523478    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc66db48-94e5-48b1-b14c-3392cac60c26" containerName="main"
Jul 06 21:57:16.523598 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.523491    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc66db48-94e5-48b1-b14c-3392cac60c26" containerName="main"
Jul 06 21:57:16.523598 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.523573    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc66db48-94e5-48b1-b14c-3392cac60c26" containerName="main"
Jul 06 21:57:16.526358 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.526336    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:57:16.529763 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.529739    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Jul 06 21:57:16.529947 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.529778    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-epp-sa-dockercfg-jkgtt\""
Jul 06 21:57:16.537373 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.537352    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"]
Jul 06 21:57:16.698533 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.698483    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a55621ef-b6f7-4a93-91d4-42e19818539f-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9\" (UID: \"a55621ef-b6f7-4a93-91d4-42e19818539f\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:57:16.698533 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.698530    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a55621ef-b6f7-4a93-91d4-42e19818539f-tokenizer-uds\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9\" (UID: \"a55621ef-b6f7-4a93-91d4-42e19818539f\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:57:16.698779 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.698552    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gpcp\" (UniqueName: \"kubernetes.io/projected/a55621ef-b6f7-4a93-91d4-42e19818539f-kube-api-access-6gpcp\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9\" (UID: \"a55621ef-b6f7-4a93-91d4-42e19818539f\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:57:16.799168 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.799072    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a55621ef-b6f7-4a93-91d4-42e19818539f-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9\" (UID: \"a55621ef-b6f7-4a93-91d4-42e19818539f\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:57:16.799168 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.799110    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a55621ef-b6f7-4a93-91d4-42e19818539f-tokenizer-uds\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9\" (UID: \"a55621ef-b6f7-4a93-91d4-42e19818539f\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:57:16.799168 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.799136    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gpcp\" (UniqueName: \"kubernetes.io/projected/a55621ef-b6f7-4a93-91d4-42e19818539f-kube-api-access-6gpcp\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9\" (UID: \"a55621ef-b6f7-4a93-91d4-42e19818539f\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:57:16.799669 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.799651    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a55621ef-b6f7-4a93-91d4-42e19818539f-tokenizer-uds\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9\" (UID: \"a55621ef-b6f7-4a93-91d4-42e19818539f\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:57:16.801632 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.801611    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a55621ef-b6f7-4a93-91d4-42e19818539f-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9\" (UID: \"a55621ef-b6f7-4a93-91d4-42e19818539f\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:57:16.808590 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.808556    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gpcp\" (UniqueName: \"kubernetes.io/projected/a55621ef-b6f7-4a93-91d4-42e19818539f-kube-api-access-6gpcp\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9\" (UID: \"a55621ef-b6f7-4a93-91d4-42e19818539f\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:57:16.836634 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.836601    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:57:16.995293 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:16.995259    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"]
Jul 06 21:57:16.998434 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:57:16.998397    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda55621ef_b6f7_4a93_91d4_42e19818539f.slice/crio-e0a1e2b2a398cfb73e02d7bea9548c8261269a926a4b73ba646becb43ab595ad WatchSource:0}: Error finding container e0a1e2b2a398cfb73e02d7bea9548c8261269a926a4b73ba646becb43ab595ad: Status 404 returned error can't find the container with id e0a1e2b2a398cfb73e02d7bea9548c8261269a926a4b73ba646becb43ab595ad
Jul 06 21:57:17.645705 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:17.645668    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9" event={"ID":"a55621ef-b6f7-4a93-91d4-42e19818539f","Type":"ContainerStarted","Data":"36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d"}
Jul 06 21:57:17.646192 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:17.645716    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9" event={"ID":"a55621ef-b6f7-4a93-91d4-42e19818539f","Type":"ContainerStarted","Data":"e0a1e2b2a398cfb73e02d7bea9548c8261269a926a4b73ba646becb43ab595ad"}
Jul 06 21:57:17.646192 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:17.645803    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:57:17.667625 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:17.667572    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9" podStartSLOduration=1.667554905 podStartE2EDuration="1.667554905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:57:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:57:17.666146356 +0000 UTC m=+4116.132387610" watchObservedRunningTime="2026-07-06 21:57:17.667554905 +0000 UTC m=+4116.133796159"
Jul 06 21:57:48.654634 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:57:48.654555    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:58:11.508403 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.508357    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"]
Jul 06 21:58:11.517763 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:58:11.517720    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:58:11.517763 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:58:11.517763 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:58:11.517763 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d" cmd=["/bin/sleep","15"]
Jul 06 21:58:11.518062 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.517781    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9" message="" err=<
Jul 06 21:58:11.518062 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:58:11.518062 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:58:11.518062 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:58:11.518062 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:58:11.517798    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:58:11.518062 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:58:11.518062 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:58:11.518062 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9" podUID="a55621ef-b6f7-4a93-91d4-42e19818539f" containerName="main" containerID="cri-o://36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d"
Jul 06 21:58:11.518062 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.517855    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9" podUID="a55621ef-b6f7-4a93-91d4-42e19818539f" containerName="main" containerID="cri-o://36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d" gracePeriod=60
Jul 06 21:58:11.777834 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.777804    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:58:11.883139 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.883103    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a55621ef-b6f7-4a93-91d4-42e19818539f-tokenizer-uds\") pod \"a55621ef-b6f7-4a93-91d4-42e19818539f\" (UID: \"a55621ef-b6f7-4a93-91d4-42e19818539f\") "
Jul 06 21:58:11.883139 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.883141    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gpcp\" (UniqueName: \"kubernetes.io/projected/a55621ef-b6f7-4a93-91d4-42e19818539f-kube-api-access-6gpcp\") pod \"a55621ef-b6f7-4a93-91d4-42e19818539f\" (UID: \"a55621ef-b6f7-4a93-91d4-42e19818539f\") "
Jul 06 21:58:11.883381 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.883234    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a55621ef-b6f7-4a93-91d4-42e19818539f-tls-certs\") pod \"a55621ef-b6f7-4a93-91d4-42e19818539f\" (UID: \"a55621ef-b6f7-4a93-91d4-42e19818539f\") "
Jul 06 21:58:11.883431 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.883382    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a55621ef-b6f7-4a93-91d4-42e19818539f-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "a55621ef-b6f7-4a93-91d4-42e19818539f" (UID: "a55621ef-b6f7-4a93-91d4-42e19818539f"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:58:11.883495 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.883477    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a55621ef-b6f7-4a93-91d4-42e19818539f-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:58:11.885421 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.885393    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a55621ef-b6f7-4a93-91d4-42e19818539f-kube-api-access-6gpcp" (OuterVolumeSpecName: "kube-api-access-6gpcp") pod "a55621ef-b6f7-4a93-91d4-42e19818539f" (UID: "a55621ef-b6f7-4a93-91d4-42e19818539f"). InnerVolumeSpecName "kube-api-access-6gpcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:58:11.885552 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.885448    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a55621ef-b6f7-4a93-91d4-42e19818539f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a55621ef-b6f7-4a93-91d4-42e19818539f" (UID: "a55621ef-b6f7-4a93-91d4-42e19818539f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:58:11.941137 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.941102    2566 generic.go:356] "Generic (PLEG): container finished" podID="a55621ef-b6f7-4a93-91d4-42e19818539f" containerID="36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d" exitCode=0
Jul 06 21:58:11.941320 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.941158    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"
Jul 06 21:58:11.941320 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.941187    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9" event={"ID":"a55621ef-b6f7-4a93-91d4-42e19818539f","Type":"ContainerDied","Data":"36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d"}
Jul 06 21:58:11.941320 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.941238    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9" event={"ID":"a55621ef-b6f7-4a93-91d4-42e19818539f","Type":"ContainerDied","Data":"e0a1e2b2a398cfb73e02d7bea9548c8261269a926a4b73ba646becb43ab595ad"}
Jul 06 21:58:11.941320 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.941269    2566 scope.go:117] "RemoveContainer" containerID="36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d"
Jul 06 21:58:11.951762 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.951746    2566 scope.go:117] "RemoveContainer" containerID="36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d"
Jul 06 21:58:11.952049 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:58:11.952031    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d\": container with ID starting with 36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d not found: ID does not exist" containerID="36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d"
Jul 06 21:58:11.952115 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.952057    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d"} err="failed to get container status \"36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d\": rpc error: code = NotFound desc = could not find container \"36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d\": container with ID starting with 36d8c3715ba5227d0e7b88d290ebbc73c5cba98b7af8d373f77d5f647a24ca3d not found: ID does not exist"
Jul 06 21:58:11.965694 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.965663    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"]
Jul 06 21:58:11.971325 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.971301    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87snbv9"]
Jul 06 21:58:11.984075 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.984052    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a55621ef-b6f7-4a93-91d4-42e19818539f-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:58:11.984075 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:11.984076    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gpcp\" (UniqueName: \"kubernetes.io/projected/a55621ef-b6f7-4a93-91d4-42e19818539f-kube-api-access-6gpcp\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:58:12.002158 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:12.002131    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a55621ef-b6f7-4a93-91d4-42e19818539f" path="/var/lib/kubelet/pods/a55621ef-b6f7-4a93-91d4-42e19818539f/volumes"
Jul 06 21:58:52.427016 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.426974    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"]
Jul 06 21:58:52.427687 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.427559    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a55621ef-b6f7-4a93-91d4-42e19818539f" containerName="main"
Jul 06 21:58:52.427687 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.427579    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a55621ef-b6f7-4a93-91d4-42e19818539f" containerName="main"
Jul 06 21:58:52.427818 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.427688    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a55621ef-b6f7-4a93-91d4-42e19818539f" containerName="main"
Jul 06 21:58:52.431105 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.431083    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:58:52.434957 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.434931    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-jjt7d\""
Jul 06 21:58:52.436009 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.435987    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Jul 06 21:58:52.445177 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.445147    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"]
Jul 06 21:58:52.570872 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.570825    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8395ea79-4d7e-43db-a18d-1376b01e6838-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh\" (UID: \"8395ea79-4d7e-43db-a18d-1376b01e6838\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:58:52.571055 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.570931    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkg9s\" (UniqueName: \"kubernetes.io/projected/8395ea79-4d7e-43db-a18d-1376b01e6838-kube-api-access-nkg9s\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh\" (UID: \"8395ea79-4d7e-43db-a18d-1376b01e6838\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:58:52.571055 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.570992    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8395ea79-4d7e-43db-a18d-1376b01e6838-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh\" (UID: \"8395ea79-4d7e-43db-a18d-1376b01e6838\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:58:52.671910 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.671869    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8395ea79-4d7e-43db-a18d-1376b01e6838-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh\" (UID: \"8395ea79-4d7e-43db-a18d-1376b01e6838\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:58:52.672092 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.671941    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8395ea79-4d7e-43db-a18d-1376b01e6838-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh\" (UID: \"8395ea79-4d7e-43db-a18d-1376b01e6838\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:58:52.672092 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.672004    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkg9s\" (UniqueName: \"kubernetes.io/projected/8395ea79-4d7e-43db-a18d-1376b01e6838-kube-api-access-nkg9s\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh\" (UID: \"8395ea79-4d7e-43db-a18d-1376b01e6838\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:58:52.672339 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.672316    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8395ea79-4d7e-43db-a18d-1376b01e6838-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh\" (UID: \"8395ea79-4d7e-43db-a18d-1376b01e6838\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:58:52.674513 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.674489    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8395ea79-4d7e-43db-a18d-1376b01e6838-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh\" (UID: \"8395ea79-4d7e-43db-a18d-1376b01e6838\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:58:52.680994 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.680932    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkg9s\" (UniqueName: \"kubernetes.io/projected/8395ea79-4d7e-43db-a18d-1376b01e6838-kube-api-access-nkg9s\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh\" (UID: \"8395ea79-4d7e-43db-a18d-1376b01e6838\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:58:52.744629 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.744594    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:58:52.893582 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:52.893554    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"]
Jul 06 21:58:52.894382 ip-10-0-131-165 kubenswrapper[2566]: W0706 21:58:52.894357    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8395ea79_4d7e_43db_a18d_1376b01e6838.slice/crio-d3a003e0cb8e2b6f73f503cfda39ae84259b30fa73b947e1c11bbc7b4e8f752a WatchSource:0}: Error finding container d3a003e0cb8e2b6f73f503cfda39ae84259b30fa73b947e1c11bbc7b4e8f752a: Status 404 returned error can't find the container with id d3a003e0cb8e2b6f73f503cfda39ae84259b30fa73b947e1c11bbc7b4e8f752a
Jul 06 21:58:53.158966 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:53.158930    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh" event={"ID":"8395ea79-4d7e-43db-a18d-1376b01e6838","Type":"ContainerStarted","Data":"3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141"}
Jul 06 21:58:53.159164 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:53.158982    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh" event={"ID":"8395ea79-4d7e-43db-a18d-1376b01e6838","Type":"ContainerStarted","Data":"d3a003e0cb8e2b6f73f503cfda39ae84259b30fa73b947e1c11bbc7b4e8f752a"}
Jul 06 21:58:53.159164 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:53.159032    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:58:53.180352 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:58:53.180292    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh" podStartSLOduration=1.180273218 podStartE2EDuration="1.180273218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:58:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:58:53.180081179 +0000 UTC m=+4211.646322436" watchObservedRunningTime="2026-07-06 21:58:53.180273218 +0000 UTC m=+4211.646514511"
Jul 06 21:59:24.166725 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:24.166640    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:59:33.823501 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:33.823435    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"]
Jul 06 21:59:33.832661 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:59:33.832608    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 21:59:33.832661 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:59:33.832661 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:59:33.832661 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141" cmd=["/bin/sleep","15"]
Jul 06 21:59:33.832940 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:33.832676    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh" message="" err=<
Jul 06 21:59:33.832940 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:59:33.832940 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:59:33.832940 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 21:59:33.832940 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:59:33.832688    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 21:59:33.832940 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 21:59:33.832940 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 21:59:33.832940 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh" podUID="8395ea79-4d7e-43db-a18d-1376b01e6838" containerName="main" containerID="cri-o://3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141"
Jul 06 21:59:33.832940 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:33.832738    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh" podUID="8395ea79-4d7e-43db-a18d-1376b01e6838" containerName="main" containerID="cri-o://3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141" gracePeriod=60
Jul 06 21:59:34.095974 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.095946    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:59:34.242211 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.242167    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkg9s\" (UniqueName: \"kubernetes.io/projected/8395ea79-4d7e-43db-a18d-1376b01e6838-kube-api-access-nkg9s\") pod \"8395ea79-4d7e-43db-a18d-1376b01e6838\" (UID: \"8395ea79-4d7e-43db-a18d-1376b01e6838\") "
Jul 06 21:59:34.242396 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.242276    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8395ea79-4d7e-43db-a18d-1376b01e6838-tokenizer-uds\") pod \"8395ea79-4d7e-43db-a18d-1376b01e6838\" (UID: \"8395ea79-4d7e-43db-a18d-1376b01e6838\") "
Jul 06 21:59:34.242396 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.242371    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8395ea79-4d7e-43db-a18d-1376b01e6838-tls-certs\") pod \"8395ea79-4d7e-43db-a18d-1376b01e6838\" (UID: \"8395ea79-4d7e-43db-a18d-1376b01e6838\") "
Jul 06 21:59:34.242569 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.242541    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8395ea79-4d7e-43db-a18d-1376b01e6838-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "8395ea79-4d7e-43db-a18d-1376b01e6838" (UID: "8395ea79-4d7e-43db-a18d-1376b01e6838"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:59:34.244227 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.244201    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8395ea79-4d7e-43db-a18d-1376b01e6838-kube-api-access-nkg9s" (OuterVolumeSpecName: "kube-api-access-nkg9s") pod "8395ea79-4d7e-43db-a18d-1376b01e6838" (UID: "8395ea79-4d7e-43db-a18d-1376b01e6838"). InnerVolumeSpecName "kube-api-access-nkg9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:59:34.244421 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.244402    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8395ea79-4d7e-43db-a18d-1376b01e6838-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8395ea79-4d7e-43db-a18d-1376b01e6838" (UID: "8395ea79-4d7e-43db-a18d-1376b01e6838"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:59:34.343261 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.343179    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8395ea79-4d7e-43db-a18d-1376b01e6838-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:59:34.343261 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.343210    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8395ea79-4d7e-43db-a18d-1376b01e6838-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:59:34.343261 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.343220    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkg9s\" (UniqueName: \"kubernetes.io/projected/8395ea79-4d7e-43db-a18d-1376b01e6838-kube-api-access-nkg9s\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 21:59:34.391039 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.391006    2566 generic.go:356] "Generic (PLEG): container finished" podID="8395ea79-4d7e-43db-a18d-1376b01e6838" containerID="3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141" exitCode=0
Jul 06 21:59:34.391237 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.391043    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"
Jul 06 21:59:34.391237 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.391086    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh" event={"ID":"8395ea79-4d7e-43db-a18d-1376b01e6838","Type":"ContainerDied","Data":"3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141"}
Jul 06 21:59:34.391237 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.391142    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh" event={"ID":"8395ea79-4d7e-43db-a18d-1376b01e6838","Type":"ContainerDied","Data":"d3a003e0cb8e2b6f73f503cfda39ae84259b30fa73b947e1c11bbc7b4e8f752a"}
Jul 06 21:59:34.391237 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.391168    2566 scope.go:117] "RemoveContainer" containerID="3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141"
Jul 06 21:59:34.401285 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.401269    2566 scope.go:117] "RemoveContainer" containerID="3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141"
Jul 06 21:59:34.401583 ip-10-0-131-165 kubenswrapper[2566]: E0706 21:59:34.401564    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141\": container with ID starting with 3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141 not found: ID does not exist" containerID="3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141"
Jul 06 21:59:34.401628 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.401591    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141"} err="failed to get container status \"3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141\": rpc error: code = NotFound desc = could not find container \"3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141\": container with ID starting with 3ce5e790b1fa1b0e8ffa4e5c1295e45bed09b20e3bf4e08b8728d7972be44141 not found: ID does not exist"
Jul 06 21:59:34.418474 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.416276    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"]
Jul 06 21:59:34.424640 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:34.424615    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dptxbh"]
Jul 06 21:59:36.001953 ip-10-0-131-165 kubenswrapper[2566]: I0706 21:59:36.001834    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8395ea79-4d7e-43db-a18d-1376b01e6838" path="/var/lib/kubelet/pods/8395ea79-4d7e-43db-a18d-1376b01e6838/volumes"
Jul 06 22:00:25.243116 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.243081    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"]
Jul 06 22:00:25.243889 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.243479    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8395ea79-4d7e-43db-a18d-1376b01e6838" containerName="main"
Jul 06 22:00:25.243889 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.243491    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8395ea79-4d7e-43db-a18d-1376b01e6838" containerName="main"
Jul 06 22:00:25.243889 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.243566    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8395ea79-4d7e-43db-a18d-1376b01e6838" containerName="main"
Jul 06 22:00:25.246608 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.246587    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.248755 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.248731    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-custom-template-test-epp-sa-dockercfg-vhtnt\""
Jul 06 22:00:25.249500 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.249481    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-custom-template-test-kserve-self-signed-certs\""
Jul 06 22:00:25.256597 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.256575    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"]
Jul 06 22:00:25.406992 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.406960    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.407167 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.407016    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.407167 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.407035    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.407167 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.407103    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tokenizer-uds\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.407167 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.407131    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp8rp\" (UniqueName: \"kubernetes.io/projected/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-kube-api-access-kp8rp\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.508297 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.508212    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.508297 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.508252    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.508297 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.508298    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tokenizer-uds\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.508604 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.508320    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp8rp\" (UniqueName: \"kubernetes.io/projected/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-kube-api-access-kp8rp\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.508604 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.508350    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.508802 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.508742    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tokenizer-uds\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.508802 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.508763    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.508926 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.508827    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.510831 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.510814    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.516150 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.516129    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp8rp\" (UniqueName: \"kubernetes.io/projected/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-kube-api-access-kp8rp\") pod \"scheduler-custom-template-test-kserve-router-scheduler-56762884\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.557736 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.557547    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:25.695991 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:25.695941    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"]
Jul 06 22:00:25.697795 ip-10-0-131-165 kubenswrapper[2566]: W0706 22:00:25.697766    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbb5619b_589a_4c6f_baa1_edd56f82b1ee.slice/crio-413fc7a22f5541ae24cd78b13e07958d52a17c0ae60742a26d4aaf9aaff0f5dc WatchSource:0}: Error finding container 413fc7a22f5541ae24cd78b13e07958d52a17c0ae60742a26d4aaf9aaff0f5dc: Status 404 returned error can't find the container with id 413fc7a22f5541ae24cd78b13e07958d52a17c0ae60742a26d4aaf9aaff0f5dc
Jul 06 22:00:26.671591 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:26.671503    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884" event={"ID":"cbb5619b-589a-4c6f-baa1-edd56f82b1ee","Type":"ContainerStarted","Data":"50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e"}
Jul 06 22:00:26.671591 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:26.671565    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884" event={"ID":"cbb5619b-589a-4c6f-baa1-edd56f82b1ee","Type":"ContainerStarted","Data":"413fc7a22f5541ae24cd78b13e07958d52a17c0ae60742a26d4aaf9aaff0f5dc"}
Jul 06 22:00:26.671981 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:26.671710    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:00:26.689399 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:26.689351    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884" podStartSLOduration=1.689336465 podStartE2EDuration="1.689336465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:00:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:00:26.687404086 +0000 UTC m=+4305.153645341" watchObservedRunningTime="2026-07-06 22:00:26.689336465 +0000 UTC m=+4305.155577718"
Jul 06 22:00:57.679098 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:00:57.679067    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:01:15.833254 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:15.833213    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"]
Jul 06 22:01:15.845261 ip-10-0-131-165 kubenswrapper[2566]: E0706 22:01:15.845208    2566 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 06 22:01:15.845261 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 22:01:15.845261 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 22:01:15.845261 ip-10-0-131-165 kubenswrapper[2566]:  > containerID="50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e" cmd=["/bin/sleep","15"]
Jul 06 22:01:15.845632 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:15.845276    2566 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884" message="" err=<
Jul 06 22:01:15.845632 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 22:01:15.845632 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 22:01:15.845632 ip-10-0-131-165 kubenswrapper[2566]:  >
Jul 06 22:01:15.845632 ip-10-0-131-165 kubenswrapper[2566]: E0706 22:01:15.845292    2566 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 06 22:01:15.845632 ip-10-0-131-165 kubenswrapper[2566]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 06 22:01:15.845632 ip-10-0-131-165 kubenswrapper[2566]:         , stdout: , stderr: , exit code -1
Jul 06 22:01:15.845632 ip-10-0-131-165 kubenswrapper[2566]:  > pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884" podUID="cbb5619b-589a-4c6f-baa1-edd56f82b1ee" containerName="main" containerID="cri-o://50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e"
Jul 06 22:01:15.845632 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:15.845354    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884" podUID="cbb5619b-589a-4c6f-baa1-edd56f82b1ee" containerName="main" containerID="cri-o://50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e" gracePeriod=60
Jul 06 22:01:16.104253 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.104223    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:01:16.253451 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.253417    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tokenizer-cache\") pod \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") "
Jul 06 22:01:16.253646 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.253488    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tls-certs\") pod \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") "
Jul 06 22:01:16.253646 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.253531    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-cachi2-cache\") pod \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") "
Jul 06 22:01:16.253646 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.253561    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kp8rp\" (UniqueName: \"kubernetes.io/projected/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-kube-api-access-kp8rp\") pod \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") "
Jul 06 22:01:16.253752 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.253682    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "cbb5619b-589a-4c6f-baa1-edd56f82b1ee" (UID: "cbb5619b-589a-4c6f-baa1-edd56f82b1ee"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:01:16.253752 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.253692    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tokenizer-uds\") pod \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\" (UID: \"cbb5619b-589a-4c6f-baa1-edd56f82b1ee\") "
Jul 06 22:01:16.253959 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.253921    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-cachi2-cache" (OuterVolumeSpecName: "cachi2-cache") pod "cbb5619b-589a-4c6f-baa1-edd56f82b1ee" (UID: "cbb5619b-589a-4c6f-baa1-edd56f82b1ee"). InnerVolumeSpecName "cachi2-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:01:16.253959 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.253939    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "cbb5619b-589a-4c6f-baa1-edd56f82b1ee" (UID: "cbb5619b-589a-4c6f-baa1-edd56f82b1ee"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:01:16.253959 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.253959    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tokenizer-cache\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:01:16.255751 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.255723    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cbb5619b-589a-4c6f-baa1-edd56f82b1ee" (UID: "cbb5619b-589a-4c6f-baa1-edd56f82b1ee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:01:16.255879 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.255760    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-kube-api-access-kp8rp" (OuterVolumeSpecName: "kube-api-access-kp8rp") pod "cbb5619b-589a-4c6f-baa1-edd56f82b1ee" (UID: "cbb5619b-589a-4c6f-baa1-edd56f82b1ee"). InnerVolumeSpecName "kube-api-access-kp8rp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:01:16.355417 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.355326    2566 reconciler_common.go:299] "Volume detached for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-cachi2-cache\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:01:16.355417 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.355361    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp8rp\" (UniqueName: \"kubernetes.io/projected/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-kube-api-access-kp8rp\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:01:16.355417 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.355373    2566 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tokenizer-uds\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:01:16.355417 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.355383    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbb5619b-589a-4c6f-baa1-edd56f82b1ee-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:01:16.948181 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.948147    2566 generic.go:356] "Generic (PLEG): container finished" podID="cbb5619b-589a-4c6f-baa1-edd56f82b1ee" containerID="50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e" exitCode=0
Jul 06 22:01:16.948735 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.948219    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"
Jul 06 22:01:16.948735 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.948233    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884" event={"ID":"cbb5619b-589a-4c6f-baa1-edd56f82b1ee","Type":"ContainerDied","Data":"50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e"}
Jul 06 22:01:16.948735 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.948307    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884" event={"ID":"cbb5619b-589a-4c6f-baa1-edd56f82b1ee","Type":"ContainerDied","Data":"413fc7a22f5541ae24cd78b13e07958d52a17c0ae60742a26d4aaf9aaff0f5dc"}
Jul 06 22:01:16.948735 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.948340    2566 scope.go:117] "RemoveContainer" containerID="50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e"
Jul 06 22:01:16.958333 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.958317    2566 scope.go:117] "RemoveContainer" containerID="50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e"
Jul 06 22:01:16.958637 ip-10-0-131-165 kubenswrapper[2566]: E0706 22:01:16.958615    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e\": container with ID starting with 50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e not found: ID does not exist" containerID="50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e"
Jul 06 22:01:16.958706 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.958646    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e"} err="failed to get container status \"50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e\": rpc error: code = NotFound desc = could not find container \"50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e\": container with ID starting with 50dd5702b763b1d2a8104dd0dfdefb08d96e27c9d99d5d49cda0e4354c5ffb6e not found: ID does not exist"
Jul 06 22:01:16.971596 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.971571    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"]
Jul 06 22:01:16.975226 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:16.975203    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-56762884"]
Jul 06 22:01:18.008665 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:01:18.008628    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbb5619b-589a-4c6f-baa1-edd56f82b1ee" path="/var/lib/kubelet/pods/cbb5619b-589a-4c6f-baa1-edd56f82b1ee/volumes"
Jul 06 22:02:14.632947 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.632869    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"]
Jul 06 22:02:14.633352 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.633289    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbb5619b-589a-4c6f-baa1-edd56f82b1ee" containerName="main"
Jul 06 22:02:14.633352 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.633302    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbb5619b-589a-4c6f-baa1-edd56f82b1ee" containerName="main"
Jul 06 22:02:14.633426 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.633386    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbb5619b-589a-4c6f-baa1-edd56f82b1ee" containerName="main"
Jul 06 22:02:14.636365 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.636347    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.638825 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.638806    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Jul 06 22:02:14.649647 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.649616    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"]
Jul 06 22:02:14.672111 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.672077    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.672111 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.672111    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.672340 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.672137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx6mv\" (UniqueName: \"kubernetes.io/projected/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-kube-api-access-gx6mv\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.672340 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.672161    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.672340 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.672224    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.672340 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.672266    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.772825 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.772786    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.773021 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.772833    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.773021 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.772861    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx6mv\" (UniqueName: \"kubernetes.io/projected/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-kube-api-access-gx6mv\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.773021 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.772900    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.773021 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.772973    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.773021 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.773016    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.773281 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.773239    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.773378 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.773356    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.773444 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.773412    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.775120 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.775096    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.775399 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.775376    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.780867 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.780844    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx6mv\" (UniqueName: \"kubernetes.io/projected/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-kube-api-access-gx6mv\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:14.948602 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:14.948498    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:15.093216 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:15.093152    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"]
Jul 06 22:02:15.095996 ip-10-0-131-165 kubenswrapper[2566]: W0706 22:02:15.095970    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dd6988c_75be_4b8b_9c90_1c89cc414f3e.slice/crio-31e2b416b8e587c435a2b9852ecf26f5d23eccfe31ad5d1e6ed1f16ecf64b520 WatchSource:0}: Error finding container 31e2b416b8e587c435a2b9852ecf26f5d23eccfe31ad5d1e6ed1f16ecf64b520: Status 404 returned error can't find the container with id 31e2b416b8e587c435a2b9852ecf26f5d23eccfe31ad5d1e6ed1f16ecf64b520
Jul 06 22:02:15.266608 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:15.266569    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2" event={"ID":"1dd6988c-75be-4b8b-9c90-1c89cc414f3e","Type":"ContainerStarted","Data":"becc816771c6106024377c247ca93c54b53328bff1a5b9e9db852dd743b38d11"}
Jul 06 22:02:15.266826 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:15.266624    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2" event={"ID":"1dd6988c-75be-4b8b-9c90-1c89cc414f3e","Type":"ContainerStarted","Data":"31e2b416b8e587c435a2b9852ecf26f5d23eccfe31ad5d1e6ed1f16ecf64b520"}
Jul 06 22:02:15.286690 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:15.286637    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2" podStartSLOduration=1.286621411 podStartE2EDuration="1.286621411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:02:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:02:15.28425242 +0000 UTC m=+4413.750493675" watchObservedRunningTime="2026-07-06 22:02:15.286621411 +0000 UTC m=+4413.752862664"
Jul 06 22:02:15.948741 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:15.948696    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:24.949009 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:24.948967    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:24.961500 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:24.961472    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:02:25.332572 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:02:25.332526    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:03:32.292383 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:32.292343    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"]
Jul 06 22:03:47.313066 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.313018    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2" podUID="1dd6988c-75be-4b8b-9c90-1c89cc414f3e" containerName="main" containerID="cri-o://becc816771c6106024377c247ca93c54b53328bff1a5b9e9db852dd743b38d11" gracePeriod=45
Jul 06 22:03:47.321772 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.321741    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2" podUID="1dd6988c-75be-4b8b-9c90-1c89cc414f3e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": dial tcp 10.132.0.68:8000: connect: connection refused"
Jul 06 22:03:47.570979 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.570909    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:03:47.612632 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.612586    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-tmp-dir\") pod \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") "
Jul 06 22:03:47.612835 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.612644    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-tls-certs\") pod \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") "
Jul 06 22:03:47.612835 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.612676    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-dshm\") pod \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") "
Jul 06 22:03:47.612835 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.612714    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-model-cache\") pod \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") "
Jul 06 22:03:47.612835 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.612743    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx6mv\" (UniqueName: \"kubernetes.io/projected/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-kube-api-access-gx6mv\") pod \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") "
Jul 06 22:03:47.612835 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.612804    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-home\") pod \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\" (UID: \"1dd6988c-75be-4b8b-9c90-1c89cc414f3e\") "
Jul 06 22:03:47.613083 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.612963    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1dd6988c-75be-4b8b-9c90-1c89cc414f3e" (UID: "1dd6988c-75be-4b8b-9c90-1c89cc414f3e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:03:47.613083 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.613027    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-model-cache" (OuterVolumeSpecName: "model-cache") pod "1dd6988c-75be-4b8b-9c90-1c89cc414f3e" (UID: "1dd6988c-75be-4b8b-9c90-1c89cc414f3e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:03:47.613209 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.613187    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-model-cache\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:03:47.613327 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.613214    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-tmp-dir\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:03:47.613327 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.613188    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-home" (OuterVolumeSpecName: "home") pod "1dd6988c-75be-4b8b-9c90-1c89cc414f3e" (UID: "1dd6988c-75be-4b8b-9c90-1c89cc414f3e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:03:47.614771 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.614747    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1dd6988c-75be-4b8b-9c90-1c89cc414f3e" (UID: "1dd6988c-75be-4b8b-9c90-1c89cc414f3e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:03:47.615228 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.615209    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-dshm" (OuterVolumeSpecName: "dshm") pod "1dd6988c-75be-4b8b-9c90-1c89cc414f3e" (UID: "1dd6988c-75be-4b8b-9c90-1c89cc414f3e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:03:47.615331 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.615225    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-kube-api-access-gx6mv" (OuterVolumeSpecName: "kube-api-access-gx6mv") pod "1dd6988c-75be-4b8b-9c90-1c89cc414f3e" (UID: "1dd6988c-75be-4b8b-9c90-1c89cc414f3e"). InnerVolumeSpecName "kube-api-access-gx6mv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:03:47.714206 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.714166    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:03:47.714206 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.714195    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-dshm\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:03:47.714206 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.714204    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx6mv\" (UniqueName: \"kubernetes.io/projected/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-kube-api-access-gx6mv\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:03:47.714206 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.714213    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1dd6988c-75be-4b8b-9c90-1c89cc414f3e-home\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:03:47.776422 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.776388    2566 generic.go:356] "Generic (PLEG): container finished" podID="1dd6988c-75be-4b8b-9c90-1c89cc414f3e" containerID="becc816771c6106024377c247ca93c54b53328bff1a5b9e9db852dd743b38d11" exitCode=0
Jul 06 22:03:47.776627 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.776435    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"
Jul 06 22:03:47.776627 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.776440    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2" event={"ID":"1dd6988c-75be-4b8b-9c90-1c89cc414f3e","Type":"ContainerDied","Data":"becc816771c6106024377c247ca93c54b53328bff1a5b9e9db852dd743b38d11"}
Jul 06 22:03:47.776627 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.776504    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2" event={"ID":"1dd6988c-75be-4b8b-9c90-1c89cc414f3e","Type":"ContainerDied","Data":"31e2b416b8e587c435a2b9852ecf26f5d23eccfe31ad5d1e6ed1f16ecf64b520"}
Jul 06 22:03:47.776627 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.776529    2566 scope.go:117] "RemoveContainer" containerID="becc816771c6106024377c247ca93c54b53328bff1a5b9e9db852dd743b38d11"
Jul 06 22:03:47.786319 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.786302    2566 scope.go:117] "RemoveContainer" containerID="becc816771c6106024377c247ca93c54b53328bff1a5b9e9db852dd743b38d11"
Jul 06 22:03:47.786635 ip-10-0-131-165 kubenswrapper[2566]: E0706 22:03:47.786614    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"becc816771c6106024377c247ca93c54b53328bff1a5b9e9db852dd743b38d11\": container with ID starting with becc816771c6106024377c247ca93c54b53328bff1a5b9e9db852dd743b38d11 not found: ID does not exist" containerID="becc816771c6106024377c247ca93c54b53328bff1a5b9e9db852dd743b38d11"
Jul 06 22:03:47.786694 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.786642    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"becc816771c6106024377c247ca93c54b53328bff1a5b9e9db852dd743b38d11"} err="failed to get container status \"becc816771c6106024377c247ca93c54b53328bff1a5b9e9db852dd743b38d11\": rpc error: code = NotFound desc = could not find container \"becc816771c6106024377c247ca93c54b53328bff1a5b9e9db852dd743b38d11\": container with ID starting with becc816771c6106024377c247ca93c54b53328bff1a5b9e9db852dd743b38d11 not found: ID does not exist"
Jul 06 22:03:47.799776 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.799753    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"]
Jul 06 22:03:47.806867 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:47.806841    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-wgdh2"]
Jul 06 22:03:48.002365 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:03:48.002327    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dd6988c-75be-4b8b-9c90-1c89cc414f3e" path="/var/lib/kubelet/pods/1dd6988c-75be-4b8b-9c90-1c89cc414f3e/volumes"
Jul 06 22:04:08.347123 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.347088    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"]
Jul 06 22:04:08.347674 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.347656    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dd6988c-75be-4b8b-9c90-1c89cc414f3e" containerName="main"
Jul 06 22:04:08.347734 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.347676    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dd6988c-75be-4b8b-9c90-1c89cc414f3e" containerName="main"
Jul 06 22:04:08.347770 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.347750    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dd6988c-75be-4b8b-9c90-1c89cc414f3e" containerName="main"
Jul 06 22:04:08.350818 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.350793    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.353476 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.353435    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\""
Jul 06 22:04:08.364039 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.364011    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"]
Jul 06 22:04:08.409384 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.409350    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.409384 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.409402    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.409659 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.409471    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.409659 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.409504    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5hmd\" (UniqueName: \"kubernetes.io/projected/555db9b9-6000-4aad-855b-9ef0b3054a20-kube-api-access-p5hmd\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.409659 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.409526    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.409659 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.409584    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/555db9b9-6000-4aad-855b-9ef0b3054a20-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.510381 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.510343    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/555db9b9-6000-4aad-855b-9ef0b3054a20-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.510617 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.510428    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.510617 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.510487    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.510617 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.510505    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.510617 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.510521    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5hmd\" (UniqueName: \"kubernetes.io/projected/555db9b9-6000-4aad-855b-9ef0b3054a20-kube-api-access-p5hmd\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.510881 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.510630    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.510949 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.510923    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.511011 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.510944    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.511066 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.511008    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.512835 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.512818    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.512945 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.512927    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/555db9b9-6000-4aad-855b-9ef0b3054a20-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.519397 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.519371    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5hmd\" (UniqueName: \"kubernetes.io/projected/555db9b9-6000-4aad-855b-9ef0b3054a20-kube-api-access-p5hmd\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.663715 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.663619    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:08.806806 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.806778    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"]
Jul 06 22:04:08.808912 ip-10-0-131-165 kubenswrapper[2566]: W0706 22:04:08.808885    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod555db9b9_6000_4aad_855b_9ef0b3054a20.slice/crio-6b2324fda7f20d6815156e4c2e399b28f2fadca37b7c3df5b3a96545a856ba92 WatchSource:0}: Error finding container 6b2324fda7f20d6815156e4c2e399b28f2fadca37b7c3df5b3a96545a856ba92: Status 404 returned error can't find the container with id 6b2324fda7f20d6815156e4c2e399b28f2fadca37b7c3df5b3a96545a856ba92
Jul 06 22:04:08.893562 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.893522    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c" event={"ID":"555db9b9-6000-4aad-855b-9ef0b3054a20","Type":"ContainerStarted","Data":"30e9a658f44ca4bd678edd9a69e67c811f2eecaf98f2682f0d604c5ffb74a3f9"}
Jul 06 22:04:08.893707 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.893588    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c" event={"ID":"555db9b9-6000-4aad-855b-9ef0b3054a20","Type":"ContainerStarted","Data":"6b2324fda7f20d6815156e4c2e399b28f2fadca37b7c3df5b3a96545a856ba92"}
Jul 06 22:04:08.912934 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:08.912877    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c" podStartSLOduration=0.912859071 podStartE2EDuration="912.859071ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:04:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:04:08.910907084 +0000 UTC m=+4527.377148338" watchObservedRunningTime="2026-07-06 22:04:08.912859071 +0000 UTC m=+4527.379100699"
Jul 06 22:04:09.664063 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:09.664021    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:18.664860 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:18.664816    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:18.677730 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:18.677701    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:18.962738 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:18.962651    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:04:54.408472 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:04:54.408416    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"]
Jul 06 22:05:09.433309 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.433208    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c" podUID="555db9b9-6000-4aad-855b-9ef0b3054a20" containerName="main" containerID="cri-o://30e9a658f44ca4bd678edd9a69e67c811f2eecaf98f2682f0d604c5ffb74a3f9" gracePeriod=45
Jul 06 22:05:09.691154 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.691092    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:05:09.771585 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.771547    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-home\") pod \"555db9b9-6000-4aad-855b-9ef0b3054a20\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") "
Jul 06 22:05:09.771761 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.771631    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-dshm\") pod \"555db9b9-6000-4aad-855b-9ef0b3054a20\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") "
Jul 06 22:05:09.771761 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.771671    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5hmd\" (UniqueName: \"kubernetes.io/projected/555db9b9-6000-4aad-855b-9ef0b3054a20-kube-api-access-p5hmd\") pod \"555db9b9-6000-4aad-855b-9ef0b3054a20\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") "
Jul 06 22:05:09.771761 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.771692    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-tmp-dir\") pod \"555db9b9-6000-4aad-855b-9ef0b3054a20\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") "
Jul 06 22:05:09.771761 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.771710    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-model-cache\") pod \"555db9b9-6000-4aad-855b-9ef0b3054a20\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") "
Jul 06 22:05:09.771761 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.771731    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/555db9b9-6000-4aad-855b-9ef0b3054a20-tls-certs\") pod \"555db9b9-6000-4aad-855b-9ef0b3054a20\" (UID: \"555db9b9-6000-4aad-855b-9ef0b3054a20\") "
Jul 06 22:05:09.771997 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.771877    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-home" (OuterVolumeSpecName: "home") pod "555db9b9-6000-4aad-855b-9ef0b3054a20" (UID: "555db9b9-6000-4aad-855b-9ef0b3054a20"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:05:09.771997 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.771958    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "555db9b9-6000-4aad-855b-9ef0b3054a20" (UID: "555db9b9-6000-4aad-855b-9ef0b3054a20"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:05:09.772084 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.771990    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-model-cache" (OuterVolumeSpecName: "model-cache") pod "555db9b9-6000-4aad-855b-9ef0b3054a20" (UID: "555db9b9-6000-4aad-855b-9ef0b3054a20"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:05:09.772143 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.772112    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-home\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:05:09.772143 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.772137    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-tmp-dir\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:05:09.772251 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.772148    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-model-cache\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:05:09.773891 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.773867    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-dshm" (OuterVolumeSpecName: "dshm") pod "555db9b9-6000-4aad-855b-9ef0b3054a20" (UID: "555db9b9-6000-4aad-855b-9ef0b3054a20"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:05:09.773999 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.773886    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/555db9b9-6000-4aad-855b-9ef0b3054a20-kube-api-access-p5hmd" (OuterVolumeSpecName: "kube-api-access-p5hmd") pod "555db9b9-6000-4aad-855b-9ef0b3054a20" (UID: "555db9b9-6000-4aad-855b-9ef0b3054a20"). InnerVolumeSpecName "kube-api-access-p5hmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:05:09.773999 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.773920    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/555db9b9-6000-4aad-855b-9ef0b3054a20-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "555db9b9-6000-4aad-855b-9ef0b3054a20" (UID: "555db9b9-6000-4aad-855b-9ef0b3054a20"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:05:09.873141 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.873102    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/555db9b9-6000-4aad-855b-9ef0b3054a20-dshm\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:05:09.873141 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.873133    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5hmd\" (UniqueName: \"kubernetes.io/projected/555db9b9-6000-4aad-855b-9ef0b3054a20-kube-api-access-p5hmd\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:05:09.873141 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:09.873147    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/555db9b9-6000-4aad-855b-9ef0b3054a20-tls-certs\") on node \"ip-10-0-131-165.ec2.internal\" DevicePath \"\""
Jul 06 22:05:10.231177 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:10.231131    2566 generic.go:356] "Generic (PLEG): container finished" podID="555db9b9-6000-4aad-855b-9ef0b3054a20" containerID="30e9a658f44ca4bd678edd9a69e67c811f2eecaf98f2682f0d604c5ffb74a3f9" exitCode=0
Jul 06 22:05:10.231177 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:10.231175    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"
Jul 06 22:05:10.231177 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:10.231179    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c" event={"ID":"555db9b9-6000-4aad-855b-9ef0b3054a20","Type":"ContainerDied","Data":"30e9a658f44ca4bd678edd9a69e67c811f2eecaf98f2682f0d604c5ffb74a3f9"}
Jul 06 22:05:10.231488 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:10.231230    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c" event={"ID":"555db9b9-6000-4aad-855b-9ef0b3054a20","Type":"ContainerDied","Data":"6b2324fda7f20d6815156e4c2e399b28f2fadca37b7c3df5b3a96545a856ba92"}
Jul 06 22:05:10.231488 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:10.231260    2566 scope.go:117] "RemoveContainer" containerID="30e9a658f44ca4bd678edd9a69e67c811f2eecaf98f2682f0d604c5ffb74a3f9"
Jul 06 22:05:10.240556 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:10.240535    2566 scope.go:117] "RemoveContainer" containerID="30e9a658f44ca4bd678edd9a69e67c811f2eecaf98f2682f0d604c5ffb74a3f9"
Jul 06 22:05:10.240859 ip-10-0-131-165 kubenswrapper[2566]: E0706 22:05:10.240837    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30e9a658f44ca4bd678edd9a69e67c811f2eecaf98f2682f0d604c5ffb74a3f9\": container with ID starting with 30e9a658f44ca4bd678edd9a69e67c811f2eecaf98f2682f0d604c5ffb74a3f9 not found: ID does not exist" containerID="30e9a658f44ca4bd678edd9a69e67c811f2eecaf98f2682f0d604c5ffb74a3f9"
Jul 06 22:05:10.240939 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:10.240870    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e9a658f44ca4bd678edd9a69e67c811f2eecaf98f2682f0d604c5ffb74a3f9"} err="failed to get container status \"30e9a658f44ca4bd678edd9a69e67c811f2eecaf98f2682f0d604c5ffb74a3f9\": rpc error: code = NotFound desc = could not find container \"30e9a658f44ca4bd678edd9a69e67c811f2eecaf98f2682f0d604c5ffb74a3f9\": container with ID starting with 30e9a658f44ca4bd678edd9a69e67c811f2eecaf98f2682f0d604c5ffb74a3f9 not found: ID does not exist"
Jul 06 22:05:10.248954 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:10.248922    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"]
Jul 06 22:05:10.251257 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:10.251230    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5ws27c"]
Jul 06 22:05:12.002654 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:12.002621    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="555db9b9-6000-4aad-855b-9ef0b3054a20" path="/var/lib/kubelet/pods/555db9b9-6000-4aad-855b-9ef0b3054a20/volumes"
Jul 06 22:05:32.595690 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:32.595660    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:05:32.622184 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:32.622156    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:05:32.686833 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:32.686803    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:05:32.697777 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:32.697751    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:05:34.880132 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:34.880103    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:05:34.891916 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:34.891890    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:05:34.962972 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:34.962946    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:05:34.975935 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:34.975907    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:05:37.209252 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:37.209221    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:05:37.220978 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:37.220952    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:05:37.285625 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:37.285594    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:05:37.296998 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:37.296973    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:05:39.439725 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:39.439698    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:05:39.450757 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:39.450728    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:05:39.514677 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:39.514650    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:05:39.525838 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:39.525814    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:05:41.685979 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:41.685949    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:05:41.711194 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:41.711166    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:05:41.808734 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:41.808704    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:05:41.825933 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:41.825906    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:05:44.127069 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:44.127031    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:05:44.137717 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:44.137690    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:05:44.209844 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:44.209817    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:05:44.222357 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:44.222322    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:05:46.351681 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:46.351647    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:05:46.362164 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:46.362135    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:05:46.423174 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:46.423142    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:05:46.435011 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:46.434985    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:05:48.581075 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:48.581044    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:05:48.591864 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:48.591836    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:05:48.655380 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:48.655350    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:05:48.666256 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:48.666227    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:05:50.801667 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:50.801636    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:05:50.813361 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:50.813330    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:05:50.880115 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:50.880088    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:05:50.894426 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:50.894399    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:05:53.077435 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:53.077399    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:05:53.088208 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:53.088178    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:05:53.149256 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:53.149227    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:05:53.159724 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:53.159692    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:05:55.309541 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:55.309510    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:05:55.322113 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:55.322090    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:05:55.388926 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:55.388897    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:05:55.400153 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:55.400123    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:05:57.560974 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:57.560941    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:05:57.572224 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:57.572198    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:05:57.632196 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:57.632163    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:05:57.642488 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:57.642449    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:05:59.769174 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:59.769144    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:05:59.780146 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:59.780122    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:05:59.845825 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:59.845793    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:05:59.856280 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:05:59.856242    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:06:02.027891 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:02.027860    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:06:02.038514 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:02.038487    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:06:02.103210 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:02.103182    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:06:02.115350 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:02.115326    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:06:04.282560 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:04.282530    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:06:04.295849 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:04.295819    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:06:04.369906 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:04.369880    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:06:04.382095 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:04.382070    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:06:06.545222 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:06.545194    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/main/0.log"
Jul 06 22:06:06.556110 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:06.556082    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1_796ba5a7-c7bc-4027-bc2c-0050ed9f9d65/storage-initializer/0.log"
Jul 06 22:06:06.622147 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:06.622115    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/main/0.log"
Jul 06 22:06:06.634561 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:06.634530    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schecvlkm_2c2c3c0d-74e2-4c8a-b483-533601eca911/storage-initializer/0.log"
Jul 06 22:06:08.197909 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:08.197879    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x2st2_c5fe011c-ee58-46d2-ab73-1ca4d34aff1c/discovery/0.log"
Jul 06 22:06:08.248544 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:08.248514    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5655df4f5-g8dcn_d1e1eb8b-dd63-41a9-9842-be5520a5cb73/router/0.log"
Jul 06 22:06:09.289978 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:09.289948    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x2st2_c5fe011c-ee58-46d2-ab73-1ca4d34aff1c/discovery/0.log"
Jul 06 22:06:09.342166 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:09.342138    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5655df4f5-g8dcn_d1e1eb8b-dd63-41a9-9842-be5520a5cb73/router/0.log"
Jul 06 22:06:10.383756 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:10.383725    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x2st2_c5fe011c-ee58-46d2-ab73-1ca4d34aff1c/discovery/0.log"
Jul 06 22:06:10.438605 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:10.438554    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5655df4f5-g8dcn_d1e1eb8b-dd63-41a9-9842-be5520a5cb73/router/0.log"
Jul 06 22:06:11.481670 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:11.481632    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x2st2_c5fe011c-ee58-46d2-ab73-1ca4d34aff1c/discovery/0.log"
Jul 06 22:06:11.532663 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:11.532625    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5655df4f5-g8dcn_d1e1eb8b-dd63-41a9-9842-be5520a5cb73/router/0.log"
Jul 06 22:06:12.507066 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:12.507037    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:12.522283 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:12.522258    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:12.537372 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:12.537347    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:12.633234 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:12.633201    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:13.707927 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:13.707902    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:13.722025 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:13.722003    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:13.742547 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:13.742526    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:13.848773 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:13.848744    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:14.919851 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:14.919823    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:14.934714 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:14.934688    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:14.950958 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:14.950936    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:15.044676 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:15.044647    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:16.108527 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:16.108496    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:16.124515 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:16.124489    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:16.139771 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:16.139747    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:16.240162 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:16.240131    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:17.293962 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:17.293934    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:17.310148 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:17.310125    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:17.323860 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:17.323839    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:17.421821 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:17.421789    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:18.484224 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:18.484191    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:18.499669 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:18.499643    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:18.515344 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:18.515321    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:18.612747 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:18.612715    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:19.659001 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:19.658972    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:19.674810 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:19.674784    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:19.690368 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:19.690345    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:19.793249 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:19.793215    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:20.848691 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:20.848663    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:20.862884 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:20.862863    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:20.877183 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:20.877157    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:20.978390 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:20.978358    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:22.014836 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:22.014808    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:22.027642 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:22.027616    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:22.041333 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:22.041308    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:22.136937 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:22.136903    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:23.168718 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:23.168686    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:23.185352 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:23.185315    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:23.198402 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:23.198381    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:23.291552 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:23.291520    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:24.336102 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:24.336071    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:24.351498 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:24.351400    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:24.365666 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:24.365636    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:24.464939 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:24.464908    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:25.481035 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:25.481007    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:25.495271 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:25.495249    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:25.508380 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:25.508360    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:25.607205 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:25.607172    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:26.683683 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:26.683655    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:26.708114 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:26.708091    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:26.731673 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:26.731649    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:26.833063 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:26.833032    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:27.969102 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:27.969074    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:27.988354 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:27.988326    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:28.005320 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:28.005295    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:28.112023 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:28.111984    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:29.198497 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:29.198472    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:29.213992 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:29.213968    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:29.228851 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:29.228829    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:29.325284 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:29.325252    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:30.359729 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:30.359686    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:30.373783 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:30.373759    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:30.386778 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:30.386755    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:30.481440 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:30.481408    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:31.522230 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:31.522197    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:31.536498 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:31.536450    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:31.550096 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:31.550064    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:31.648408 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:31.648373    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:32.671685 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:32.671649    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:32.686047 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:32.686020    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:32.701859 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:32.701832    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:32.798915 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:32.798880    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:33.867006 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:33.866975    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:33.883296 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:33.883270    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:33.897880 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:33.897854    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:34.004587 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:34.004526    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:35.056846 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:35.056816    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:35.070774 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:35.070751    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:35.085420 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:35.085391    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:35.183252 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:35.183218    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:36.205827 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:36.205795    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:36.221238 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:36.221213    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:36.236133 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:36.236106    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:36.333614 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:36.333583    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:38.941721 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:38.941682    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-95ll7/must-gather-88v5n"]
Jul 06 22:06:38.942109 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:38.942063    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="555db9b9-6000-4aad-855b-9ef0b3054a20" containerName="main"
Jul 06 22:06:38.942109 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:38.942074    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="555db9b9-6000-4aad-855b-9ef0b3054a20" containerName="main"
Jul 06 22:06:38.942183 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:38.942144    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="555db9b9-6000-4aad-855b-9ef0b3054a20" containerName="main"
Jul 06 22:06:38.945479 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:38.945432    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-95ll7/must-gather-88v5n"
Jul 06 22:06:38.948186 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:38.948164    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-95ll7\"/\"kube-root-ca.crt\""
Jul 06 22:06:38.948523 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:38.948489    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-95ll7\"/\"openshift-service-ca.crt\""
Jul 06 22:06:38.949381 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:38.949345    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-95ll7\"/\"default-dockercfg-brp8c\""
Jul 06 22:06:38.958069 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:38.958047    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-95ll7/must-gather-88v5n"]
Jul 06 22:06:39.112988 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:39.112952    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llkbh\" (UniqueName: \"kubernetes.io/projected/7a353c08-7be6-4846-be40-618d1b15b7b3-kube-api-access-llkbh\") pod \"must-gather-88v5n\" (UID: \"7a353c08-7be6-4846-be40-618d1b15b7b3\") " pod="openshift-must-gather-95ll7/must-gather-88v5n"
Jul 06 22:06:39.113171 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:39.113036    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7a353c08-7be6-4846-be40-618d1b15b7b3-must-gather-output\") pod \"must-gather-88v5n\" (UID: \"7a353c08-7be6-4846-be40-618d1b15b7b3\") " pod="openshift-must-gather-95ll7/must-gather-88v5n"
Jul 06 22:06:39.214234 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:39.214153    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7a353c08-7be6-4846-be40-618d1b15b7b3-must-gather-output\") pod \"must-gather-88v5n\" (UID: \"7a353c08-7be6-4846-be40-618d1b15b7b3\") " pod="openshift-must-gather-95ll7/must-gather-88v5n"
Jul 06 22:06:39.214234 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:39.214203    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llkbh\" (UniqueName: \"kubernetes.io/projected/7a353c08-7be6-4846-be40-618d1b15b7b3-kube-api-access-llkbh\") pod \"must-gather-88v5n\" (UID: \"7a353c08-7be6-4846-be40-618d1b15b7b3\") " pod="openshift-must-gather-95ll7/must-gather-88v5n"
Jul 06 22:06:39.214587 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:39.214564    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7a353c08-7be6-4846-be40-618d1b15b7b3-must-gather-output\") pod \"must-gather-88v5n\" (UID: \"7a353c08-7be6-4846-be40-618d1b15b7b3\") " pod="openshift-must-gather-95ll7/must-gather-88v5n"
Jul 06 22:06:39.239518 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:39.239489    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llkbh\" (UniqueName: \"kubernetes.io/projected/7a353c08-7be6-4846-be40-618d1b15b7b3-kube-api-access-llkbh\") pod \"must-gather-88v5n\" (UID: \"7a353c08-7be6-4846-be40-618d1b15b7b3\") " pod="openshift-must-gather-95ll7/must-gather-88v5n"
Jul 06 22:06:39.255257 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:39.255222    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-95ll7/must-gather-88v5n"
Jul 06 22:06:39.415791 ip-10-0-131-165 kubenswrapper[2566]: W0706 22:06:39.415762    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a353c08_7be6_4846_be40_618d1b15b7b3.slice/crio-f6f22fb8c34a8cf597e92120b6f585d54ec6d1e96eaf1b8758c5b3f701a73d7d WatchSource:0}: Error finding container f6f22fb8c34a8cf597e92120b6f585d54ec6d1e96eaf1b8758c5b3f701a73d7d: Status 404 returned error can't find the container with id f6f22fb8c34a8cf597e92120b6f585d54ec6d1e96eaf1b8758c5b3f701a73d7d
Jul 06 22:06:39.415911 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:39.415883    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-95ll7/must-gather-88v5n"]
Jul 06 22:06:39.417701 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:39.417680    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 22:06:39.726559 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:39.726527    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-95ll7/must-gather-88v5n" event={"ID":"7a353c08-7be6-4846-be40-618d1b15b7b3","Type":"ContainerStarted","Data":"f6f22fb8c34a8cf597e92120b6f585d54ec6d1e96eaf1b8758c5b3f701a73d7d"}
Jul 06 22:06:40.738682 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:40.737847    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-95ll7/must-gather-88v5n" event={"ID":"7a353c08-7be6-4846-be40-618d1b15b7b3","Type":"ContainerStarted","Data":"2fc666acc5014a692ce54baaf459bb7b246edde4abd9d7e0352d40b20e1838ad"}
Jul 06 22:06:40.738682 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:40.737900    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-95ll7/must-gather-88v5n" event={"ID":"7a353c08-7be6-4846-be40-618d1b15b7b3","Type":"ContainerStarted","Data":"7f8e01f67e419e458e8b0d80d06630198cc273ac6f7048e61cfe8d4f26d41612"}
Jul 06 22:06:40.758545 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:40.758483    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-95ll7/must-gather-88v5n" podStartSLOduration=1.7989549390000001 podStartE2EDuration="2.758444812s" totalImagesPullingTime="959.489873ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:06:40.75455579 +0000 UTC m=+4679.220797045" watchObservedRunningTime="2026-07-06 22:06:40.758444812 +0000 UTC m=+4679.224686067"
Jul 06 22:06:42.020338 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:42.020289    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7lk82_04009265-aa63-49ba-a715-0a555264ccd7/global-pull-secret-syncer/0.log"
Jul 06 22:06:42.190494 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:42.190443    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mrll5_5002d46b-a551-424b-bd08-f61d84e3c45b/konnectivity-agent/0.log"
Jul 06 22:06:42.215590 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:42.215564    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-165.ec2.internal_e45831538ee9d5022543246cbe3ad035/haproxy/0.log"
Jul 06 22:06:47.394630 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:47.394592    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-kv2wg_06e719d0-5657-4579-b88e-7a9c83065efc/manager/0.log"
Jul 06 22:06:47.428043 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:47.428013    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-l2xrg_b5b1cf4b-6117-4fcd-b3f7-97ece99b88fb/manager/0.log"
Jul 06 22:06:47.463070 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:47.462877    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-fq2x9_699c1536-62a3-4b51-a035-d30962929ec8/kuadrant-console-plugin/0.log"
Jul 06 22:06:47.619480 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:47.618483    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-h6vpf_5221abb2-42b1-4dbe-abeb-0c1ed9d57d76/manager/0.log"
Jul 06 22:06:48.880704 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:48.880677    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2dea7a7a-5804-44e6-be5e-0855f2a5f3a8/alertmanager/0.log"
Jul 06 22:06:48.904945 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:48.904914    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2dea7a7a-5804-44e6-be5e-0855f2a5f3a8/config-reloader/0.log"
Jul 06 22:06:48.928812 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:48.928784    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2dea7a7a-5804-44e6-be5e-0855f2a5f3a8/kube-rbac-proxy-web/0.log"
Jul 06 22:06:48.952921 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:48.952865    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2dea7a7a-5804-44e6-be5e-0855f2a5f3a8/kube-rbac-proxy/0.log"
Jul 06 22:06:48.989493 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:48.989439    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2dea7a7a-5804-44e6-be5e-0855f2a5f3a8/kube-rbac-proxy-metric/0.log"
Jul 06 22:06:49.018122 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:49.018093    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2dea7a7a-5804-44e6-be5e-0855f2a5f3a8/prom-label-proxy/0.log"
Jul 06 22:06:49.053189 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:49.053159    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2dea7a7a-5804-44e6-be5e-0855f2a5f3a8/init-config-reloader/0.log"
Jul 06 22:06:49.134387 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:49.134301    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5xxtk_2bc829e1-259f-4427-8499-59dab0aa7a4b/kube-state-metrics/0.log"
Jul 06 22:06:49.183636 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:49.183572    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5xxtk_2bc829e1-259f-4427-8499-59dab0aa7a4b/kube-rbac-proxy-main/0.log"
Jul 06 22:06:49.204632 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:49.204602    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5xxtk_2bc829e1-259f-4427-8499-59dab0aa7a4b/kube-rbac-proxy-self/0.log"
Jul 06 22:06:49.553598 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:49.553568    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wsr8m_12ddb476-403e-4335-8030-935061abc21a/node-exporter/0.log"
Jul 06 22:06:49.601314 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:49.601279    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wsr8m_12ddb476-403e-4335-8030-935061abc21a/kube-rbac-proxy/0.log"
Jul 06 22:06:49.637735 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:49.637702    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wsr8m_12ddb476-403e-4335-8030-935061abc21a/init-textfile/0.log"
Jul 06 22:06:50.016324 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:50.016290    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6bcbc65985-8ww22_32b2f426-a65f-4ab1-b617-b2a3e50afe26/telemeter-client/0.log"
Jul 06 22:06:50.042115 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:50.042089    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6bcbc65985-8ww22_32b2f426-a65f-4ab1-b617-b2a3e50afe26/reload/0.log"
Jul 06 22:06:50.064748 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:50.064706    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6bcbc65985-8ww22_32b2f426-a65f-4ab1-b617-b2a3e50afe26/kube-rbac-proxy/0.log"
Jul 06 22:06:52.173206 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.173170    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"]
Jul 06 22:06:52.176824 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.176802    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.188489 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.188439    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"]
Jul 06 22:06:52.253597 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.253560    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-podres\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.253884 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.253863    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-lib-modules\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.254078 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.254060    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-sys\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.254242 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.254221    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-proc\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.254387 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.254369    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp6b2\" (UniqueName: \"kubernetes.io/projected/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-kube-api-access-fp6b2\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.355271 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.355233    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-podres\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.355478 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.355337    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-lib-modules\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.355478 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.355396    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-sys\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.355478 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.355425    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-podres\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.355636 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.355492    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-sys\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.355636 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.355507    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-proc\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.355636 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.355492    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-lib-modules\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.355636 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.355554    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-proc\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.355636 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.355616    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp6b2\" (UniqueName: \"kubernetes.io/projected/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-kube-api-access-fp6b2\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.364836 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.364804    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp6b2\" (UniqueName: \"kubernetes.io/projected/2e4e460c-6191-4c6e-bd7e-ce8c938e4e83-kube-api-access-fp6b2\") pod \"perf-node-gather-daemonset-gbq7h\" (UID: \"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83\") " pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.453825 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.453745    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84f7f87c5-64vbr_c388ef5c-704f-4ce1-b290-62f7b1da4669/console/0.log"
Jul 06 22:06:52.492082 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.492048    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.672282 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.672253    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"]
Jul 06 22:06:52.673690 ip-10-0-131-165 kubenswrapper[2566]: W0706 22:06:52.673658    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2e4e460c_6191_4c6e_bd7e_ce8c938e4e83.slice/crio-24cc1ba833a5ef50c09fe314ee804df14af1960707d82ed3b3ce9880eb982642 WatchSource:0}: Error finding container 24cc1ba833a5ef50c09fe314ee804df14af1960707d82ed3b3ce9880eb982642: Status 404 returned error can't find the container with id 24cc1ba833a5ef50c09fe314ee804df14af1960707d82ed3b3ce9880eb982642
Jul 06 22:06:52.854870 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.854760    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h" event={"ID":"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83","Type":"ContainerStarted","Data":"a779560809f2eb08fba6c3c9cfa8c7a47205b6a415064925667325783a9871bb"}
Jul 06 22:06:52.854870 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.854824    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h" event={"ID":"2e4e460c-6191-4c6e-bd7e-ce8c938e4e83","Type":"ContainerStarted","Data":"24cc1ba833a5ef50c09fe314ee804df14af1960707d82ed3b3ce9880eb982642"}
Jul 06 22:06:52.855135 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.854975    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:06:52.872669 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:52.872422    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h" podStartSLOduration=0.872401917 podStartE2EDuration="872.401917ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:06:52.87194782 +0000 UTC m=+4691.338189074" watchObservedRunningTime="2026-07-06 22:06:52.872401917 +0000 UTC m=+4691.338643172"
Jul 06 22:06:54.127447 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:54.127411    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6mllp_4133d58d-63a9-46cd-b3d5-eddf0406fe31/dns/0.log"
Jul 06 22:06:54.152127 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:54.152094    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6mllp_4133d58d-63a9-46cd-b3d5-eddf0406fe31/kube-rbac-proxy/0.log"
Jul 06 22:06:54.302768 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:54.302731    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ptn9v_1236b937-e2b7-41ac-a612-142a60a86806/dns-node-resolver/0.log"
Jul 06 22:06:54.972985 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:54.972956    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qxhpq_8f813d5c-13a4-485c-b5ab-5f26fd9f84fe/node-ca/0.log"
Jul 06 22:06:56.170371 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:56.170332    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x2st2_c5fe011c-ee58-46d2-ab73-1ca4d34aff1c/discovery/0.log"
Jul 06 22:06:56.242230 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:56.242189    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5655df4f5-g8dcn_d1e1eb8b-dd63-41a9-9842-be5520a5cb73/router/0.log"
Jul 06 22:06:56.833862 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:56.833817    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fswwk_a36da1e0-2692-4633-b274-158c32c67c8a/serve-healthcheck-canary/0.log"
Jul 06 22:06:57.424081 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:57.424054    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jnd7m_aba8779f-d992-4819-bbac-b519fe6e7e0d/kube-rbac-proxy/0.log"
Jul 06 22:06:57.446953 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:57.446922    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jnd7m_aba8779f-d992-4819-bbac-b519fe6e7e0d/exporter/0.log"
Jul 06 22:06:57.470168 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:57.470139    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jnd7m_aba8779f-d992-4819-bbac-b519fe6e7e0d/extractor/0.log"
Jul 06 22:06:58.872244 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:06:58.872208    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-95ll7/perf-node-gather-daemonset-gbq7h"
Jul 06 22:07:01.054329 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:01.054299    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-664cc55bd7-rblpn_0cf9ea48-0611-4266-acfb-65a78d0d63d0/manager/0.log"
Jul 06 22:07:01.472993 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:01.472892    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-5bcc55b7f6-2glkl_ff0c4ffd-7f29-4a03-9fd0-d0f204d09a26/manager/0.log"
Jul 06 22:07:01.497241 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:01.497202    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-kwjqs_f6297f73-cf52-48ae-aeb7-959e9a06bf33/server/0.log"
Jul 06 22:07:02.014152 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:02.014120    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-7vtnv_9d31ef97-bdf0-4662-88e0-1e00f66ba388/manager/0.log"
Jul 06 22:07:02.052901 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:02.052818    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-x9rxz_6d457925-4653-4133-a036-b07ee89d4cd8/s3-init/0.log"
Jul 06 22:07:02.074904 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:02.074882    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-x9rxz_6d457925-4653-4133-a036-b07ee89d4cd8/download-hf-model/0.log"
Jul 06 22:07:03.990143 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:03.990111    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x2st2_c5fe011c-ee58-46d2-ab73-1ca4d34aff1c/discovery/0.log"
Jul 06 22:07:04.057353 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:04.057326    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5655df4f5-g8dcn_d1e1eb8b-dd63-41a9-9842-be5520a5cb73/router/0.log"
Jul 06 22:07:05.294139 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:05.292819    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x2st2_c5fe011c-ee58-46d2-ab73-1ca4d34aff1c/discovery/0.log"
Jul 06 22:07:05.364911 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:05.364884    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5655df4f5-g8dcn_d1e1eb8b-dd63-41a9-9842-be5520a5cb73/router/0.log"
Jul 06 22:07:07.366669 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:07.366638    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-pbp6v_b6935f66-ae75-4b3a-8b05-904a42a0dacb/migrator/0.log"
Jul 06 22:07:07.393234 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:07.393209    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-pbp6v_b6935f66-ae75-4b3a-8b05-904a42a0dacb/graceful-termination/0.log"
Jul 06 22:07:09.041171 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:09.041143    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hjc94_089e53a7-3267-465e-8abb-52b283f3d756/kube-multus-additional-cni-plugins/0.log"
Jul 06 22:07:09.065504 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:09.065473    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hjc94_089e53a7-3267-465e-8abb-52b283f3d756/egress-router-binary-copy/0.log"
Jul 06 22:07:09.087564 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:09.087535    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hjc94_089e53a7-3267-465e-8abb-52b283f3d756/cni-plugins/0.log"
Jul 06 22:07:09.109041 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:09.109013    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hjc94_089e53a7-3267-465e-8abb-52b283f3d756/bond-cni-plugin/0.log"
Jul 06 22:07:09.131099 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:09.131076    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hjc94_089e53a7-3267-465e-8abb-52b283f3d756/routeoverride-cni/0.log"
Jul 06 22:07:09.153184 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:09.153154    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hjc94_089e53a7-3267-465e-8abb-52b283f3d756/whereabouts-cni-bincopy/0.log"
Jul 06 22:07:09.175857 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:09.175827    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hjc94_089e53a7-3267-465e-8abb-52b283f3d756/whereabouts-cni/0.log"
Jul 06 22:07:09.420762 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:09.420681    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xh69c_e267afa3-e061-4f08-bc2c-4c90e6b26753/kube-multus/0.log"
Jul 06 22:07:09.543674 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:09.543647    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w4sfv_997ba244-1db8-4c94-bb4e-eda91480d511/network-metrics-daemon/0.log"
Jul 06 22:07:09.565907 ip-10-0-131-165 kubenswrapper[2566]: I0706 22:07:09.565880    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w4sfv_997ba244-1db8-4c94-bb4e-eda91480d511/kube-rbac-proxy/0.log"