Jul 09 09:33:16.891065 ip-10-0-132-105 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.218500    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222458    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222469    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222472    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222475    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222478    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222481    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222484    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 09:33:17.226614 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222487    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222489    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222492    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222494    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222498    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222500    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222503    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222505    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222508    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222511    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222513    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222516    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222520    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222522    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222525    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222528    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222530    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222533    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222535    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222538    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 09:33:17.241623 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222541    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222543    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222547    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222550    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222552    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222555    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222558    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222561    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222563    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222566    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222568    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222571    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222574    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222577    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222579    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222582    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222584    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222587    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222590    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222592    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 09:33:17.243399 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222595    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222597    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222600    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222602    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222605    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222607    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222610    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222613    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222615    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222617    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222620    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222623    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222625    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222628    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222631    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222633    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222636    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222639    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222642    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 09:33:17.257482 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222645    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222649    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222653    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222656    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222658    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222661    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222664    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222666    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222669    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222671    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222673    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222679    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222682    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222685    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222688    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222690    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222693    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222697    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222702    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 09:33:17.259263 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222705    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222708    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222712    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222714    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222717    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222720    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222722    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222725    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222727    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222730    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222734    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222737    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222740    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.222743    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223466    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223475    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223478    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223481    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223485    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 09:33:17.261631 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223489    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223493    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223496    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223499    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223501    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223504    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223507    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223510    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223513    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223515    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223518    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223521    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223523    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223526    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223528    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223531    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223534    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223536    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223539    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223541    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 09:33:17.339634 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223544    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 09:33:17.280866 ip-10-0-132-105 systemd[1]: Started Kubernetes Kubelet.
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223546    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223549    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223551    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223553    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223556    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223558    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223561    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223563    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223566    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223568    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223571    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223574    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223577    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223579    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223582    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223585    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223587    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223590    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223593    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 09:33:17.487158 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223595    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223598    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223601    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223603    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223608    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223611    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223614    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223617    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223620    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223623    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223626    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223629    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223632    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223635    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223637    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223640    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223643    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223645    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223650    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 09:33:17.488668 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223652    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223655    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223658    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223660    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223663    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223666    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223668    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223671    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223673    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223676    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223679    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223682    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223685    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223688    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223691    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223694    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223697    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223701    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223704    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223707    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 09:33:17.489330 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223710    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223712    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223715    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223718    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223720    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223723    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223725    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223728    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223730    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223732    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223735    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223739    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223742    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223744    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.223747    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224124    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224130    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224132    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224135    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224138    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 09:33:17.489932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224141    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224143    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224146    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224148    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224151    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224154    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224157    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224159    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224162    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224164    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224167    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224170    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224173    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224176    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224178    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224181    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224183    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224188    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224190    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224192    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 09:33:17.490528 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224195    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224198    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224200    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224204    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224206    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224209    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224212    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224215    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224217    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224220    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224222    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224224    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224227    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224229    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224232    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224234    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224236    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224239    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224242    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224246    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 09:33:17.491124 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224250    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224252    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224255    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224258    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224261    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224264    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224266    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224269    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224272    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224276    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224279    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224281    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224284    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224286    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224289    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224293    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224296    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224298    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224301    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224303    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 09:33:17.491719 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224306    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224308    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224311    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224315    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224318    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224321    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224324    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224327    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224330    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224333    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224335    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224338    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224340    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224343    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224346    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224349    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224352    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224354    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 09:33:17.492340 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224356    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224359    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224362    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224365    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224368    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224370    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224373    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224375    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224378    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224381    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224384    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224386    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224389    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224392    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224395    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.224397    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224463    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224469    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224476    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224480    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224484    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 09:33:17.492926 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224487    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224491    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224499    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224503    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224506    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224510    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224513    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224515    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224518    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224521    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224524    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224526    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224530    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224533    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224537    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224541    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224544    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224547    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224550    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224553    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224557    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224560    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224563    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224566    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 09:33:17.493541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224569    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224572    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224575    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224579    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224582    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224585    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224587    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224590    2577 flags.go:64] FLAG: --enable-server="true"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224593    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224600    2577 flags.go:64] FLAG: --event-burst="100"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224603    2577 flags.go:64] FLAG: --event-qps="50"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224605    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224608    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224611    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224615    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224618    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224621    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224624    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224626    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224629    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224632    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224635    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224639    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224642    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224645    2577 flags.go:64] FLAG: --feature-gates=""
Jul 09 09:33:17.494279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224649    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224651    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224654    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224659    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224662    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224665    2577 flags.go:64] FLAG: --help="false"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224668    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224671    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224674    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224677    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224680    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224684    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224687    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224689    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224692    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224695    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224698    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224701    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224703    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224706    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224709    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224712    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224715    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224717    2577 flags.go:64] FLAG: --lock-file=""
Jul 09 09:33:17.495054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224720    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224723    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224726    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224731    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224734    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224738    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224740    2577 flags.go:64] FLAG: --logging-format="text"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224743    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224746    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224749    2577 flags.go:64] FLAG: --manifest-url=""
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224752    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224757    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224760    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224764    2577 flags.go:64] FLAG: --max-pods="110"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224767    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224770    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224773    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224776    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224778    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224781    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224784    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224790    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224807    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224810    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 09:33:17.495696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224813    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224816    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224821    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224824    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224828    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224830    2577 flags.go:64] FLAG: --port="10250"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224833    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224836    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b9e314ec598d5c46"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224840    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224843    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224846    2577 flags.go:64] FLAG: --register-node="true"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224849    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224852    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224855    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224859    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224862    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224865    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224868    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224871    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224875    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224878    2577 flags.go:64] FLAG: --runonce="false"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224881    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224884    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224886    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224889    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 09:33:17.501473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224892    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224895    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224898    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224901    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224904    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224907    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224909    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224912    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224915    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224918    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224921    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224926    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224929    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224932    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224935    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224938    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224941    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224944    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224947    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224949    2577 flags.go:64] FLAG: --v="2"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224953    2577 flags.go:64] FLAG: --version="false"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224958    2577 flags.go:64] FLAG: --vmodule=""
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224961    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.224965    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 09:33:17.502178 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225070    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225074    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225079    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225082    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225085    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225088    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225091    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225093    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225096    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225098    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225101    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225104    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225107    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225109    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225111    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225114    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225116    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225119    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225122    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225124    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225127    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 09:33:17.502787 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225129    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225132    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225135    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225137    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225140    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225143    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225147    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225151    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225156    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225159    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225161    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225164    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225167    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225170    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225173    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225175    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225177    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225180    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225183    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 09:33:17.503445 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225186    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225188    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225191    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225193    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225196    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225198    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225201    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225203    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225206    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225208    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225211    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225214    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225216    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225219    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225221    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225224    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225226    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225229    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225232    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225234    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 09:33:17.504788 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225237    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225241    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225243    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225247    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225251    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225254    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225259    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225261    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225264    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225267    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225269    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225272    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225274    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225277    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225280    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225282    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225285    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225287    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225290    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 09:33:17.506278 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225292    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225295    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225297    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225300    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225302    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225305    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225307    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225310    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225312    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225315    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225317    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225320    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225323    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225325    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225328    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225331    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225334    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225336    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225339    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 09:33:17.506821 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.225343    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.225348    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.232335    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.232350    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.232369    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.232375    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232427    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232432    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232436    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232439    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232442    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232445    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232448    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232451    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 09:33:17.617116 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232454    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232456    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232459    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232467    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232469    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232472    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232475    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232477    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232480    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232483    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232486    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232488    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232492    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232495    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232498    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232502    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232505    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232508    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232511    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 09:33:17.642386 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232514    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232516    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232519    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232522    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232524    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232528    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232530    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232533    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232536    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232539    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232541    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232544    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232547    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232550    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232553    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232555    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232560    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232562    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232565    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232567    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 09:33:17.642999 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232570    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232573    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232575    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232578    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232580    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232583    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232585    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232588    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232591    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232593    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232596    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232600    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232604    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232607    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232609    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232612    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232614    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232617    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232619    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232622    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 09:33:17.644852 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232625    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232627    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232630    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232632    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232635    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232637    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232640    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232643    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232647    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232651    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232655    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232658    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232661    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232664    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232667    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232670    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232673    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232676    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232678    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 09:33:17.645439 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232681    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232683    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232686    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232688    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232691    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232694    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232696    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232699    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232701    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232704    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232706    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232709    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232711    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.232716    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232832    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 09:33:17.645980 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232838    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232841    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232844    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232846    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232849    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232852    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232854    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232859    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232861    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232865    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232870    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232877    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232880    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232883    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232886    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232888    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232891    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232895    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232898    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 09:33:17.646464 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232900    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232903    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232905    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232908    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232911    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232913    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232916    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232919    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232921    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232924    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232926    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232929    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232932    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232934    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232937    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232940    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232942    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232945    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232947    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232950    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 09:33:17.647238 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232955    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232957    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232960    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232962    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232965    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232968    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232970    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232973    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232976    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232978    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232982    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232985    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232987    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232990    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232992    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232995    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.232997    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233000    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233002    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233005    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 09:33:17.647865 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233007    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233010    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233013    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233015    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233017    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233020    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233022    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233025    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233028    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233030    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233033    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233035    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233039    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233041    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233044    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233046    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233049    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233052    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233055    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 09:33:17.793448 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233058    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233062    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233065    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233070    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233072    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233075    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233077    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233081    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233084    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233087    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233090    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233093    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233095    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233098    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233100    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233103    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233106    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233108    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233111    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 09:33:17.794610 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:17.233114    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 09:33:17.795224 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.233118    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 09:33:17.795224 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.233213    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 09:33:17.795224 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.235254    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 09:33:17.795224 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.236442    2577 server.go:1019] "Starting client certificate rotation"
Jul 09 09:33:17.795224 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.236544    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 09:33:17.795224 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.236573    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 09:33:17.795224 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.238771    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 09:33:17.795224 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.240078    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 09:33:17.795224 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.248425    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 09 09:33:17.795224 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.250417    2577 log.go:25] "Validated CRI v1 image API"
Jul 09 09:33:17.795224 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.253299    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 09:33:17.795224 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.255449    2577 fs.go:135] Filesystem UUIDs: map[0d7eeedc-0ba4-4f5d-9ec8-780a189819aa:/dev/nvme0n1p3 1565e352-e543-46a9-8957-0442b080837e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 09 09:33:17.795634 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.255471    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 09:33:17.796051 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.260953    2577 manager.go:216] Machine: {Timestamp:2026-07-09 09:33:17.259815706 +0000 UTC m=+0.265984773 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101069 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2978aaa9e5ce09c96a49e527303c25 SystemUUID:ec2978aa-a9e5-ce09-c96a-49e527303c25 BootID:55f6b762-31a2-40cd-98bf-c8d99b640bb1 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c0:14:35:bb:15 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c0:14:35:bb:15 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:54:86:2f:72:42 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 09:33:17.796051 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.261047    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 09:33:17.796051 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.261112    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 09:33:17.796051 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.263685    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 09:33:17.796051 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.263709    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-105.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 09:33:17.796051 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.264096    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 09:33:17.796051 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.264904    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.264914    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.264921    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.264941    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.266253    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.270089    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.270185    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.270211    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.270220    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.270233    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.270242    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.270250    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.270557    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.270573    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.273475    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.275448    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.275474    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.276507    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.276520    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.276526    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.276531    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.276543    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.276549    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.276557    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.276564    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.276570    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.276575    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.276584    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.276590    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.277206    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.277212    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.280048    2577 server.go:1268] "Started kubelet"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.280146    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.280765    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.280870    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.280886    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.281175    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-105.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.281233    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.281948    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 09:33:17.796461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.282398    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.285053    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.288095    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.288216    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-105.ec2.internal\" not found"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.288732    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.288746    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.288832    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.288904    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.288912    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.289096    2577 factory.go:55] Registering systemd factory
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.289107    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.288978    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.287769    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-105.ec2.internal.18c09549f2f53691  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-105.ec2.internal,UID:ip-10-0-132-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-105.ec2.internal,},FirstTimestamp:2026-07-09 09:33:17.280024209 +0000 UTC m=+0.286193272,LastTimestamp:2026-07-09 09:33:17.280024209 +0000 UTC m=+0.286193272,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-105.ec2.internal,}"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.289317    2577 factory.go:153] Registering CRI-O factory
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.289331    2577 factory.go:223] Registration of the crio container factory successfully
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.289412    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.289434    2577 factory.go:103] Registering Raw factory
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.289468    2577 manager.go:1196] Started watching for new ooms in manager
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.290585    2577 manager.go:349] Starting recovery of all containers
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.292690    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.292693    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.303554    2577 manager.go:354] Recovery completed
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.309604    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.311131    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.311158    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.311168    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.311586    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.311588    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.311613    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.311627    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.312221    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.312244    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:33:17.797495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.312254    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:33:17.798413 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.312993    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-105.ec2.internal.18c09549f4d00e40  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-105.ec2.internal,UID:ip-10-0-132-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-105.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-105.ec2.internal,},FirstTimestamp:2026-07-09 09:33:17.311143488 +0000 UTC m=+0.317312550,LastTimestamp:2026-07-09 09:33:17.311143488 +0000 UTC m=+0.317312550,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-105.ec2.internal,}"
Jul 09 09:33:17.798413 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.313832    2577 policy_none.go:49] "None policy: Start"
Jul 09 09:33:17.798413 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.313849    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 09:33:17.798413 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.313863    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 09:33:17.798413 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.315950    2577 policy_none.go:47] "Start"
Jul 09 09:33:17.798413 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.318113    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7lk76"
Jul 09 09:33:17.798413 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.326703    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7lk76"
Jul 09 09:33:17.798413 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.328666    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-105.ec2.internal.18c09549f4d058fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-105.ec2.internal,UID:ip-10-0-132-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-105.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-105.ec2.internal,},FirstTimestamp:2026-07-09 09:33:17.311162619 +0000 UTC m=+0.317331681,LastTimestamp:2026-07-09 09:33:17.311162619 +0000 UTC m=+0.317331681,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-105.ec2.internal,}"
Jul 09 09:33:17.798413 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.336540    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-105.ec2.internal.18c09549f4d07bf4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-105.ec2.internal,UID:ip-10-0-132-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-105.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-105.ec2.internal,},FirstTimestamp:2026-07-09 09:33:17.311171572 +0000 UTC m=+0.317340634,LastTimestamp:2026-07-09 09:33:17.311171572 +0000 UTC m=+0.317340634,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-105.ec2.internal,}"
Jul 09 09:33:17.798413 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.345913    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-105.ec2.internal.18c09549f4d00e40\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-105.ec2.internal.18c09549f4d00e40  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-105.ec2.internal,UID:ip-10-0-132-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-105.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-105.ec2.internal,},FirstTimestamp:2026-07-09 09:33:17.311143488 +0000 UTC m=+0.317312550,LastTimestamp:2026-07-09 09:33:17.312232018 +0000 UTC m=+0.318401083,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-105.ec2.internal,}"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.356485    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-105.ec2.internal.18c09549f4d058fb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-105.ec2.internal.18c09549f4d058fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-105.ec2.internal,UID:ip-10-0-132-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-105.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-105.ec2.internal,},FirstTimestamp:2026-07-09 09:33:17.311162619 +0000 UTC m=+0.317331681,LastTimestamp:2026-07-09 09:33:17.312248412 +0000 UTC m=+0.318417475,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-105.ec2.internal,}"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.357340    2577 manager.go:337] "Starting Device Plugin manager"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.357369    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.357377    2577 server.go:87] "Starting device plugin registration server"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.357588    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.357599    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.357669    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.357737    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.357745    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.358348    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.358389    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-105.ec2.internal\" not found"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.365595    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-105.ec2.internal.18c09549f4d07bf4\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-105.ec2.internal.18c09549f4d07bf4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-105.ec2.internal,UID:ip-10-0-132-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-105.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-105.ec2.internal,},FirstTimestamp:2026-07-09 09:33:17.311171572 +0000 UTC m=+0.317340634,LastTimestamp:2026-07-09 09:33:17.312257464 +0000 UTC m=+0.318426526,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-105.ec2.internal,}"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.373102    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-105.ec2.internal.18c09549f7b0f984  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-105.ec2.internal,UID:ip-10-0-132-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-132-105.ec2.internal,},FirstTimestamp:2026-07-09 09:33:17.359438212 +0000 UTC m=+0.365607275,LastTimestamp:2026-07-09 09:33:17.359438212 +0000 UTC m=+0.365607275,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-105.ec2.internal,}"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.458357    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.459047    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.459070    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.459079    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:33:17.798739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.459100    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.468767    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-105.ec2.internal.18c09549f4d00e40\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-105.ec2.internal.18c09549f4d00e40  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-105.ec2.internal,UID:ip-10-0-132-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-105.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-105.ec2.internal,},FirstTimestamp:2026-07-09 09:33:17.311143488 +0000 UTC m=+0.317312550,LastTimestamp:2026-07-09 09:33:17.459056638 +0000 UTC m=+0.465225699,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-105.ec2.internal,}"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.477705    2577 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-132-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.477690    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-105.ec2.internal.18c09549f4d058fb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-105.ec2.internal.18c09549f4d058fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-105.ec2.internal,UID:ip-10-0-132-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-105.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-105.ec2.internal,},FirstTimestamp:2026-07-09 09:33:17.311162619 +0000 UTC m=+0.317331681,LastTimestamp:2026-07-09 09:33:17.459074048 +0000 UTC m=+0.465243110,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-105.ec2.internal,}"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.485506    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-105.ec2.internal.18c09549f4d07bf4\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-105.ec2.internal.18c09549f4d07bf4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-105.ec2.internal,UID:ip-10-0-132-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-105.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-105.ec2.internal,},FirstTimestamp:2026-07-09 09:33:17.311171572 +0000 UTC m=+0.317340634,LastTimestamp:2026-07-09 09:33:17.459083278 +0000 UTC m=+0.465252340,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-105.ec2.internal,}"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.509262    2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-105.ec2.internal\" not found" node="ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.543900    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.545088    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.545120    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.545140    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.545171    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.547869    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.645664    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-105.ec2.internal"]
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.645731    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.646577    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:33:17.799321 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.646611    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.646626    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.649019    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.649145    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.649172    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.650050    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.650081    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.650096    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.650363    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.650397    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.650412    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.652766    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.652789    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.653460    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.653482    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.653491    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.678372    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.679128    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.679155    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.679165    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.679181    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.686383    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-105.ec2.internal\" not found" node="ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.687171    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.687186    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-105.ec2.internal\": node \"ip-10-0-132-105.ec2.internal\" not found"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.690949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5098063099125629f18ce56638bbe71b-config\") pod \"kube-apiserver-proxy-ip-10-0-132-105.ec2.internal\" (UID: \"5098063099125629f18ce56638bbe71b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.690971    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/52dd71d3230253d58a73e46fba17ae9a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal\" (UID: \"52dd71d3230253d58a73e46fba17ae9a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.690986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52dd71d3230253d58a73e46fba17ae9a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal\" (UID: \"52dd71d3230253d58a73e46fba17ae9a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.799758 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.691302    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-105.ec2.internal\" not found" node="ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.800530 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.717383    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-105.ec2.internal\" not found"
Jul 09 09:33:17.800530 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.791504    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5098063099125629f18ce56638bbe71b-config\") pod \"kube-apiserver-proxy-ip-10-0-132-105.ec2.internal\" (UID: \"5098063099125629f18ce56638bbe71b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.800530 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.791541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/52dd71d3230253d58a73e46fba17ae9a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal\" (UID: \"52dd71d3230253d58a73e46fba17ae9a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.800530 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.791558    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52dd71d3230253d58a73e46fba17ae9a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal\" (UID: \"52dd71d3230253d58a73e46fba17ae9a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.800530 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.791617    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/52dd71d3230253d58a73e46fba17ae9a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal\" (UID: \"52dd71d3230253d58a73e46fba17ae9a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.800530 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.791646    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52dd71d3230253d58a73e46fba17ae9a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal\" (UID: \"52dd71d3230253d58a73e46fba17ae9a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.800530 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.791617    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5098063099125629f18ce56638bbe71b-config\") pod \"kube-apiserver-proxy-ip-10-0-132-105.ec2.internal\" (UID: \"5098063099125629f18ce56638bbe71b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:17.945057 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.817618    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-105.ec2.internal\" not found"
Jul 09 09:33:17.945057 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:17.918044    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-105.ec2.internal\" not found"
Jul 09 09:33:18.016870 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:17.989222    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:18.016870 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.015671    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:18.091367 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:18.018607    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-105.ec2.internal\" not found"
Jul 09 09:33:18.186520 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:18.120723    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-105.ec2.internal\" not found"
Jul 09 09:33:18.186520 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.125276    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 09:33:18.285198 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:18.222506    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-105.ec2.internal\" not found"
Jul 09 09:33:18.285198 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.235640    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 09:33:18.285198 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.235759    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 09:33:18.285198 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.235813    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 09:33:18.378674 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:18.323851    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-105.ec2.internal\" not found"
Jul 09 09:33:18.460105 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:18.424377    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-105.ec2.internal\" not found"
Jul 09 09:33:18.468312 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.467636    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:33:18.469851 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.469327    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:33:18.469851 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.469357    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:33:18.469851 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.469372    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:33:18.556301 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.484149    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 09:28:18 +0000 UTC" deadline="2028-01-05 09:45:22.606608712 +0000 UTC"
Jul 09 09:33:18.556301 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.484171    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13080h12m4.122440741s"
Jul 09 09:33:18.556301 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.498422    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 09:33:18.649673 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.590893    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:18.649673 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.600326    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 09:33:18.649673 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.601898    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal"
Jul 09 09:33:18.649673 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.620386    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 09:33:18.852472 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:18.749608    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52dd71d3230253d58a73e46fba17ae9a.slice/crio-38e97de5e75a693c00450f62f265ba888b063b23f62ad96c368870832208da2c WatchSource:0}: Error finding container 38e97de5e75a693c00450f62f265ba888b063b23f62ad96c368870832208da2c: Status 404 returned error can't find the container with id 38e97de5e75a693c00450f62f265ba888b063b23f62ad96c368870832208da2c
Jul 09 09:33:18.852472 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:18.749839    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5098063099125629f18ce56638bbe71b.slice/crio-e974632904eb686c7392cf7dfcca7f49d491d05775fffffef53909427421137e WatchSource:0}: Error finding container e974632904eb686c7392cf7dfcca7f49d491d05775fffffef53909427421137e: Status 404 returned error can't find the container with id e974632904eb686c7392cf7dfcca7f49d491d05775fffffef53909427421137e
Jul 09 09:33:18.852472 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.754996    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 09:33:18.852472 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:18.785204    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 09:33:19.272543 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.272521    2577 apiserver.go:52] "Watching apiserver"
Jul 09 09:33:19.279717 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.279500    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 09:33:19.279986 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.279959    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-spd2m","openshift-image-registry/node-ca-skw5s","openshift-multus/multus-4mwbb","openshift-multus/multus-additional-cni-plugins-9gtlt","openshift-network-diagnostics/network-check-target-sjr8w","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6","openshift-dns/node-resolver-4nx9g","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal","openshift-multus/network-metrics-daemon-dsk4h","openshift-network-operator/iptables-alerter-dmr4s","openshift-ovn-kubernetes/ovnkube-node-v8sxk","kube-system/global-pull-secret-syncer-dh44v","kube-system/konnectivity-agent-4ztl5","kube-system/kube-apiserver-proxy-ip-10-0-132-105.ec2.internal"]
Jul 09 09:33:19.282573 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.282548    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4nx9g"
Jul 09 09:33:19.285028 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.285007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4ztl5"
Jul 09 09:33:19.285210 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.285185    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 09:33:19.285580 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.285534    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 09:33:19.285642 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.285605    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xxt2g\""
Jul 09 09:33:19.287157 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.287135    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 09:33:19.287268 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.287250    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-k8c2g\""
Jul 09 09:33:19.287651 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.287630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.288522 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.288500    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 09:33:19.290063 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.290034    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 09:33:19.290322 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.290300    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kms9d\""
Jul 09 09:33:19.291188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.290867    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.291894 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.291636    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 09:33:19.291996 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.291908    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 09:33:19.293936 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.292182    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 09:33:19.293936 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.293565    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:19.293936 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:19.293638    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:19.296172 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.296155    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.298691 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.298670    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8bc44b5-f4ab-4662-b621-acabb3056f84-multus-daemon-config\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.298791 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.298710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-cnibin\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.298791 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.298737    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-os-release\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.298791 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.298765    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e639bf5-c03c-46fa-a7ba-7d6770be1e10-tmp-dir\") pod \"node-resolver-4nx9g\" (UID: \"7e639bf5-c03c-46fa-a7ba-7d6770be1e10\") " pod="openshift-dns/node-resolver-4nx9g"
Jul 09 09:33:19.298791 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.298787    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngwlr\" (UniqueName: \"kubernetes.io/projected/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-kube-api-access-ngwlr\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.299031 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.298816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-run-k8s-cni-cncf-io\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.299031 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.298842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-var-lib-kubelet\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.299031 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.298868    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e639bf5-c03c-46fa-a7ba-7d6770be1e10-hosts-file\") pod \"node-resolver-4nx9g\" (UID: \"7e639bf5-c03c-46fa-a7ba-7d6770be1e10\") " pod="openshift-dns/node-resolver-4nx9g"
Jul 09 09:33:19.299031 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.298911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n9zv\" (UniqueName: \"kubernetes.io/projected/b8bc44b5-f4ab-4662-b621-acabb3056f84-kube-api-access-4n9zv\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.299031 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.298934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.299031 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.298960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-multus-conf-dir\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.299031 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.298985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.302667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.302648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-var-lib-cni-multus\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.303018 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.302903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-multus-socket-dir-parent\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.303291 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303221    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/216f2721-d98c-411d-952b-dd600dc4642e-agent-certs\") pod \"konnectivity-agent-4ztl5\" (UID: \"216f2721-d98c-411d-952b-dd600dc4642e\") " pod="kube-system/konnectivity-agent-4ztl5"
Jul 09 09:33:19.303366 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303315    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 09:33:19.303366 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/216f2721-d98c-411d-952b-dd600dc4642e-konnectivity-ca\") pod \"konnectivity-agent-4ztl5\" (UID: \"216f2721-d98c-411d-952b-dd600dc4642e\") " pod="kube-system/konnectivity-agent-4ztl5"
Jul 09 09:33:19.303366 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303355    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-var-lib-cni-bin\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.303518 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303404    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.303577 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303522    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 09:33:19.303577 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-multus-cni-dir\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.303676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303597    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8bc44b5-f4ab-4662-b621-acabb3056f84-cni-binary-copy\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.303676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303659    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-etc-kubernetes\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.303780 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303695    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8b6h9\""
Jul 09 09:33:19.303780 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.303780 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqs7j\" (UniqueName: \"kubernetes.io/projected/7e639bf5-c03c-46fa-a7ba-7d6770be1e10-kube-api-access-qqs7j\") pod \"node-resolver-4nx9g\" (UID: \"7e639bf5-c03c-46fa-a7ba-7d6770be1e10\") " pod="openshift-dns/node-resolver-4nx9g"
Jul 09 09:33:19.303963 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303785    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-run-netns\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.303963 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303791    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 09:33:19.303963 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303813    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-system-cni-dir\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.303963 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303833    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wq74z\""
Jul 09 09:33:19.304160 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.303982    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 09:33:19.304647 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.304623    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-os-release\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.304745 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.304663    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-system-cni-dir\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.304745 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.304716    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-cnibin\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.304910 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.304772    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-run-multus-certs\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.304910 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.304797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-hostroot\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.305015 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.304940    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 09:33:19.305697 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.305658    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:19.305908 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:19.305769    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:19.306951 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.306930    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 09:33:19.308726 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.308700    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 09:33:19.309011 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.308992    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.311088 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.311067    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-skw5s"
Jul 09 09:33:19.311269 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.311251    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 09:33:19.311546 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.311523    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 09:33:19.311807 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.311789    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mglgr\""
Jul 09 09:33:19.313630 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.313198    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 09:33:19.313630 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.313459    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 09:33:19.313630 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.313616    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 09:33:19.313855 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.313725    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9vhdz\""
Jul 09 09:33:19.314192 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.314169    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dmr4s"
Jul 09 09:33:19.316530 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.316511    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 09:33:19.316768 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.316747    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fn48m\""
Jul 09 09:33:19.317086 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.317038    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 09:33:19.317252 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.317237    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 09:33:19.317525 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.317383    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.320081 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.320060    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 09:33:19.320365 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.320346    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 09:33:19.320448 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.320371    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 09:33:19.320510 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.320501    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:19.320663 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:19.320561    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:19.320663 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.320600    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cshn4\""
Jul 09 09:33:19.320775 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.320670    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 09:33:19.320890 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.320826    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 09:33:19.320957 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.320829    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 09:33:19.373587 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.373562    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-spzq7"
Jul 09 09:33:19.381732 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.381712    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-spzq7"
Jul 09 09:33:19.394896 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.393200    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 09:33:19.405503 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405478    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89c02a74-d88c-49c1-b9fc-d5c915e51d7a-host-slash\") pod \"iptables-alerter-dmr4s\" (UID: \"89c02a74-d88c-49c1-b9fc-d5c915e51d7a\") " pod="openshift-network-operator/iptables-alerter-dmr4s"
Jul 09 09:33:19.405622 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2f492c23-72ed-4bbf-899d-3d5f588ef133-kubelet-config\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:19.405622 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-var-lib-kubelet\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.405622 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405582    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.405622 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/45c0bd3d-9996-4fa7-9afd-b4027c3286a0-host\") pod \"node-ca-skw5s\" (UID: \"45c0bd3d-9996-4fa7-9afd-b4027c3286a0\") " pod="openshift-image-registry/node-ca-skw5s"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e639bf5-c03c-46fa-a7ba-7d6770be1e10-hosts-file\") pod \"node-resolver-4nx9g\" (UID: \"7e639bf5-c03c-46fa-a7ba-7d6770be1e10\") " pod="openshift-dns/node-resolver-4nx9g"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-systemd-units\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-run-openvswitch\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ljz8\" (UniqueName: \"kubernetes.io/projected/25f651bd-0410-4535-8d61-adcd93c21da8-kube-api-access-2ljz8\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405839    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-var-lib-kubelet\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405827    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-kubernetes\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-multus-conf-dir\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405915    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e639bf5-c03c-46fa-a7ba-7d6770be1e10-hosts-file\") pod \"node-resolver-4nx9g\" (UID: \"7e639bf5-c03c-46fa-a7ba-7d6770be1e10\") " pod="openshift-dns/node-resolver-4nx9g"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-systemd\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.405973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-run-ovn-kubernetes\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.406026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/216f2721-d98c-411d-952b-dd600dc4642e-agent-certs\") pod \"konnectivity-agent-4ztl5\" (UID: \"216f2721-d98c-411d-952b-dd600dc4642e\") " pod="kube-system/konnectivity-agent-4ztl5"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.406441    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 09:33:19.407265 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.406587    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.406638    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-run-systemd\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.406658    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-multus-conf-dir\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.406698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-sysctl-conf\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.406728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-multus-cni-dir\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.406751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8bc44b5-f4ab-4662-b621-acabb3056f84-cni-binary-copy\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.406774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.406969    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-multus-cni-dir\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvl8c\" (UniqueName: \"kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c\") pod \"network-check-target-sjr8w\" (UID: \"ca9f20c6-5f63-4f75-87b1-00dc649acbcc\") " pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-lib-modules\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-tuned\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqs7j\" (UniqueName: \"kubernetes.io/projected/7e639bf5-c03c-46fa-a7ba-7d6770be1e10-kube-api-access-qqs7j\") pod \"node-resolver-4nx9g\" (UID: \"7e639bf5-c03c-46fa-a7ba-7d6770be1e10\") " pod="openshift-dns/node-resolver-4nx9g"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407088    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-os-release\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407112    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjjrr\" (UniqueName: \"kubernetes.io/projected/2a16053a-b5b2-4359-80a5-4c07ffce9851-kube-api-access-rjjrr\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.408060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407136    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-run\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407158    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-sys\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407182    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/45c0bd3d-9996-4fa7-9afd-b4027c3286a0-serviceca\") pod \"node-ca-skw5s\" (UID: \"45c0bd3d-9996-4fa7-9afd-b4027c3286a0\") " pod="openshift-image-registry/node-ca-skw5s"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407208    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrcht\" (UniqueName: \"kubernetes.io/projected/45c0bd3d-9996-4fa7-9afd-b4027c3286a0-kube-api-access-mrcht\") pod \"node-ca-skw5s\" (UID: \"45c0bd3d-9996-4fa7-9afd-b4027c3286a0\") " pod="openshift-image-registry/node-ca-skw5s"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407233    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-system-cni-dir\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-cnibin\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-run-multus-certs\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407293    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8bc44b5-f4ab-4662-b621-acabb3056f84-cni-binary-copy\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407310    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-sys-fs\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-hostroot\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407361    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-cnibin\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407385    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e639bf5-c03c-46fa-a7ba-7d6770be1e10-tmp-dir\") pod \"node-resolver-4nx9g\" (UID: \"7e639bf5-c03c-46fa-a7ba-7d6770be1e10\") " pod="openshift-dns/node-resolver-4nx9g"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407411    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-var-lib-openvswitch\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407436    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/25f651bd-0410-4535-8d61-adcd93c21da8-ovnkube-config\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.408676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407461    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-device-dir\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407488    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz8b2\" (UniqueName: \"kubernetes.io/projected/f67c0d3d-90bd-44c9-89b0-754fe29f288e-kube-api-access-sz8b2\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-run-k8s-cni-cncf-io\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407550    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-etc-openvswitch\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407576    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-registration-dir\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407613    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-host\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407639    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n9zv\" (UniqueName: \"kubernetes.io/projected/b8bc44b5-f4ab-4662-b621-acabb3056f84-kube-api-access-4n9zv\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407663    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-cni-bin\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407687    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-var-lib-kubelet\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407699    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-os-release\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-slash\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407791    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-hostroot\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.407835    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-cnibin\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.409373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408148    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e639bf5-c03c-46fa-a7ba-7d6770be1e10-tmp-dir\") pod \"node-resolver-4nx9g\" (UID: \"7e639bf5-c03c-46fa-a7ba-7d6770be1e10\") " pod="openshift-dns/node-resolver-4nx9g"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408259    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408262    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-run-k8s-cni-cncf-io\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408336    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-cnibin\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408380    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-system-cni-dir\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408410    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-run-multus-certs\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-run-netns\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-node-log\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/25f651bd-0410-4535-8d61-adcd93c21da8-ovn-node-metrics-cert\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-modprobe-d\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408554    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/89c02a74-d88c-49c1-b9fc-d5c915e51d7a-iptables-alerter-script\") pod \"iptables-alerter-dmr4s\" (UID: \"89c02a74-d88c-49c1-b9fc-d5c915e51d7a\") " pod="openshift-network-operator/iptables-alerter-dmr4s"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlsph\" (UniqueName: \"kubernetes.io/projected/89c02a74-d88c-49c1-b9fc-d5c915e51d7a-kube-api-access-nlsph\") pod \"iptables-alerter-dmr4s\" (UID: \"89c02a74-d88c-49c1-b9fc-d5c915e51d7a\") " pod="openshift-network-operator/iptables-alerter-dmr4s"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-run-ovn\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408659    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/25f651bd-0410-4535-8d61-adcd93c21da8-env-overrides\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.410213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/25f651bd-0410-4535-8d61-adcd93c21da8-ovnkube-script-lib\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408746    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-var-lib-cni-multus\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408820    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-var-lib-cni-multus\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-multus-socket-dir-parent\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408915    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/216f2721-d98c-411d-952b-dd600dc4642e-konnectivity-ca\") pod \"konnectivity-agent-4ztl5\" (UID: \"216f2721-d98c-411d-952b-dd600dc4642e\") " pod="kube-system/konnectivity-agent-4ztl5"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-var-lib-cni-bin\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.408995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-etc-kubernetes\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-kubelet\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409168    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-socket-dir\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2f492c23-72ed-4bbf-899d-3d5f588ef133-dbus\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409231    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs\") pod \"network-metrics-daemon-dsk4h\" (UID: \"8e94d5d6-6e3b-4c0e-b73d-e8f086218175\") " pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:19.410935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409295    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-run-netns\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409323    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-system-cni-dir\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409395    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-sysctl-d\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409432    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggvkq\" (UniqueName: \"kubernetes.io/projected/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-kube-api-access-ggvkq\") pod \"network-metrics-daemon-dsk4h\" (UID: \"8e94d5d6-6e3b-4c0e-b73d-e8f086218175\") " pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-log-socket\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-cni-netd\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-etc-selinux\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409536    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-sysconfig\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409551    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/216f2721-d98c-411d-952b-dd600dc4642e-konnectivity-ca\") pod \"konnectivity-agent-4ztl5\" (UID: \"216f2721-d98c-411d-952b-dd600dc4642e\") " pod="kube-system/konnectivity-agent-4ztl5"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f67c0d3d-90bd-44c9-89b0-754fe29f288e-tmp\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409595    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-var-lib-cni-bin\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8bc44b5-f4ab-4662-b621-acabb3056f84-multus-daemon-config\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409626    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-os-release\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409643    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-multus-socket-dir-parent\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.411635 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.409672    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-system-cni-dir\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.412368 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.410092    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-etc-kubernetes\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.412368 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.410103    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.412368 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.410170    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-os-release\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.412368 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.410210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngwlr\" (UniqueName: \"kubernetes.io/projected/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-kube-api-access-ngwlr\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.412368 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.410210    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8bc44b5-f4ab-4662-b621-acabb3056f84-multus-daemon-config\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.412368 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.410232    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.412368 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.410262    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8bc44b5-f4ab-4662-b621-acabb3056f84-host-run-netns\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.412712 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.412692    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/216f2721-d98c-411d-952b-dd600dc4642e-agent-certs\") pod \"konnectivity-agent-4ztl5\" (UID: \"216f2721-d98c-411d-952b-dd600dc4642e\") " pod="kube-system/konnectivity-agent-4ztl5"
Jul 09 09:33:19.416519 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.416491    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqs7j\" (UniqueName: \"kubernetes.io/projected/7e639bf5-c03c-46fa-a7ba-7d6770be1e10-kube-api-access-qqs7j\") pod \"node-resolver-4nx9g\" (UID: \"7e639bf5-c03c-46fa-a7ba-7d6770be1e10\") " pod="openshift-dns/node-resolver-4nx9g"
Jul 09 09:33:19.417016 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.416996    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n9zv\" (UniqueName: \"kubernetes.io/projected/b8bc44b5-f4ab-4662-b621-acabb3056f84-kube-api-access-4n9zv\") pod \"multus-4mwbb\" (UID: \"b8bc44b5-f4ab-4662-b621-acabb3056f84\") " pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.419420 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.419377    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngwlr\" (UniqueName: \"kubernetes.io/projected/6e8230b2-ffba-4769-888d-6fbf1a2ee9c7-kube-api-access-ngwlr\") pod \"multus-additional-cni-plugins-9gtlt\" (UID: \"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7\") " pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.511106 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-kubelet\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.511106 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511084    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-socket-dir\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.511279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511110    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2f492c23-72ed-4bbf-899d-3d5f588ef133-dbus\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:19.511279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:19.511279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511158    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs\") pod \"network-metrics-daemon-dsk4h\" (UID: \"8e94d5d6-6e3b-4c0e-b73d-e8f086218175\") " pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:19.511279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511186    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-sysctl-d\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.511279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggvkq\" (UniqueName: \"kubernetes.io/projected/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-kube-api-access-ggvkq\") pod \"network-metrics-daemon-dsk4h\" (UID: \"8e94d5d6-6e3b-4c0e-b73d-e8f086218175\") " pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:19.511279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-log-socket\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.511279 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511260    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-socket-dir\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.511279 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:19.511267    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:19.511667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511312    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-cni-netd\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.511667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511329    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2f492c23-72ed-4bbf-899d-3d5f588ef133-dbus\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:19.511667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511186    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-kubelet\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.511667 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:19.511354    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret podName:2f492c23-72ed-4bbf-899d-3d5f588ef133 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:20.011330114 +0000 UTC m=+3.017499168 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret") pod "global-pull-secret-syncer-dh44v" (UID: "2f492c23-72ed-4bbf-899d-3d5f588ef133") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:19.511667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-cni-netd\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.511667 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:19.511406    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:19.511667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-etc-selinux\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.511667 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:19.511453    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs podName:8e94d5d6-6e3b-4c0e-b73d-e8f086218175 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:20.011437054 +0000 UTC m=+3.017606123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs") pod "network-metrics-daemon-dsk4h" (UID: "8e94d5d6-6e3b-4c0e-b73d-e8f086218175") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:19.511667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511485    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-sysconfig\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.511667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511510    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f67c0d3d-90bd-44c9-89b0-754fe29f288e-tmp\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.511667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511532    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-etc-selinux\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.511667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511538    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89c02a74-d88c-49c1-b9fc-d5c915e51d7a-host-slash\") pod \"iptables-alerter-dmr4s\" (UID: \"89c02a74-d88c-49c1-b9fc-d5c915e51d7a\") " pod="openshift-network-operator/iptables-alerter-dmr4s"
Jul 09 09:33:19.511667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511564    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2f492c23-72ed-4bbf-899d-3d5f588ef133-kubelet-config\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511584    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-log-socket\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511617    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-sysctl-d\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511667    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89c02a74-d88c-49c1-b9fc-d5c915e51d7a-host-slash\") pod \"iptables-alerter-dmr4s\" (UID: \"89c02a74-d88c-49c1-b9fc-d5c915e51d7a\") " pod="openshift-network-operator/iptables-alerter-dmr4s"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511621    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/45c0bd3d-9996-4fa7-9afd-b4027c3286a0-host\") pod \"node-ca-skw5s\" (UID: \"45c0bd3d-9996-4fa7-9afd-b4027c3286a0\") " pod="openshift-image-registry/node-ca-skw5s"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511705    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-systemd-units\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511725    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-sysconfig\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-run-openvswitch\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ljz8\" (UniqueName: \"kubernetes.io/projected/25f651bd-0410-4535-8d61-adcd93c21da8-kube-api-access-2ljz8\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511771    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-kubernetes\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511839    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-systemd\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511872    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-run-openvswitch\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-run-ovn-kubernetes\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.512329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511917    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-run-systemd\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511810    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2f492c23-72ed-4bbf-899d-3d5f588ef133-kubelet-config\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511917    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-systemd-units\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511962    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-sysctl-conf\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvl8c\" (UniqueName: \"kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c\") pod \"network-check-target-sjr8w\" (UID: \"ca9f20c6-5f63-4f75-87b1-00dc649acbcc\") " pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512067    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-lib-modules\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-tuned\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjjrr\" (UniqueName: \"kubernetes.io/projected/2a16053a-b5b2-4359-80a5-4c07ffce9851-kube-api-access-rjjrr\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512142    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-run\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512181    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-sys\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512194    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-kubernetes\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/45c0bd3d-9996-4fa7-9afd-b4027c3286a0-serviceca\") pod \"node-ca-skw5s\" (UID: \"45c0bd3d-9996-4fa7-9afd-b4027c3286a0\") " pod="openshift-image-registry/node-ca-skw5s"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512253    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-systemd\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.513105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512290    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrcht\" (UniqueName: \"kubernetes.io/projected/45c0bd3d-9996-4fa7-9afd-b4027c3286a0-kube-api-access-mrcht\") pod \"node-ca-skw5s\" (UID: \"45c0bd3d-9996-4fa7-9afd-b4027c3286a0\") " pod="openshift-image-registry/node-ca-skw5s"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512322    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-sys-fs\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512384    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-sys-fs\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-var-lib-openvswitch\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/25f651bd-0410-4535-8d61-adcd93c21da8-ovnkube-config\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512464    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-device-dir\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz8b2\" (UniqueName: \"kubernetes.io/projected/f67c0d3d-90bd-44c9-89b0-754fe29f288e-kube-api-access-sz8b2\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512520    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-run-systemd\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-etc-openvswitch\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512569    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-etc-openvswitch\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512590    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512604    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-run-ovn-kubernetes\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.511670    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/45c0bd3d-9996-4fa7-9afd-b4027c3286a0-host\") pod \"node-ca-skw5s\" (UID: \"45c0bd3d-9996-4fa7-9afd-b4027c3286a0\") " pod="openshift-image-registry/node-ca-skw5s"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-registration-dir\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.513779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512658    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/45c0bd3d-9996-4fa7-9afd-b4027c3286a0-serviceca\") pod \"node-ca-skw5s\" (UID: \"45c0bd3d-9996-4fa7-9afd-b4027c3286a0\") " pod="openshift-image-registry/node-ca-skw5s"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512665    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-sysctl-conf\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512670    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-registration-dir\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512661    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-host\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512702    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-host\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-cni-bin\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-var-lib-kubelet\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512786    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-slash\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512790    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-run\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512813    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-run-netns\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512836    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-node-log\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/25f651bd-0410-4535-8d61-adcd93c21da8-ovn-node-metrics-cert\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-modprobe-d\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512920    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-lib-modules\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512937    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/89c02a74-d88c-49c1-b9fc-d5c915e51d7a-iptables-alerter-script\") pod \"iptables-alerter-dmr4s\" (UID: \"89c02a74-d88c-49c1-b9fc-d5c915e51d7a\") " pod="openshift-network-operator/iptables-alerter-dmr4s"
Jul 09 09:33:19.514490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512964    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlsph\" (UniqueName: \"kubernetes.io/projected/89c02a74-d88c-49c1-b9fc-d5c915e51d7a-kube-api-access-nlsph\") pod \"iptables-alerter-dmr4s\" (UID: \"89c02a74-d88c-49c1-b9fc-d5c915e51d7a\") " pod="openshift-network-operator/iptables-alerter-dmr4s"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-run-ovn\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512995    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-sys\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.513014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/25f651bd-0410-4535-8d61-adcd93c21da8-env-overrides\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.513040    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-cni-bin\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512724    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2a16053a-b5b2-4359-80a5-4c07ffce9851-device-dir\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.513077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/25f651bd-0410-4535-8d61-adcd93c21da8-ovnkube-script-lib\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.513105    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-var-lib-kubelet\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.512755    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-var-lib-openvswitch\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.513121    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/25f651bd-0410-4535-8d61-adcd93c21da8-ovnkube-config\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.513176    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-run-netns\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.513198    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-host-slash\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.513426    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-modprobe-d\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.513590    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-node-log\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.513656    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/25f651bd-0410-4535-8d61-adcd93c21da8-run-ovn\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.515300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.513727    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/25f651bd-0410-4535-8d61-adcd93c21da8-ovnkube-script-lib\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.516061 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.513777    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/89c02a74-d88c-49c1-b9fc-d5c915e51d7a-iptables-alerter-script\") pod \"iptables-alerter-dmr4s\" (UID: \"89c02a74-d88c-49c1-b9fc-d5c915e51d7a\") " pod="openshift-network-operator/iptables-alerter-dmr4s"
Jul 09 09:33:19.516061 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.514052    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/25f651bd-0410-4535-8d61-adcd93c21da8-env-overrides\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.516061 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.515174    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f67c0d3d-90bd-44c9-89b0-754fe29f288e-tmp\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.518144 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.518082    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/25f651bd-0410-4535-8d61-adcd93c21da8-ovn-node-metrics-cert\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.520746 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.520725    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f67c0d3d-90bd-44c9-89b0-754fe29f288e-etc-tuned\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.522170 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:19.522140    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:33:19.522251 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:19.522179    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:33:19.522251 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:19.522193    2577 projected.go:196] Error preparing data for projected volume kube-api-access-tvl8c for pod openshift-network-diagnostics/network-check-target-sjr8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:19.522370 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:19.522274    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c podName:ca9f20c6-5f63-4f75-87b1-00dc649acbcc nodeName:}" failed. No retries permitted until 2026-07-09 09:33:20.022258079 +0000 UTC m=+3.028427144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tvl8c" (UniqueName: "kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c") pod "network-check-target-sjr8w" (UID: "ca9f20c6-5f63-4f75-87b1-00dc649acbcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:19.524793 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.524365    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggvkq\" (UniqueName: \"kubernetes.io/projected/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-kube-api-access-ggvkq\") pod \"network-metrics-daemon-dsk4h\" (UID: \"8e94d5d6-6e3b-4c0e-b73d-e8f086218175\") " pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:19.528174 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.528128    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ljz8\" (UniqueName: \"kubernetes.io/projected/25f651bd-0410-4535-8d61-adcd93c21da8-kube-api-access-2ljz8\") pod \"ovnkube-node-v8sxk\" (UID: \"25f651bd-0410-4535-8d61-adcd93c21da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.530389 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.530342    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlsph\" (UniqueName: \"kubernetes.io/projected/89c02a74-d88c-49c1-b9fc-d5c915e51d7a-kube-api-access-nlsph\") pod \"iptables-alerter-dmr4s\" (UID: \"89c02a74-d88c-49c1-b9fc-d5c915e51d7a\") " pod="openshift-network-operator/iptables-alerter-dmr4s"
Jul 09 09:33:19.530612 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.530587    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrcht\" (UniqueName: \"kubernetes.io/projected/45c0bd3d-9996-4fa7-9afd-b4027c3286a0-kube-api-access-mrcht\") pod \"node-ca-skw5s\" (UID: \"45c0bd3d-9996-4fa7-9afd-b4027c3286a0\") " pod="openshift-image-registry/node-ca-skw5s"
Jul 09 09:33:19.530700 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.530594    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjjrr\" (UniqueName: \"kubernetes.io/projected/2a16053a-b5b2-4359-80a5-4c07ffce9851-kube-api-access-rjjrr\") pod \"aws-ebs-csi-driver-node-zmpg6\" (UID: \"2a16053a-b5b2-4359-80a5-4c07ffce9851\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.532998 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.532975    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz8b2\" (UniqueName: \"kubernetes.io/projected/f67c0d3d-90bd-44c9-89b0-754fe29f288e-kube-api-access-sz8b2\") pod \"tuned-spd2m\" (UID: \"f67c0d3d-90bd-44c9-89b0-754fe29f288e\") " pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.544702 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.544674    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 09:33:19.553704 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.553655    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal" event={"ID":"52dd71d3230253d58a73e46fba17ae9a","Type":"ContainerStarted","Data":"38e97de5e75a693c00450f62f265ba888b063b23f62ad96c368870832208da2c"}
Jul 09 09:33:19.555281 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.555253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-105.ec2.internal" event={"ID":"5098063099125629f18ce56638bbe71b","Type":"ContainerStarted","Data":"e974632904eb686c7392cf7dfcca7f49d491d05775fffffef53909427421137e"}
Jul 09 09:33:19.598767 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.598741    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4nx9g"
Jul 09 09:33:19.602903 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.602881    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4ztl5"
Jul 09 09:33:19.610793 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:19.610768    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e639bf5_c03c_46fa_a7ba_7d6770be1e10.slice/crio-0162f0ae43a617bef957827fb22b551a557ba65e4d0cd4ca5e3275f3684c73cf WatchSource:0}: Error finding container 0162f0ae43a617bef957827fb22b551a557ba65e4d0cd4ca5e3275f3684c73cf: Status 404 returned error can't find the container with id 0162f0ae43a617bef957827fb22b551a557ba65e4d0cd4ca5e3275f3684c73cf
Jul 09 09:33:19.614699 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:19.614662    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216f2721_d98c_411d_952b_dd600dc4642e.slice/crio-d6072ac8e1983f3287e98954167318291d805eba69f954e8f7a103f2f0f473f3 WatchSource:0}: Error finding container d6072ac8e1983f3287e98954167318291d805eba69f954e8f7a103f2f0f473f3: Status 404 returned error can't find the container with id d6072ac8e1983f3287e98954167318291d805eba69f954e8f7a103f2f0f473f3
Jul 09 09:33:19.618417 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.618394    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4mwbb"
Jul 09 09:33:19.624670 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.624644    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9gtlt"
Jul 09 09:33:19.629524 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:19.629497    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8bc44b5_f4ab_4662_b621_acabb3056f84.slice/crio-be8333f06514c05a61f553513f576ff3a50a5e1d8c07789593f15b880328641d WatchSource:0}: Error finding container be8333f06514c05a61f553513f576ff3a50a5e1d8c07789593f15b880328641d: Status 404 returned error can't find the container with id be8333f06514c05a61f553513f576ff3a50a5e1d8c07789593f15b880328641d
Jul 09 09:33:19.633254 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.633230    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6"
Jul 09 09:33:19.636771 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:19.636748    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e8230b2_ffba_4769_888d_6fbf1a2ee9c7.slice/crio-9355237abdc86f46b5077ede555ae0d26c5bb28a8579c6d77a726509744f8dd3 WatchSource:0}: Error finding container 9355237abdc86f46b5077ede555ae0d26c5bb28a8579c6d77a726509744f8dd3: Status 404 returned error can't find the container with id 9355237abdc86f46b5077ede555ae0d26c5bb28a8579c6d77a726509744f8dd3
Jul 09 09:33:19.642305 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.642284    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-spd2m"
Jul 09 09:33:19.643770 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:19.643735    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a16053a_b5b2_4359_80a5_4c07ffce9851.slice/crio-4cc62249b802871023c2b97bca40b66cbff09c29cadc622a8375ea577403ac1a WatchSource:0}: Error finding container 4cc62249b802871023c2b97bca40b66cbff09c29cadc622a8375ea577403ac1a: Status 404 returned error can't find the container with id 4cc62249b802871023c2b97bca40b66cbff09c29cadc622a8375ea577403ac1a
Jul 09 09:33:19.650305 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.650287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-skw5s"
Jul 09 09:33:19.653312 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:19.653279    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf67c0d3d_90bd_44c9_89b0_754fe29f288e.slice/crio-7267e5f68db0426e58c34eae5c0b79a6418f042ea81a3e790cd7df8bb9bda39d WatchSource:0}: Error finding container 7267e5f68db0426e58c34eae5c0b79a6418f042ea81a3e790cd7df8bb9bda39d: Status 404 returned error can't find the container with id 7267e5f68db0426e58c34eae5c0b79a6418f042ea81a3e790cd7df8bb9bda39d
Jul 09 09:33:19.658827 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.658667    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dmr4s"
Jul 09 09:33:19.661813 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:19.661755    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45c0bd3d_9996_4fa7_9afd_b4027c3286a0.slice/crio-52865ce5dd48ff368827da9c00a2140ed275c8ee0231d176ad7cac7f7fbf462a WatchSource:0}: Error finding container 52865ce5dd48ff368827da9c00a2140ed275c8ee0231d176ad7cac7f7fbf462a: Status 404 returned error can't find the container with id 52865ce5dd48ff368827da9c00a2140ed275c8ee0231d176ad7cac7f7fbf462a
Jul 09 09:33:19.667520 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:19.667501    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:19.673701 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:19.673680    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c02a74_d88c_49c1_b9fc_d5c915e51d7a.slice/crio-79c88f002b3b2d800582cd26bde77e007465a97903060f16fdd2de4c7c085d51 WatchSource:0}: Error finding container 79c88f002b3b2d800582cd26bde77e007465a97903060f16fdd2de4c7c085d51: Status 404 returned error can't find the container with id 79c88f002b3b2d800582cd26bde77e007465a97903060f16fdd2de4c7c085d51
Jul 09 09:33:19.679948 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:19.679924    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25f651bd_0410_4535_8d61_adcd93c21da8.slice/crio-cbf3e85544e398f712889df2506c86d5c3c46570bcb9009aa41b35fde56ff848 WatchSource:0}: Error finding container cbf3e85544e398f712889df2506c86d5c3c46570bcb9009aa41b35fde56ff848: Status 404 returned error can't find the container with id cbf3e85544e398f712889df2506c86d5c3c46570bcb9009aa41b35fde56ff848
Jul 09 09:33:20.017377 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.017340    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:20.017377 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.017381    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs\") pod \"network-metrics-daemon-dsk4h\" (UID: \"8e94d5d6-6e3b-4c0e-b73d-e8f086218175\") " pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:20.017593 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:20.017515    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:20.017593 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:20.017524    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:20.017593 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:20.017575    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs podName:8e94d5d6-6e3b-4c0e-b73d-e8f086218175 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:21.017557227 +0000 UTC m=+4.023726296 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs") pod "network-metrics-daemon-dsk4h" (UID: "8e94d5d6-6e3b-4c0e-b73d-e8f086218175") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:20.017738 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:20.017595    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret podName:2f492c23-72ed-4bbf-899d-3d5f588ef133 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:21.017584223 +0000 UTC m=+4.023753279 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret") pod "global-pull-secret-syncer-dh44v" (UID: "2f492c23-72ed-4bbf-899d-3d5f588ef133") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:20.118145 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.118115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvl8c\" (UniqueName: \"kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c\") pod \"network-check-target-sjr8w\" (UID: \"ca9f20c6-5f63-4f75-87b1-00dc649acbcc\") " pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:20.118311 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:20.118242    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:33:20.118311 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:20.118264    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:33:20.118311 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:20.118276    2577 projected.go:196] Error preparing data for projected volume kube-api-access-tvl8c for pod openshift-network-diagnostics/network-check-target-sjr8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:20.118442 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:20.118333    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c podName:ca9f20c6-5f63-4f75-87b1-00dc649acbcc nodeName:}" failed. No retries permitted until 2026-07-09 09:33:21.118317141 +0000 UTC m=+4.124486193 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tvl8c" (UniqueName: "kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c") pod "network-check-target-sjr8w" (UID: "ca9f20c6-5f63-4f75-87b1-00dc649acbcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:20.260990 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.260964    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 09:33:20.385101 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.384990    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 09:28:20 +0000 UTC" deadline="2028-04-19 00:16:39.87654944 +0000 UTC"
Jul 09 09:33:20.385101 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.385054    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15590h43m19.491497957s"
Jul 09 09:33:20.546134 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.546107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:20.546298 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.546107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:20.546298 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:20.546210    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:20.546406 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:20.546310    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:20.558050 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.558024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk" event={"ID":"25f651bd-0410-4535-8d61-adcd93c21da8","Type":"ContainerStarted","Data":"cbf3e85544e398f712889df2506c86d5c3c46570bcb9009aa41b35fde56ff848"}
Jul 09 09:33:20.559052 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.559022    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-skw5s" event={"ID":"45c0bd3d-9996-4fa7-9afd-b4027c3286a0","Type":"ContainerStarted","Data":"52865ce5dd48ff368827da9c00a2140ed275c8ee0231d176ad7cac7f7fbf462a"}
Jul 09 09:33:20.560276 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.560192    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6" event={"ID":"2a16053a-b5b2-4359-80a5-4c07ffce9851","Type":"ContainerStarted","Data":"4cc62249b802871023c2b97bca40b66cbff09c29cadc622a8375ea577403ac1a"}
Jul 09 09:33:20.561400 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.561368    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9gtlt" event={"ID":"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7","Type":"ContainerStarted","Data":"9355237abdc86f46b5077ede555ae0d26c5bb28a8579c6d77a726509744f8dd3"}
Jul 09 09:33:20.562501 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.562481    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4mwbb" event={"ID":"b8bc44b5-f4ab-4662-b621-acabb3056f84","Type":"ContainerStarted","Data":"be8333f06514c05a61f553513f576ff3a50a5e1d8c07789593f15b880328641d"}
Jul 09 09:33:20.563576 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.563552    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4nx9g" event={"ID":"7e639bf5-c03c-46fa-a7ba-7d6770be1e10","Type":"ContainerStarted","Data":"0162f0ae43a617bef957827fb22b551a557ba65e4d0cd4ca5e3275f3684c73cf"}
Jul 09 09:33:20.564638 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.564619    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dmr4s" event={"ID":"89c02a74-d88c-49c1-b9fc-d5c915e51d7a","Type":"ContainerStarted","Data":"79c88f002b3b2d800582cd26bde77e007465a97903060f16fdd2de4c7c085d51"}
Jul 09 09:33:20.565684 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.565654    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-spd2m" event={"ID":"f67c0d3d-90bd-44c9-89b0-754fe29f288e","Type":"ContainerStarted","Data":"7267e5f68db0426e58c34eae5c0b79a6418f042ea81a3e790cd7df8bb9bda39d"}
Jul 09 09:33:20.566796 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:20.566771    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4ztl5" event={"ID":"216f2721-d98c-411d-952b-dd600dc4642e","Type":"ContainerStarted","Data":"d6072ac8e1983f3287e98954167318291d805eba69f954e8f7a103f2f0f473f3"}
Jul 09 09:33:21.026178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:21.026141    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:21.026353 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:21.026192    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs\") pod \"network-metrics-daemon-dsk4h\" (UID: \"8e94d5d6-6e3b-4c0e-b73d-e8f086218175\") " pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:21.026416 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:21.026354    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:21.026487 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:21.026415    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs podName:8e94d5d6-6e3b-4c0e-b73d-e8f086218175 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:23.026395763 +0000 UTC m=+6.032564823 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs") pod "network-metrics-daemon-dsk4h" (UID: "8e94d5d6-6e3b-4c0e-b73d-e8f086218175") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:21.026555 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:21.026485    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:21.026555 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:21.026521    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret podName:2f492c23-72ed-4bbf-899d-3d5f588ef133 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:23.026509517 +0000 UTC m=+6.032678568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret") pod "global-pull-secret-syncer-dh44v" (UID: "2f492c23-72ed-4bbf-899d-3d5f588ef133") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:21.127219 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:21.127180    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvl8c\" (UniqueName: \"kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c\") pod \"network-check-target-sjr8w\" (UID: \"ca9f20c6-5f63-4f75-87b1-00dc649acbcc\") " pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:21.127380 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:21.127356    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:33:21.127380 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:21.127379    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:33:21.127511 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:21.127394    2577 projected.go:196] Error preparing data for projected volume kube-api-access-tvl8c for pod openshift-network-diagnostics/network-check-target-sjr8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:21.127511 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:21.127448    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c podName:ca9f20c6-5f63-4f75-87b1-00dc649acbcc nodeName:}" failed. No retries permitted until 2026-07-09 09:33:23.127430684 +0000 UTC m=+6.133599750 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tvl8c" (UniqueName: "kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c") pod "network-check-target-sjr8w" (UID: "ca9f20c6-5f63-4f75-87b1-00dc649acbcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:21.385814 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:21.385729    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 09:28:20 +0000 UTC" deadline="2028-03-03 03:48:22.328249916 +0000 UTC"
Jul 09 09:33:21.385814 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:21.385769    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14466h15m0.942484837s"
Jul 09 09:33:21.549215 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:21.549190    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:21.549362 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:21.549334    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:21.580779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:21.580127    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-105.ec2.internal" event={"ID":"5098063099125629f18ce56638bbe71b","Type":"ContainerStarted","Data":"785ec717a72c08f7bb71cfcd6208787a68ab03c80ad54882c39e90a62e3e5444"}
Jul 09 09:33:22.547235 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:22.546225    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:22.547235 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:22.546603    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:22.547235 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:22.547047    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:22.547235 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:22.547142    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:22.606860 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:22.606825    2577 generic.go:356] "Generic (PLEG): container finished" podID="52dd71d3230253d58a73e46fba17ae9a" containerID="a9d4dc59b22f185f7d709955a363f480807b23f27fdf0f388cfb7a7bf3ee68aa" exitCode=0
Jul 09 09:33:22.607789 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:22.607761    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal" event={"ID":"52dd71d3230253d58a73e46fba17ae9a","Type":"ContainerDied","Data":"a9d4dc59b22f185f7d709955a363f480807b23f27fdf0f388cfb7a7bf3ee68aa"}
Jul 09 09:33:22.627322 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:22.626512    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-105.ec2.internal" podStartSLOduration=3.626492465 podStartE2EDuration="3.626492465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:33:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:33:21.604060894 +0000 UTC m=+4.610229970" watchObservedRunningTime="2026-07-09 09:33:22.626492465 +0000 UTC m=+5.632661542"
Jul 09 09:33:23.044508 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:23.044472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:23.044697 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:23.044560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs\") pod \"network-metrics-daemon-dsk4h\" (UID: \"8e94d5d6-6e3b-4c0e-b73d-e8f086218175\") " pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:23.044893 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:23.044870    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:23.044972 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:23.044949    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs podName:8e94d5d6-6e3b-4c0e-b73d-e8f086218175 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:27.044928083 +0000 UTC m=+10.051097148 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs") pod "network-metrics-daemon-dsk4h" (UID: "8e94d5d6-6e3b-4c0e-b73d-e8f086218175") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:23.045429 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:23.045410    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:23.045491 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:23.045469    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret podName:2f492c23-72ed-4bbf-899d-3d5f588ef133 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:27.045452851 +0000 UTC m=+10.051621909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret") pod "global-pull-secret-syncer-dh44v" (UID: "2f492c23-72ed-4bbf-899d-3d5f588ef133") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:23.145477 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:23.145392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvl8c\" (UniqueName: \"kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c\") pod \"network-check-target-sjr8w\" (UID: \"ca9f20c6-5f63-4f75-87b1-00dc649acbcc\") " pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:23.145640 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:23.145587    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:33:23.145640 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:23.145611    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:33:23.145640 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:23.145624    2577 projected.go:196] Error preparing data for projected volume kube-api-access-tvl8c for pod openshift-network-diagnostics/network-check-target-sjr8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:23.145808 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:23.145686    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c podName:ca9f20c6-5f63-4f75-87b1-00dc649acbcc nodeName:}" failed. No retries permitted until 2026-07-09 09:33:27.145665883 +0000 UTC m=+10.151834940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tvl8c" (UniqueName: "kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c") pod "network-check-target-sjr8w" (UID: "ca9f20c6-5f63-4f75-87b1-00dc649acbcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:23.546122 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:23.546088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:23.546313 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:23.546241    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:23.616528 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:23.616490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal" event={"ID":"52dd71d3230253d58a73e46fba17ae9a","Type":"ContainerStarted","Data":"4d67f58bff4769250111b1abdbe76ffae17e894d20aa1885107c8475bd19c92c"}
Jul 09 09:33:24.545391 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:24.545359    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:24.545586 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:24.545499    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:24.545586 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:24.545579    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:24.545728 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:24.545684    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:25.547349 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:25.546449    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:25.547349 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:25.546605    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:26.546069 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:26.546037    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:26.546250 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:26.546172    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:26.546334 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:26.546245    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:26.546389 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:26.546352    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:27.082396 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:27.082357    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:27.082919 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:27.082408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs\") pod \"network-metrics-daemon-dsk4h\" (UID: \"8e94d5d6-6e3b-4c0e-b73d-e8f086218175\") " pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:27.082919 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:27.082534    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:27.082919 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:27.082595    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs podName:8e94d5d6-6e3b-4c0e-b73d-e8f086218175 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:35.08257579 +0000 UTC m=+18.088744848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs") pod "network-metrics-daemon-dsk4h" (UID: "8e94d5d6-6e3b-4c0e-b73d-e8f086218175") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:27.082919 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:27.082722    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:27.082919 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:27.082778    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret podName:2f492c23-72ed-4bbf-899d-3d5f588ef133 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:35.082762157 +0000 UTC m=+18.088931211 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret") pod "global-pull-secret-syncer-dh44v" (UID: "2f492c23-72ed-4bbf-899d-3d5f588ef133") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:27.182761 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:27.182728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvl8c\" (UniqueName: \"kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c\") pod \"network-check-target-sjr8w\" (UID: \"ca9f20c6-5f63-4f75-87b1-00dc649acbcc\") " pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:27.182940 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:27.182908    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:33:27.182940 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:27.182928    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:33:27.182940 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:27.182938    2577 projected.go:196] Error preparing data for projected volume kube-api-access-tvl8c for pod openshift-network-diagnostics/network-check-target-sjr8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:27.183091 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:27.182991    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c podName:ca9f20c6-5f63-4f75-87b1-00dc649acbcc nodeName:}" failed. No retries permitted until 2026-07-09 09:33:35.182972861 +0000 UTC m=+18.189141916 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tvl8c" (UniqueName: "kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c") pod "network-check-target-sjr8w" (UID: "ca9f20c6-5f63-4f75-87b1-00dc649acbcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:27.548094 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:27.547492    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:27.548094 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:27.547621    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:28.546318 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:28.546182    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:28.546318 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:28.546289    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:28.546807 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:28.546350    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:28.546807 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:28.546453    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:29.545713 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:29.545678    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:29.545925 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:29.545847    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:30.545897 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:30.545867    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:30.546311 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:30.545868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:30.546311 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:30.545977    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:30.546311 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:30.546081    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:31.546285 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:31.546252    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:31.546631 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:31.546397    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:32.545555 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:32.545521    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:32.545715 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:32.545557    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:32.545715 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:32.545632    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:32.545816 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:32.545713    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:33.546003 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:33.545973    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:33.546405 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:33.546136    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:34.546057 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:34.546025    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:34.546519 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:34.546026    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:34.546519 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:34.546135    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:34.546519 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:34.546227    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:35.147850 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:35.147811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:35.147850 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:35.147847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs\") pod \"network-metrics-daemon-dsk4h\" (UID: \"8e94d5d6-6e3b-4c0e-b73d-e8f086218175\") " pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:35.148080 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:35.147959    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:35.148080 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:35.147971    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:35.148080 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:35.148019    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret podName:2f492c23-72ed-4bbf-899d-3d5f588ef133 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:51.148004839 +0000 UTC m=+34.154173888 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret") pod "global-pull-secret-syncer-dh44v" (UID: "2f492c23-72ed-4bbf-899d-3d5f588ef133") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:35.148080 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:35.148038    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs podName:8e94d5d6-6e3b-4c0e-b73d-e8f086218175 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:51.148028484 +0000 UTC m=+34.154197541 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs") pod "network-metrics-daemon-dsk4h" (UID: "8e94d5d6-6e3b-4c0e-b73d-e8f086218175") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:35.248440 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:35.248404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvl8c\" (UniqueName: \"kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c\") pod \"network-check-target-sjr8w\" (UID: \"ca9f20c6-5f63-4f75-87b1-00dc649acbcc\") " pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:35.248601 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:35.248554    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:33:35.248601 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:35.248575    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:33:35.248601 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:35.248587    2577 projected.go:196] Error preparing data for projected volume kube-api-access-tvl8c for pod openshift-network-diagnostics/network-check-target-sjr8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:35.248746 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:35.248644    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c podName:ca9f20c6-5f63-4f75-87b1-00dc649acbcc nodeName:}" failed. No retries permitted until 2026-07-09 09:33:51.248627005 +0000 UTC m=+34.254796060 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tvl8c" (UniqueName: "kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c") pod "network-check-target-sjr8w" (UID: "ca9f20c6-5f63-4f75-87b1-00dc649acbcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:35.549029 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:35.549000    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:35.549448 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:35.549123    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:36.545749 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:36.545715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:36.545933 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:36.545754    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:36.545933 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:36.545861    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:36.546027 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:36.545957    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:37.548509 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:37.548477    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:37.548956 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:37.548609    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:38.545531 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:38.545497    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:38.545531 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:38.545511    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:38.545731 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:38.545598    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:38.545776 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:38.545723    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:39.548676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:39.548644    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:39.549093 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:39.548752    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:40.545675 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.545501    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:40.545835 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.545539    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:40.545835 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:40.545769    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:40.545950 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:40.545865    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:40.647197 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.647114    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk" event={"ID":"25f651bd-0410-4535-8d61-adcd93c21da8","Type":"ContainerStarted","Data":"bac4b86de2921082d4a525c3c31a172a1272483125c79fc13ac50dabde7519c0"}
Jul 09 09:33:40.647197 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.647192    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk" event={"ID":"25f651bd-0410-4535-8d61-adcd93c21da8","Type":"ContainerStarted","Data":"9cd9500e6da83375b11e642a404844e92cf4ad4a16a4cc8bc607332fadfc927b"}
Jul 09 09:33:40.647898 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.647238    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk" event={"ID":"25f651bd-0410-4535-8d61-adcd93c21da8","Type":"ContainerStarted","Data":"50122efa2c9e984ebc20da7fd2ed6c01a88983729f96ecbc04bf1fc0ad3593e0"}
Jul 09 09:33:40.647898 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.647284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk" event={"ID":"25f651bd-0410-4535-8d61-adcd93c21da8","Type":"ContainerStarted","Data":"9c831b865be7781cd7289e416da79918c9fad914bd8d6d142f9f92bfa09fa60e"}
Jul 09 09:33:40.647898 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.647330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk" event={"ID":"25f651bd-0410-4535-8d61-adcd93c21da8","Type":"ContainerStarted","Data":"acf01e671303c2f1ab2f61e43ed73df7aceb8f9673fcff50d938e5bede31cf38"}
Jul 09 09:33:40.647898 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.647380    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk" event={"ID":"25f651bd-0410-4535-8d61-adcd93c21da8","Type":"ContainerStarted","Data":"d9b2913497d7a21b7de1293fe53a07b68aafad36a8f0855740d746877162ecc8"}
Jul 09 09:33:40.648713 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.648684    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-skw5s" event={"ID":"45c0bd3d-9996-4fa7-9afd-b4027c3286a0","Type":"ContainerStarted","Data":"3f754bc7716f58c7063f02a6e3ff9e7a86506957715b51b677ae4e005481d998"}
Jul 09 09:33:40.650142 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.650123    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6" event={"ID":"2a16053a-b5b2-4359-80a5-4c07ffce9851","Type":"ContainerStarted","Data":"85c75342c3e41883e7532aa1994b4d5d6804634d66269763fb1cb8818d9b26e8"}
Jul 09 09:33:40.651555 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.651524    2577 generic.go:356] "Generic (PLEG): container finished" podID="6e8230b2-ffba-4769-888d-6fbf1a2ee9c7" containerID="bce5aa69a384bf98393df8c10457f1a8cdfbd54a444bef10a30f7ea164bf5541" exitCode=0
Jul 09 09:33:40.651650 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.651594    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9gtlt" event={"ID":"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7","Type":"ContainerDied","Data":"bce5aa69a384bf98393df8c10457f1a8cdfbd54a444bef10a30f7ea164bf5541"}
Jul 09 09:33:40.653052 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.653001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4mwbb" event={"ID":"b8bc44b5-f4ab-4662-b621-acabb3056f84","Type":"ContainerStarted","Data":"520fb058d1463fe7ce459dd8314145bfff594e3abe210db7f1238cfed50005dd"}
Jul 09 09:33:40.654397 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.654378    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4nx9g" event={"ID":"7e639bf5-c03c-46fa-a7ba-7d6770be1e10","Type":"ContainerStarted","Data":"084ce9b22b74b24c319855364e80af3aec8422288d7d16a737cb01b8ff534619"}
Jul 09 09:33:40.657275 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.657256    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-spd2m" event={"ID":"f67c0d3d-90bd-44c9-89b0-754fe29f288e","Type":"ContainerStarted","Data":"6581d68160251228be5cde208c98a9150293ec6b0fa16f6eb4a7106972ea1eda"}
Jul 09 09:33:40.658695 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.658674    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4ztl5" event={"ID":"216f2721-d98c-411d-952b-dd600dc4642e","Type":"ContainerStarted","Data":"20f8a5cfa5e8ce3ffa69d47cce9ff5d022d77f8d4661a5537971cd20cd212365"}
Jul 09 09:33:40.665059 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.665022    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-105.ec2.internal" podStartSLOduration=21.665010987 podStartE2EDuration="21.665010987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:33:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:33:23.63750685 +0000 UTC m=+6.643675924" watchObservedRunningTime="2026-07-09 09:33:40.665010987 +0000 UTC m=+23.671180058"
Jul 09 09:33:40.665399 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.665379    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-skw5s" podStartSLOduration=2.526833622 podStartE2EDuration="22.66537255s" totalImagesPullingTime="20.138538928s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:33:40.664749038 +0000 UTC m=+23.670918119" watchObservedRunningTime="2026-07-09 09:33:40.66537255 +0000 UTC m=+23.671541621"
Jul 09 09:33:40.712161 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.712127    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4mwbb" podStartSLOduration=2.491350354 podStartE2EDuration="22.7121152s" totalImagesPullingTime="20.220764846s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:33:40.686100645 +0000 UTC m=+23.692269716" watchObservedRunningTime="2026-07-09 09:33:40.7121152 +0000 UTC m=+23.718284269"
Jul 09 09:33:40.729153 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.729122    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4ztl5" podStartSLOduration=2.542814617 podStartE2EDuration="22.729112234s" totalImagesPullingTime="20.186297617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:33:40.728725233 +0000 UTC m=+23.734894314" watchObservedRunningTime="2026-07-09 09:33:40.729112234 +0000 UTC m=+23.735281304"
Jul 09 09:33:40.767386 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:40.767354    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-spd2m" podStartSLOduration=2.621211617 podStartE2EDuration="22.767344267s" totalImagesPullingTime="20.14613265s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:33:40.748311567 +0000 UTC m=+23.754480629" watchObservedRunningTime="2026-07-09 09:33:40.767344267 +0000 UTC m=+23.773513338"
Jul 09 09:33:41.546343 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:41.546305    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:41.546521 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:41.546432    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:41.662389 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:41.662353    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dmr4s" event={"ID":"89c02a74-d88c-49c1-b9fc-d5c915e51d7a","Type":"ContainerStarted","Data":"dd0418c33bb6557bd63cbffc47a3997160b68d3b99827780e16a61b4aead0210"}
Jul 09 09:33:41.679083 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:41.679028    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4nx9g" podStartSLOduration=3.489695409 podStartE2EDuration="23.679008357s" totalImagesPullingTime="20.189312948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:33:40.768478449 +0000 UTC m=+23.774647532" watchObservedRunningTime="2026-07-09 09:33:41.679008357 +0000 UTC m=+24.685177431"
Jul 09 09:33:41.820071 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:41.820037    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 09:33:42.371615 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:42.371527    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T09:33:41.820056112Z","UUID":"842fb41e-0d9a-4ae0-88b3-db1c72511f87","Handler":null,"Name":"","Endpoint":""}
Jul 09 09:33:42.373172 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:42.373150    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 09:33:42.373172 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:42.373177    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 09:33:42.545463 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:42.545434    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:42.545622 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:42.545561    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:42.545849 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:42.545443    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:42.546010 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:42.545990    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:42.667319 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:42.667288    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6" event={"ID":"2a16053a-b5b2-4359-80a5-4c07ffce9851","Type":"ContainerStarted","Data":"bc77705c0ec6dfc4f07a327b4cf29f7bfd00a2b1199c2abc6869875fb4a0bebf"}
Jul 09 09:33:43.551091 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:43.550930    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:43.551262 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:43.551206    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:43.672697 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:43.672664    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk" event={"ID":"25f651bd-0410-4535-8d61-adcd93c21da8","Type":"ContainerStarted","Data":"2acd5dbf9cee076e5d33f7f080686376d615dc50e525e6f6bc03a40fc260b42a"}
Jul 09 09:33:43.674871 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:43.674844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6" event={"ID":"2a16053a-b5b2-4359-80a5-4c07ffce9851","Type":"ContainerStarted","Data":"070f9e636fa3d3b5e23fb5b4532d6b8140974b36ea286968fe0efc87d6d99184"}
Jul 09 09:33:43.693993 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:43.693953    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmpg6" podStartSLOduration=2.653568677 podStartE2EDuration="25.693941171s" totalImagesPullingTime="23.040372494s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:33:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:33:43.693751531 +0000 UTC m=+26.699920631" watchObservedRunningTime="2026-07-09 09:33:43.693941171 +0000 UTC m=+26.700110241"
Jul 09 09:33:43.694161 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:43.694139    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dmr4s" podStartSLOduration=5.5672153049999995 podStartE2EDuration="25.694132392s" totalImagesPullingTime="20.126917087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:33:41.679312901 +0000 UTC m=+24.685481973" watchObservedRunningTime="2026-07-09 09:33:43.694132392 +0000 UTC m=+26.700301463"
Jul 09 09:33:44.545868 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:44.545832    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:44.546061 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:44.545832    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:44.546061 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:44.545934    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:44.546061 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:44.546021    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:44.849115 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:44.849049    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4ztl5"
Jul 09 09:33:44.849538 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:44.849526    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4ztl5"
Jul 09 09:33:45.545810 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:45.545771    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:45.545948 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:45.545884    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:45.682141 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:45.682102    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk" event={"ID":"25f651bd-0410-4535-8d61-adcd93c21da8","Type":"ContainerStarted","Data":"5a4ad78378a187c6c2baa648fff72bb94f8314cb10738b003958a1317c23b76f"}
Jul 09 09:33:45.682378 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:45.682356    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:45.683838 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:45.683810    2577 generic.go:356] "Generic (PLEG): container finished" podID="6e8230b2-ffba-4769-888d-6fbf1a2ee9c7" containerID="d2ced9d88c39d8f6cbdd47f3ea79b7598035c15f7635d40c822027cd060f41b1" exitCode=0
Jul 09 09:33:45.683971 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:45.683895    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9gtlt" event={"ID":"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7","Type":"ContainerDied","Data":"d2ced9d88c39d8f6cbdd47f3ea79b7598035c15f7635d40c822027cd060f41b1"}
Jul 09 09:33:45.683971 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:45.683943    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4ztl5"
Jul 09 09:33:45.684659 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:45.684552    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4ztl5"
Jul 09 09:33:45.699493 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:45.699474    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:45.721462 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:45.721425    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk" podStartSLOduration=7.536798036 podStartE2EDuration="27.721412849s" totalImagesPullingTime="20.184614813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:33:45.720979354 +0000 UTC m=+28.727148439" watchObservedRunningTime="2026-07-09 09:33:45.721412849 +0000 UTC m=+28.727581917"
Jul 09 09:33:46.546091 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:46.546059    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:46.546774 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:46.546182    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:46.546774 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:46.546231    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:46.546774 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:46.546331    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:46.686362 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:46.686336    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:46.686491 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:46.686416    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:46.699952 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:46.699931    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:33:47.003168 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:47.003137    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dh44v"]
Jul 09 09:33:47.003271 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:47.003248    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:47.003361 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:47.003341    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:47.006286 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:47.006253    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sjr8w"]
Jul 09 09:33:47.006421 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:47.006352    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:47.006495 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:47.006446    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:47.006855 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:47.006830    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dsk4h"]
Jul 09 09:33:47.006946 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:47.006934    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:47.007066 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:47.007038    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:47.690021 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:47.689991    2577 generic.go:356] "Generic (PLEG): container finished" podID="6e8230b2-ffba-4769-888d-6fbf1a2ee9c7" containerID="b473bfe794068c00484c6f65afba6d469ef64d95f51b57d2d79d4dddc20f6923" exitCode=0
Jul 09 09:33:47.690446 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:47.690081    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9gtlt" event={"ID":"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7","Type":"ContainerDied","Data":"b473bfe794068c00484c6f65afba6d469ef64d95f51b57d2d79d4dddc20f6923"}
Jul 09 09:33:48.546327 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:48.546294    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:48.546327 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:48.546327    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:48.546515 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:48.546405    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:48.546515 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:48.546457    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:48.546582 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:48.546538    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:48.546644 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:48.546618    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:49.697901 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:49.697745    2577 generic.go:356] "Generic (PLEG): container finished" podID="6e8230b2-ffba-4769-888d-6fbf1a2ee9c7" containerID="94473bb3239b15491c6aa3368b4d1953e520c180e9ab57dc847be28994cfe508" exitCode=0
Jul 09 09:33:49.698282 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:49.697807    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9gtlt" event={"ID":"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7","Type":"ContainerDied","Data":"94473bb3239b15491c6aa3368b4d1953e520c180e9ab57dc847be28994cfe508"}
Jul 09 09:33:50.546348 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:50.546317    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:50.546348 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:50.546345    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:50.546565 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:50.546357    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:50.546565 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:50.546435    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:50.546565 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:50.546498    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:50.546666 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:50.546595    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:51.177342 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:51.177311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:51.177793 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:51.177354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs\") pod \"network-metrics-daemon-dsk4h\" (UID: \"8e94d5d6-6e3b-4c0e-b73d-e8f086218175\") " pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:51.177793 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:51.177473    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:51.177793 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:51.177477    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:51.177793 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:51.177520    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs podName:8e94d5d6-6e3b-4c0e-b73d-e8f086218175 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:23.177507078 +0000 UTC m=+66.183676128 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs") pod "network-metrics-daemon-dsk4h" (UID: "8e94d5d6-6e3b-4c0e-b73d-e8f086218175") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:33:51.177793 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:51.177533    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret podName:2f492c23-72ed-4bbf-899d-3d5f588ef133 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:23.177527045 +0000 UTC m=+66.183696094 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret") pod "global-pull-secret-syncer-dh44v" (UID: "2f492c23-72ed-4bbf-899d-3d5f588ef133") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:33:51.278059 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:51.277984    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvl8c\" (UniqueName: \"kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c\") pod \"network-check-target-sjr8w\" (UID: \"ca9f20c6-5f63-4f75-87b1-00dc649acbcc\") " pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:51.278199 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:51.278116    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:33:51.278199 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:51.278135    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:33:51.278199 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:51.278144    2577 projected.go:196] Error preparing data for projected volume kube-api-access-tvl8c for pod openshift-network-diagnostics/network-check-target-sjr8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:51.278199 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:51.278188    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c podName:ca9f20c6-5f63-4f75-87b1-00dc649acbcc nodeName:}" failed. No retries permitted until 2026-07-09 09:34:23.278176805 +0000 UTC m=+66.284345854 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tvl8c" (UniqueName: "kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c") pod "network-check-target-sjr8w" (UID: "ca9f20c6-5f63-4f75-87b1-00dc649acbcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:33:52.546356 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.546325    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:52.546886 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.546365    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:52.546886 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.546447    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:52.546886 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:52.546461    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dsk4h" podUID="8e94d5d6-6e3b-4c0e-b73d-e8f086218175"
Jul 09 09:33:52.546886 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:52.546533    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh44v" podUID="2f492c23-72ed-4bbf-899d-3d5f588ef133"
Jul 09 09:33:52.546886 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:52.546579    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjr8w" podUID="ca9f20c6-5f63-4f75-87b1-00dc649acbcc"
Jul 09 09:33:52.789836 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.789788    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-105.ec2.internal" event="NodeReady"
Jul 09 09:33:52.790001 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.789920    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 09:33:52.832962 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.832922    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59df9f699f-s9nfn"]
Jul 09 09:33:52.836667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.836643    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.837624 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.837596    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9ztxj"]
Jul 09 09:33:52.841547 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.841526    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"]
Jul 09 09:33:52.841682 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.841658    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9ztxj"
Jul 09 09:33:52.844740 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.844724    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:33:52.846322 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.846302    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 09 09:33:52.846408 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.846307    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 09 09:33:52.846472 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.846384    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 09 09:33:52.847371 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.847354    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 09 09:33:52.848900 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.848882    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 09 09:33:52.848989 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.848885    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 09:33:52.849583 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.849560    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 09 09:33:52.850097 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.849924    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-f8ppn\""
Jul 09 09:33:52.850097 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.849927    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pjjd7\""
Jul 09 09:33:52.850097 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.850029    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-69h7n\""
Jul 09 09:33:52.851033 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.850887    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 09:33:52.854269 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.852319    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 09 09:33:52.858343 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.858320    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 09 09:33:52.862822 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.862783    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59df9f699f-s9nfn"]
Jul 09 09:33:52.863654 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.863635    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-c7f6c5c4f-cjdpd"]
Jul 09 09:33:52.868219 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.868198    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-99ggf"]
Jul 09 09:33:52.869130 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.869113    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:52.871410 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.871391    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"]
Jul 09 09:33:52.871757 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.871737    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:52.872322 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.872308    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4rqw2\""
Jul 09 09:33:52.872519 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.872499    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 09 09:33:52.872790 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.872774    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 09 09:33:52.873019 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.873002    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 09 09:33:52.873067 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.873037    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 09:33:52.873189 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.873174    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 09 09:33:52.873362 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.873329    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 09:33:52.874545 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.874529    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"]
Jul 09 09:33:52.874677 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.874660    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:33:52.875123 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.875103    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 09:33:52.875642 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.875625    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 09:33:52.876687 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.876669    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cfzpw\""
Jul 09 09:33:52.877156 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.877135    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 09 09:33:52.877615 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.877551    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-d5888"]
Jul 09 09:33:52.877714 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.877662    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"
Jul 09 09:33:52.878183 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.878163    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tbkpb\""
Jul 09 09:33:52.878275 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.878261    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 09 09:33:52.879348 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.879330    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 09 09:33:52.880720 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.880702    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"]
Jul 09 09:33:52.880837 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.880762    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:52.882331 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.882316    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 09 09:33:52.882437 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.882350    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 09 09:33:52.882598 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.882582    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 09 09:33:52.882675 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.882610    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 09 09:33:52.883684 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.883665    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-ms4r4"]
Jul 09 09:33:52.883822 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.883786    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"
Jul 09 09:33:52.886681 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.886663    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9ztxj"]
Jul 09 09:33:52.886681 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.886695    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"]
Jul 09 09:33:52.887517 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.886765    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 09 09:33:52.887517 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.886854    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:52.887517 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.886944    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7ng27\""
Jul 09 09:33:52.887811 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.887766    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-2hhr8\""
Jul 09 09:33:52.890084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.887790    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 09:33:52.890084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.887867    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 09 09:33:52.890084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.888177    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rrl6c\""
Jul 09 09:33:52.890084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.888865    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 09 09:33:52.890084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.889498    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7365b00b-2bda-4d1e-8b42-d83097184402-ca-trust-extracted\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.890084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.889520    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 09 09:33:52.890084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.889526    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7365b00b-2bda-4d1e-8b42-d83097184402-registry-certificates\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.890084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.889563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh4w8\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-kube-api-access-vh4w8\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.890084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.889592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7365b00b-2bda-4d1e-8b42-d83097184402-installation-pull-secrets\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.890084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.889635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-bound-sa-token\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.890084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.889666    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.890084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.889707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7365b00b-2bda-4d1e-8b42-d83097184402-trusted-ca\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.890084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.889777    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7365b00b-2bda-4d1e-8b42-d83097184402-image-registry-private-configuration\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.890859 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.890218    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 09 09:33:52.890859 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.890401    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 09 09:33:52.890859 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.890686    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 09:33:52.891724 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.891023    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 09 09:33:52.891724 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.891175    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 09 09:33:52.891724 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.891340    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-d5888"]
Jul 09 09:33:52.891724 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.891420    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 09 09:33:52.891724 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.891549    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 09 09:33:52.891984 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.891760    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ss9db\""
Jul 09 09:33:52.892438 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.892326    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"]
Jul 09 09:33:52.895326 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.895222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"]
Jul 09 09:33:52.897671 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.897639    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"]
Jul 09 09:33:52.901221 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.901205    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"]
Jul 09 09:33:52.905032 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.905013    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 09 09:33:52.905378 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.905078    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 09 09:33:52.909792 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.909658    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v9drg"]
Jul 09 09:33:52.910751 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.910733    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:33:52.913029 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.913010    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-6fsgb"]
Jul 09 09:33:52.913128 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.913114    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v9drg"
Jul 09 09:33:52.913666 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.913646    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 09 09:33:52.913748 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.913693    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-d89w5\""
Jul 09 09:33:52.913924 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.913904    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 09 09:33:52.916090 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.916075    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-ms4r4"]
Jul 09 09:33:52.916364 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.916347    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6fsgb"
Jul 09 09:33:52.916446 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.916401    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 09:33:52.916853 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.916836    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c7f6c5c4f-cjdpd"]
Jul 09 09:33:52.917992 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.917973    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 09:33:52.918395 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.918379    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 09:33:52.919000 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.918980    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-g7jnx\""
Jul 09 09:33:52.921170 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.921149    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v9drg"]
Jul 09 09:33:52.922041 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.922026    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 09:33:52.922981 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.922962    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-vhngs\""
Jul 09 09:33:52.923372 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.923272    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 09:33:52.924451 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.924435    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"]
Jul 09 09:33:52.951958 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.951940    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-99ggf"]
Jul 09 09:33:52.952810 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.952774    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-6fsgb"]
Jul 09 09:33:52.991243 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991224    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38d5ce80-3b12-4729-9e14-510a84a1e8d0-trusted-ca\") pod \"console-operator-68b757fb67-ms4r4\" (UID: \"38d5ce80-3b12-4729-9e14-510a84a1e8d0\") " pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:52.991469 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991435    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c0f6b837-c192-44ae-8c7f-02ee427acafc-snapshots\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:52.991574 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh46x\" (UniqueName: \"kubernetes.io/projected/38d5ce80-3b12-4729-9e14-510a84a1e8d0-kube-api-access-bh46x\") pod \"console-operator-68b757fb67-ms4r4\" (UID: \"38d5ce80-3b12-4729-9e14-510a84a1e8d0\") " pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:52.991574 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991508    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/960ec500-4be8-4e41-8f4b-cd8f20938c08-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-gcwjv\" (UID: \"960ec500-4be8-4e41-8f4b-cd8f20938c08\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:33:52.991688 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:52.991688 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7365b00b-2bda-4d1e-8b42-d83097184402-ca-trust-extracted\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.991688 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7365b00b-2bda-4d1e-8b42-d83097184402-registry-certificates\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.991821 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991694    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh4w8\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-kube-api-access-vh4w8\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.991821 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991715    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82w5d\" (UniqueName: \"kubernetes.io/projected/fbaaef5e-87e5-4dae-a367-b10fc0d62031-kube-api-access-82w5d\") pod \"cluster-samples-operator-56fcf86bf-sv5mc\" (UID: \"fbaaef5e-87e5-4dae-a367-b10fc0d62031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:33:52.991884 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991861    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-gcwjv\" (UID: \"960ec500-4be8-4e41-8f4b-cd8f20938c08\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:33:52.991938 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxm2p\" (UniqueName: \"kubernetes.io/projected/0576329a-dc3f-459e-a951-f2b0aed69679-kube-api-access-nxm2p\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:52.991938 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991930    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7365b00b-2bda-4d1e-8b42-d83097184402-installation-pull-secrets\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.992029 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft28v\" (UniqueName: \"kubernetes.io/projected/780a8630-14fc-41e6-801c-a8271ae77ddc-kube-api-access-ft28v\") pod \"volume-data-source-validator-5d456fdbcd-9ztxj\" (UID: \"780a8630-14fc-41e6-801c-a8271ae77ddc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9ztxj"
Jul 09 09:33:52.992029 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c0f6b837-c192-44ae-8c7f-02ee427acafc-serving-cert\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:52.992029 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992011    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/38bfa326-e79b-447e-ae2b-c31b088b6249-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:33:52.992029 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.991986    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7365b00b-2bda-4d1e-8b42-d83097184402-ca-trust-extracted\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.992222 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992037    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38d5ce80-3b12-4729-9e14-510a84a1e8d0-config\") pod \"console-operator-68b757fb67-ms4r4\" (UID: \"38d5ce80-3b12-4729-9e14-510a84a1e8d0\") " pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:52.992222 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992078    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-bound-sa-token\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.992222 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7365b00b-2bda-4d1e-8b42-d83097184402-trusted-ca\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.992222 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b57904de-b3f2-4b07-9098-9d1fb25f91a5-cert\") pod \"ingress-canary-v9drg\" (UID: \"b57904de-b3f2-4b07-9098-9d1fb25f91a5\") " pod="openshift-ingress-canary/ingress-canary-v9drg"
Jul 09 09:33:52.992405 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt4g4\" (UniqueName: \"kubernetes.io/projected/412bfb04-3abc-46f3-b9d3-76c904b79c2a-kube-api-access-pt4g4\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:52.992405 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992278    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7365b00b-2bda-4d1e-8b42-d83097184402-registry-certificates\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.992405 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:33:52.992405 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:52.992405 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7365b00b-2bda-4d1e-8b42-d83097184402-image-registry-private-configuration\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.992405 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992358    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-default-certificate\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:52.992405 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24c5f723-d578-4374-8f99-2a7fa1f38727-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-f2zvs\" (UID: \"24c5f723-d578-4374-8f99-2a7fa1f38727\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"
Jul 09 09:33:52.992722 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5ac7be9-438f-473e-814d-c2a998a389f8-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-g6r9d\" (UID: \"d5ac7be9-438f-473e-814d-c2a998a389f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"
Jul 09 09:33:52.992722 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbd65\" (UniqueName: \"kubernetes.io/projected/24c5f723-d578-4374-8f99-2a7fa1f38727-kube-api-access-sbd65\") pod \"service-ca-operator-5d46d9ffd4-f2zvs\" (UID: \"24c5f723-d578-4374-8f99-2a7fa1f38727\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"
Jul 09 09:33:52.992722 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38d5ce80-3b12-4729-9e14-510a84a1e8d0-serving-cert\") pod \"console-operator-68b757fb67-ms4r4\" (UID: \"38d5ce80-3b12-4729-9e14-510a84a1e8d0\") " pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:52.992722 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992521    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5ac7be9-438f-473e-814d-c2a998a389f8-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-g6r9d\" (UID: \"d5ac7be9-438f-473e-814d-c2a998a389f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"
Jul 09 09:33:52.992722 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgtq7\" (UniqueName: \"kubernetes.io/projected/d5ac7be9-438f-473e-814d-c2a998a389f8-kube-api-access-sgtq7\") pod \"kube-storage-version-migrator-operator-846bf668f7-g6r9d\" (UID: \"d5ac7be9-438f-473e-814d-c2a998a389f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"
Jul 09 09:33:52.992722 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992576    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/412bfb04-3abc-46f3-b9d3-76c904b79c2a-config-volume\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:52.992722 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992616    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8dtq\" (UniqueName: \"kubernetes.io/projected/38bfa326-e79b-447e-ae2b-c31b088b6249-kube-api-access-j8dtq\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:33:52.992722 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992651    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gslj\" (UniqueName: \"kubernetes.io/projected/35d84c6e-4025-4ff5-83c5-92f505e2f9ca-kube-api-access-4gslj\") pod \"network-check-source-989f8b8dc-6fsgb\" (UID: \"35d84c6e-4025-4ff5-83c5-92f505e2f9ca\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6fsgb"
Jul 09 09:33:52.992722 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992697    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-stats-auth\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:52.992722 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f6b837-c192-44ae-8c7f-02ee427acafc-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:52.993188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992785    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0f6b837-c192-44ae-8c7f-02ee427acafc-tmp\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:52.993188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992831    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6zjm\" (UniqueName: \"kubernetes.io/projected/c0f6b837-c192-44ae-8c7f-02ee427acafc-kube-api-access-m6zjm\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:52.993188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992857    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.993188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992875    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:52.993188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.992896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-sv5mc\" (UID: \"fbaaef5e-87e5-4dae-a367-b10fc0d62031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:33:52.993188 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:52.992973    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 09:33:52.993188 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:52.992990    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59df9f699f-s9nfn: secret "image-registry-tls" not found
Jul 09 09:33:52.993188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.993008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f6b837-c192-44ae-8c7f-02ee427acafc-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:52.993188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.993046    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7365b00b-2bda-4d1e-8b42-d83097184402-trusted-ca\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.993188 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:52.993053    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls podName:7365b00b-2bda-4d1e-8b42-d83097184402 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:53.493034454 +0000 UTC m=+36.499203526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls") pod "image-registry-59df9f699f-s9nfn" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402") : secret "image-registry-tls" not found
Jul 09 09:33:52.993188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.993091    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndw6d\" (UniqueName: \"kubernetes.io/projected/b57904de-b3f2-4b07-9098-9d1fb25f91a5-kube-api-access-ndw6d\") pod \"ingress-canary-v9drg\" (UID: \"b57904de-b3f2-4b07-9098-9d1fb25f91a5\") " pod="openshift-ingress-canary/ingress-canary-v9drg"
Jul 09 09:33:52.993188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.993111    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/24c5f723-d578-4374-8f99-2a7fa1f38727-config\") pod \"service-ca-operator-5d46d9ffd4-f2zvs\" (UID: \"24c5f723-d578-4374-8f99-2a7fa1f38727\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"
Jul 09 09:33:52.993188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.993149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/412bfb04-3abc-46f3-b9d3-76c904b79c2a-tmp-dir\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:52.996883 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.996855    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7365b00b-2bda-4d1e-8b42-d83097184402-image-registry-private-configuration\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:52.996966 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:52.996861    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7365b00b-2bda-4d1e-8b42-d83097184402-installation-pull-secrets\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:53.030638 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.030613    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh4w8\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-kube-api-access-vh4w8\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:53.031537 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.031515    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-bound-sa-token\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:53.094448 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.094388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b57904de-b3f2-4b07-9098-9d1fb25f91a5-cert\") pod \"ingress-canary-v9drg\" (UID: \"b57904de-b3f2-4b07-9098-9d1fb25f91a5\") " pod="openshift-ingress-canary/ingress-canary-v9drg"
Jul 09 09:33:53.094563 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.094448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt4g4\" (UniqueName: \"kubernetes.io/projected/412bfb04-3abc-46f3-b9d3-76c904b79c2a-kube-api-access-pt4g4\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:53.094563 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.094478    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:33:53.094563 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.094503    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:53.094563 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.094521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-default-certificate\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:53.094773 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.094569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24c5f723-d578-4374-8f99-2a7fa1f38727-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-f2zvs\" (UID: \"24c5f723-d578-4374-8f99-2a7fa1f38727\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"
Jul 09 09:33:53.094773 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.094612    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 09 09:33:53.094773 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.094673    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs podName:0576329a-dc3f-459e-a951-f2b0aed69679 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:53.594654657 +0000 UTC m=+36.600823721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs") pod "router-default-c7f6c5c4f-cjdpd" (UID: "0576329a-dc3f-459e-a951-f2b0aed69679") : secret "router-metrics-certs-default" not found
Jul 09 09:33:53.094974 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.094893    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 09:33:53.094974 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.094935    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls podName:38bfa326-e79b-447e-ae2b-c31b088b6249 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:53.594920891 +0000 UTC m=+36.601089946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-msxxf" (UID: "38bfa326-e79b-447e-ae2b-c31b088b6249") : secret "cluster-monitoring-operator-tls" not found
Jul 09 09:33:53.094974 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.094614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5ac7be9-438f-473e-814d-c2a998a389f8-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-g6r9d\" (UID: \"d5ac7be9-438f-473e-814d-c2a998a389f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"
Jul 09 09:33:53.095128 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.094975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbd65\" (UniqueName: \"kubernetes.io/projected/24c5f723-d578-4374-8f99-2a7fa1f38727-kube-api-access-sbd65\") pod \"service-ca-operator-5d46d9ffd4-f2zvs\" (UID: \"24c5f723-d578-4374-8f99-2a7fa1f38727\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"
Jul 09 09:33:53.095128 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095006    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38d5ce80-3b12-4729-9e14-510a84a1e8d0-serving-cert\") pod \"console-operator-68b757fb67-ms4r4\" (UID: \"38d5ce80-3b12-4729-9e14-510a84a1e8d0\") " pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:53.095128 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5ac7be9-438f-473e-814d-c2a998a389f8-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-g6r9d\" (UID: \"d5ac7be9-438f-473e-814d-c2a998a389f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"
Jul 09 09:33:53.095128 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgtq7\" (UniqueName: \"kubernetes.io/projected/d5ac7be9-438f-473e-814d-c2a998a389f8-kube-api-access-sgtq7\") pod \"kube-storage-version-migrator-operator-846bf668f7-g6r9d\" (UID: \"d5ac7be9-438f-473e-814d-c2a998a389f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"
Jul 09 09:33:53.095128 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/412bfb04-3abc-46f3-b9d3-76c904b79c2a-config-volume\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:53.095128 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095112    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8dtq\" (UniqueName: \"kubernetes.io/projected/38bfa326-e79b-447e-ae2b-c31b088b6249-kube-api-access-j8dtq\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:33:53.095128 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095118    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5ac7be9-438f-473e-814d-c2a998a389f8-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-g6r9d\" (UID: \"d5ac7be9-438f-473e-814d-c2a998a389f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"
Jul 09 09:33:53.095481 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095138    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gslj\" (UniqueName: \"kubernetes.io/projected/35d84c6e-4025-4ff5-83c5-92f505e2f9ca-kube-api-access-4gslj\") pod \"network-check-source-989f8b8dc-6fsgb\" (UID: \"35d84c6e-4025-4ff5-83c5-92f505e2f9ca\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6fsgb"
Jul 09 09:33:53.095481 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-stats-auth\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:53.095481 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f6b837-c192-44ae-8c7f-02ee427acafc-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:53.095481 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0f6b837-c192-44ae-8c7f-02ee427acafc-tmp\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:53.095481 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095259    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6zjm\" (UniqueName: \"kubernetes.io/projected/c0f6b837-c192-44ae-8c7f-02ee427acafc-kube-api-access-m6zjm\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:53.095481 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095304    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:53.095481 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-sv5mc\" (UID: \"fbaaef5e-87e5-4dae-a367-b10fc0d62031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:33:53.095481 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f6b837-c192-44ae-8c7f-02ee427acafc-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:53.095481 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndw6d\" (UniqueName: \"kubernetes.io/projected/b57904de-b3f2-4b07-9098-9d1fb25f91a5-kube-api-access-ndw6d\") pod \"ingress-canary-v9drg\" (UID: \"b57904de-b3f2-4b07-9098-9d1fb25f91a5\") " pod="openshift-ingress-canary/ingress-canary-v9drg"
Jul 09 09:33:53.095481 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/24c5f723-d578-4374-8f99-2a7fa1f38727-config\") pod \"service-ca-operator-5d46d9ffd4-f2zvs\" (UID: \"24c5f723-d578-4374-8f99-2a7fa1f38727\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"
Jul 09 09:33:53.095481 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/412bfb04-3abc-46f3-b9d3-76c904b79c2a-tmp-dir\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:53.095481 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38d5ce80-3b12-4729-9e14-510a84a1e8d0-trusted-ca\") pod \"console-operator-68b757fb67-ms4r4\" (UID: \"38d5ce80-3b12-4729-9e14-510a84a1e8d0\") " pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:53.096068 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.095490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c0f6b837-c192-44ae-8c7f-02ee427acafc-snapshots\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:53.096068 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.095641    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 09:33:53.096068 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.095703    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls podName:fbaaef5e-87e5-4dae-a367-b10fc0d62031 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:53.595687112 +0000 UTC m=+36.601856174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-sv5mc" (UID: "fbaaef5e-87e5-4dae-a367-b10fc0d62031") : secret "samples-operator-tls" not found
Jul 09 09:33:53.096068 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096003    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/412bfb04-3abc-46f3-b9d3-76c904b79c2a-config-volume\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:53.096068 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096048    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh46x\" (UniqueName: \"kubernetes.io/projected/38d5ce80-3b12-4729-9e14-510a84a1e8d0-kube-api-access-bh46x\") pod \"console-operator-68b757fb67-ms4r4\" (UID: \"38d5ce80-3b12-4729-9e14-510a84a1e8d0\") " pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:53.096326 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/960ec500-4be8-4e41-8f4b-cd8f20938c08-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-gcwjv\" (UID: \"960ec500-4be8-4e41-8f4b-cd8f20938c08\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:33:53.096326 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096105    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:53.096326 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096150    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82w5d\" (UniqueName: \"kubernetes.io/projected/fbaaef5e-87e5-4dae-a367-b10fc0d62031-kube-api-access-82w5d\") pod \"cluster-samples-operator-56fcf86bf-sv5mc\" (UID: \"fbaaef5e-87e5-4dae-a367-b10fc0d62031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:33:53.096326 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096198    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-gcwjv\" (UID: \"960ec500-4be8-4e41-8f4b-cd8f20938c08\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:33:53.096326 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096232    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxm2p\" (UniqueName: \"kubernetes.io/projected/0576329a-dc3f-459e-a951-f2b0aed69679-kube-api-access-nxm2p\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:53.096326 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft28v\" (UniqueName: \"kubernetes.io/projected/780a8630-14fc-41e6-801c-a8271ae77ddc-kube-api-access-ft28v\") pod \"volume-data-source-validator-5d456fdbcd-9ztxj\" (UID: \"780a8630-14fc-41e6-801c-a8271ae77ddc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9ztxj"
Jul 09 09:33:53.096326 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c0f6b837-c192-44ae-8c7f-02ee427acafc-serving-cert\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:53.096326 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/38bfa326-e79b-447e-ae2b-c31b088b6249-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:33:53.096692 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38d5ce80-3b12-4729-9e14-510a84a1e8d0-config\") pod \"console-operator-68b757fb67-ms4r4\" (UID: \"38d5ce80-3b12-4729-9e14-510a84a1e8d0\") " pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:53.096692 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096348    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f6b837-c192-44ae-8c7f-02ee427acafc-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:53.096692 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096479    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c0f6b837-c192-44ae-8c7f-02ee427acafc-snapshots\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:53.096692 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.096570    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 09:33:53.096692 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.096622    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert podName:960ec500-4be8-4e41-8f4b-cd8f20938c08 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:53.596608321 +0000 UTC m=+36.602777384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-gcwjv" (UID: "960ec500-4be8-4e41-8f4b-cd8f20938c08") : secret "networking-console-plugin-cert" not found
Jul 09 09:33:53.096692 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.096623    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0f6b837-c192-44ae-8c7f-02ee427acafc-tmp\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:53.097000 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.096932    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle podName:0576329a-dc3f-459e-a951-f2b0aed69679 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:53.596917733 +0000 UTC m=+36.603086801 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle") pod "router-default-c7f6c5c4f-cjdpd" (UID: "0576329a-dc3f-459e-a951-f2b0aed69679") : configmap references non-existent config key: service-ca.crt
Jul 09 09:33:53.097272 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.097251    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 09:33:53.097359 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.097306    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls podName:412bfb04-3abc-46f3-b9d3-76c904b79c2a nodeName:}" failed. No retries permitted until 2026-07-09 09:33:53.597290309 +0000 UTC m=+36.603459393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls") pod "dns-default-99ggf" (UID: "412bfb04-3abc-46f3-b9d3-76c904b79c2a") : secret "dns-default-metrics-tls" not found
Jul 09 09:33:53.097359 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.097258    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b57904de-b3f2-4b07-9098-9d1fb25f91a5-cert\") pod \"ingress-canary-v9drg\" (UID: \"b57904de-b3f2-4b07-9098-9d1fb25f91a5\") " pod="openshift-ingress-canary/ingress-canary-v9drg"
Jul 09 09:33:53.098446 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.097489    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/24c5f723-d578-4374-8f99-2a7fa1f38727-config\") pod \"service-ca-operator-5d46d9ffd4-f2zvs\" (UID: \"24c5f723-d578-4374-8f99-2a7fa1f38727\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"
Jul 09 09:33:53.098446 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.097627    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/412bfb04-3abc-46f3-b9d3-76c904b79c2a-tmp-dir\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:53.098446 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.097749    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/38bfa326-e79b-447e-ae2b-c31b088b6249-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:33:53.098446 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.098349    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38d5ce80-3b12-4729-9e14-510a84a1e8d0-config\") pod \"console-operator-68b757fb67-ms4r4\" (UID: \"38d5ce80-3b12-4729-9e14-510a84a1e8d0\") " pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:53.098446 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.098404    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f6b837-c192-44ae-8c7f-02ee427acafc-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:53.098870 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.098826    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38d5ce80-3b12-4729-9e14-510a84a1e8d0-trusted-ca\") pod \"console-operator-68b757fb67-ms4r4\" (UID: \"38d5ce80-3b12-4729-9e14-510a84a1e8d0\") " pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:53.098870 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.098860    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/960ec500-4be8-4e41-8f4b-cd8f20938c08-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-gcwjv\" (UID: \"960ec500-4be8-4e41-8f4b-cd8f20938c08\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:33:53.099384 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.099365    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c0f6b837-c192-44ae-8c7f-02ee427acafc-serving-cert\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:53.099527 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.099508    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-default-certificate\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:53.099598 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.099534    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5ac7be9-438f-473e-814d-c2a998a389f8-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-g6r9d\" (UID: \"d5ac7be9-438f-473e-814d-c2a998a389f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"
Jul 09 09:33:53.099727 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.099710    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24c5f723-d578-4374-8f99-2a7fa1f38727-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-f2zvs\" (UID: \"24c5f723-d578-4374-8f99-2a7fa1f38727\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"
Jul 09 09:33:53.100098 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.100080    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-stats-auth\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:53.100740 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.100719    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38d5ce80-3b12-4729-9e14-510a84a1e8d0-serving-cert\") pod \"console-operator-68b757fb67-ms4r4\" (UID: \"38d5ce80-3b12-4729-9e14-510a84a1e8d0\") " pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:53.110748 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.110702    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndw6d\" (UniqueName: \"kubernetes.io/projected/b57904de-b3f2-4b07-9098-9d1fb25f91a5-kube-api-access-ndw6d\") pod \"ingress-canary-v9drg\" (UID: \"b57904de-b3f2-4b07-9098-9d1fb25f91a5\") " pod="openshift-ingress-canary/ingress-canary-v9drg"
Jul 09 09:33:53.111227 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.111183    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt4g4\" (UniqueName: \"kubernetes.io/projected/412bfb04-3abc-46f3-b9d3-76c904b79c2a-kube-api-access-pt4g4\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:53.111460 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.111427    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh46x\" (UniqueName: \"kubernetes.io/projected/38d5ce80-3b12-4729-9e14-510a84a1e8d0-kube-api-access-bh46x\") pod \"console-operator-68b757fb67-ms4r4\" (UID: \"38d5ce80-3b12-4729-9e14-510a84a1e8d0\") " pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:53.111566 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.111500    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgtq7\" (UniqueName: \"kubernetes.io/projected/d5ac7be9-438f-473e-814d-c2a998a389f8-kube-api-access-sgtq7\") pod \"kube-storage-version-migrator-operator-846bf668f7-g6r9d\" (UID: \"d5ac7be9-438f-473e-814d-c2a998a389f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"
Jul 09 09:33:53.112249 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.112230    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gslj\" (UniqueName: \"kubernetes.io/projected/35d84c6e-4025-4ff5-83c5-92f505e2f9ca-kube-api-access-4gslj\") pod \"network-check-source-989f8b8dc-6fsgb\" (UID: \"35d84c6e-4025-4ff5-83c5-92f505e2f9ca\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6fsgb"
Jul 09 09:33:53.118905 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.116268    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8dtq\" (UniqueName: \"kubernetes.io/projected/38bfa326-e79b-447e-ae2b-c31b088b6249-kube-api-access-j8dtq\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:33:53.118905 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.116320    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82w5d\" (UniqueName: \"kubernetes.io/projected/fbaaef5e-87e5-4dae-a367-b10fc0d62031-kube-api-access-82w5d\") pod \"cluster-samples-operator-56fcf86bf-sv5mc\" (UID: \"fbaaef5e-87e5-4dae-a367-b10fc0d62031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:33:53.118905 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.117492    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6zjm\" (UniqueName: \"kubernetes.io/projected/c0f6b837-c192-44ae-8c7f-02ee427acafc-kube-api-access-m6zjm\") pod \"insights-operator-5c847dcfc4-d5888\" (UID: \"c0f6b837-c192-44ae-8c7f-02ee427acafc\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:53.118905 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.118241    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbd65\" (UniqueName: \"kubernetes.io/projected/24c5f723-d578-4374-8f99-2a7fa1f38727-kube-api-access-sbd65\") pod \"service-ca-operator-5d46d9ffd4-f2zvs\" (UID: \"24c5f723-d578-4374-8f99-2a7fa1f38727\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"
Jul 09 09:33:53.118905 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.118424    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxm2p\" (UniqueName: \"kubernetes.io/projected/0576329a-dc3f-459e-a951-f2b0aed69679-kube-api-access-nxm2p\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:53.121187 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.121157    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft28v\" (UniqueName: \"kubernetes.io/projected/780a8630-14fc-41e6-801c-a8271ae77ddc-kube-api-access-ft28v\") pod \"volume-data-source-validator-5d456fdbcd-9ztxj\" (UID: \"780a8630-14fc-41e6-801c-a8271ae77ddc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9ztxj"
Jul 09 09:33:53.158515 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.158489    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9ztxj"
Jul 09 09:33:53.165105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.165073    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:33:53.212787 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.212761    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"
Jul 09 09:33:53.219613 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.219596    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-d5888"
Jul 09 09:33:53.228350 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.228331    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"
Jul 09 09:33:53.234892 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.234876    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:33:53.247871 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.247850    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v9drg"
Jul 09 09:33:53.254558 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.254538    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6fsgb"
Jul 09 09:33:53.257885 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.257860    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:33:53.500119 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.500083    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:53.500265 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.500232    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 09:33:53.500265 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.500249    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59df9f699f-s9nfn: secret "image-registry-tls" not found
Jul 09 09:33:53.500386 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.500302    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls podName:7365b00b-2bda-4d1e-8b42-d83097184402 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:54.500286136 +0000 UTC m=+37.506455189 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls") pod "image-registry-59df9f699f-s9nfn" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402") : secret "image-registry-tls" not found
Jul 09 09:33:53.600906 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.600878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:33:53.601489 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.600916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:53.601489 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.600983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:53.601489 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.601003    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-sv5mc\" (UID: \"fbaaef5e-87e5-4dae-a367-b10fc0d62031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:33:53.601489 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.601031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:53.601489 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.601041    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 09:33:53.601489 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.601106    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 09 09:33:53.601489 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.601115    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 09:33:53.601489 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.601114    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls podName:38bfa326-e79b-447e-ae2b-c31b088b6249 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:54.601092301 +0000 UTC m=+37.607261366 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-msxxf" (UID: "38bfa326-e79b-447e-ae2b-c31b088b6249") : secret "cluster-monitoring-operator-tls" not found
Jul 09 09:33:53.601489 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.601156    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 09:33:53.601489 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.601162    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle podName:0576329a-dc3f-459e-a951-f2b0aed69679 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:54.601147468 +0000 UTC m=+37.607316532 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle") pod "router-default-c7f6c5c4f-cjdpd" (UID: "0576329a-dc3f-459e-a951-f2b0aed69679") : configmap references non-existent config key: service-ca.crt
Jul 09 09:33:53.601489 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:53.601262    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-gcwjv\" (UID: \"960ec500-4be8-4e41-8f4b-cd8f20938c08\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:33:53.601489 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.601319    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls podName:412bfb04-3abc-46f3-b9d3-76c904b79c2a nodeName:}" failed. No retries permitted until 2026-07-09 09:33:54.601302499 +0000 UTC m=+37.607471563 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls") pod "dns-default-99ggf" (UID: "412bfb04-3abc-46f3-b9d3-76c904b79c2a") : secret "dns-default-metrics-tls" not found
Jul 09 09:33:53.601489 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.601326    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 09:33:53.601895 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.601339    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs podName:0576329a-dc3f-459e-a951-f2b0aed69679 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:54.601333382 +0000 UTC m=+37.607502431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs") pod "router-default-c7f6c5c4f-cjdpd" (UID: "0576329a-dc3f-459e-a951-f2b0aed69679") : secret "router-metrics-certs-default" not found
Jul 09 09:33:53.601895 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.601363    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls podName:fbaaef5e-87e5-4dae-a367-b10fc0d62031 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:54.601346042 +0000 UTC m=+37.607515091 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-sv5mc" (UID: "fbaaef5e-87e5-4dae-a367-b10fc0d62031") : secret "samples-operator-tls" not found
Jul 09 09:33:53.601895 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:53.601378    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert podName:960ec500-4be8-4e41-8f4b-cd8f20938c08 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:54.601372256 +0000 UTC m=+37.607541306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-gcwjv" (UID: "960ec500-4be8-4e41-8f4b-cd8f20938c08") : secret "networking-console-plugin-cert" not found
Jul 09 09:33:54.426126 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.425812    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-d5888"]
Jul 09 09:33:54.434474 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.434407    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-6fsgb"]
Jul 09 09:33:54.436093 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.435419    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9ztxj"]
Jul 09 09:33:54.459108 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.459075    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d"]
Jul 09 09:33:54.460666 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.460646    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-ms4r4"]
Jul 09 09:33:54.461499 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.461481    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs"]
Jul 09 09:33:54.470322 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.470304    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v9drg"]
Jul 09 09:33:54.473811 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:54.473773    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb57904de_b3f2_4b07_9098_9d1fb25f91a5.slice/crio-b4c042856341955952074c01ff7ee76844dc9354e64674ff5f7c57f6dd48354a WatchSource:0}: Error finding container b4c042856341955952074c01ff7ee76844dc9354e64674ff5f7c57f6dd48354a: Status 404 returned error can't find the container with id b4c042856341955952074c01ff7ee76844dc9354e64674ff5f7c57f6dd48354a
Jul 09 09:33:54.475232 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:54.475004    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38d5ce80_3b12_4729_9e14_510a84a1e8d0.slice/crio-63c973dbb56188f596631f98481e31e44159aede08b6757c6b456d303e86b2c2 WatchSource:0}: Error finding container 63c973dbb56188f596631f98481e31e44159aede08b6757c6b456d303e86b2c2: Status 404 returned error can't find the container with id 63c973dbb56188f596631f98481e31e44159aede08b6757c6b456d303e86b2c2
Jul 09 09:33:54.475982 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:54.475937    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5ac7be9_438f_473e_814d_c2a998a389f8.slice/crio-1d6435bd497825c76de9effc0a30b3ca5b3381c75d432d3868bcebf6315134ed WatchSource:0}: Error finding container 1d6435bd497825c76de9effc0a30b3ca5b3381c75d432d3868bcebf6315134ed: Status 404 returned error can't find the container with id 1d6435bd497825c76de9effc0a30b3ca5b3381c75d432d3868bcebf6315134ed
Jul 09 09:33:54.476382 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:33:54.476348    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c5f723_d578_4374_8f99_2a7fa1f38727.slice/crio-50aeb80423a0c61feb636b782c850f5c6c6be66dca8fda1e1b1e2babac880bd2 WatchSource:0}: Error finding container 50aeb80423a0c61feb636b782c850f5c6c6be66dca8fda1e1b1e2babac880bd2: Status 404 returned error can't find the container with id 50aeb80423a0c61feb636b782c850f5c6c6be66dca8fda1e1b1e2babac880bd2
Jul 09 09:33:54.509766 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.509741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:54.509897 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.509882    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 09:33:54.509945 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.509900    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59df9f699f-s9nfn: secret "image-registry-tls" not found
Jul 09 09:33:54.509983 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.509945    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls podName:7365b00b-2bda-4d1e-8b42-d83097184402 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:56.509929603 +0000 UTC m=+39.516098671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls") pod "image-registry-59df9f699f-s9nfn" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402") : secret "image-registry-tls" not found
Jul 09 09:33:54.545593 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.545572    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:33:54.545682 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.545665    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:33:54.545722 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.545695    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:33:54.549492 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.549475    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 09:33:54.550018 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.550001    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 09:33:54.550068 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.550009    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r7trk\""
Jul 09 09:33:54.550068 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.550009    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ntnhc\""
Jul 09 09:33:54.610836 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.610816    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:54.611269 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.610849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-sv5mc\" (UID: \"fbaaef5e-87e5-4dae-a367-b10fc0d62031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:33:54.611269 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.610897    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:54.611269 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.610940    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 09:33:54.611269 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.610958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-gcwjv\" (UID: \"960ec500-4be8-4e41-8f4b-cd8f20938c08\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:33:54.611269 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.610986    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls podName:412bfb04-3abc-46f3-b9d3-76c904b79c2a nodeName:}" failed. No retries permitted until 2026-07-09 09:33:56.610972112 +0000 UTC m=+39.617141161 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls") pod "dns-default-99ggf" (UID: "412bfb04-3abc-46f3-b9d3-76c904b79c2a") : secret "dns-default-metrics-tls" not found
Jul 09 09:33:54.611269 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.611001    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 09:33:54.611269 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.611044    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls podName:fbaaef5e-87e5-4dae-a367-b10fc0d62031 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:56.611028244 +0000 UTC m=+39.617197292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-sv5mc" (UID: "fbaaef5e-87e5-4dae-a367-b10fc0d62031") : secret "samples-operator-tls" not found
Jul 09 09:33:54.611269 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.611070    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle podName:0576329a-dc3f-459e-a951-f2b0aed69679 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:56.611056123 +0000 UTC m=+39.617225189 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle") pod "router-default-c7f6c5c4f-cjdpd" (UID: "0576329a-dc3f-459e-a951-f2b0aed69679") : configmap references non-existent config key: service-ca.crt
Jul 09 09:33:54.611269 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.611079    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 09:33:54.611269 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.611105    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:33:54.611269 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.611122    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert podName:960ec500-4be8-4e41-8f4b-cd8f20938c08 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:56.611108825 +0000 UTC m=+39.617277888 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-gcwjv" (UID: "960ec500-4be8-4e41-8f4b-cd8f20938c08") : secret "networking-console-plugin-cert" not found
Jul 09 09:33:54.611269 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.611152    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:54.611641 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.611176    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 09:33:54.611641 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.611214    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls podName:38bfa326-e79b-447e-ae2b-c31b088b6249 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:56.61120339 +0000 UTC m=+39.617372483 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-msxxf" (UID: "38bfa326-e79b-447e-ae2b-c31b088b6249") : secret "cluster-monitoring-operator-tls" not found
Jul 09 09:33:54.611641 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.611245    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 09 09:33:54.611641 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:54.611286    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs podName:0576329a-dc3f-459e-a951-f2b0aed69679 nodeName:}" failed. No retries permitted until 2026-07-09 09:33:56.611275183 +0000 UTC m=+39.617444261 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs") pod "router-default-c7f6c5c4f-cjdpd" (UID: "0576329a-dc3f-459e-a951-f2b0aed69679") : secret "router-metrics-certs-default" not found
Jul 09 09:33:54.710787 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.710752    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-d5888" event={"ID":"c0f6b837-c192-44ae-8c7f-02ee427acafc","Type":"ContainerStarted","Data":"736ab1355454d8cb4d7a78b3a745d3d3929be4a8f2ab21067cd3aa10dbe44676"}
Jul 09 09:33:54.711852 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.711829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9ztxj" event={"ID":"780a8630-14fc-41e6-801c-a8271ae77ddc","Type":"ContainerStarted","Data":"935435b2e7015667c7027b3d8f579ce2955e81771c551a7193629d8a72d91c19"}
Jul 09 09:33:54.713008 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.712988    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs" event={"ID":"24c5f723-d578-4374-8f99-2a7fa1f38727","Type":"ContainerStarted","Data":"50aeb80423a0c61feb636b782c850f5c6c6be66dca8fda1e1b1e2babac880bd2"}
Jul 09 09:33:54.714016 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.713999    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v9drg" event={"ID":"b57904de-b3f2-4b07-9098-9d1fb25f91a5","Type":"ContainerStarted","Data":"b4c042856341955952074c01ff7ee76844dc9354e64674ff5f7c57f6dd48354a"}
Jul 09 09:33:54.715047 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.715026    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4" event={"ID":"38d5ce80-3b12-4729-9e14-510a84a1e8d0","Type":"ContainerStarted","Data":"63c973dbb56188f596631f98481e31e44159aede08b6757c6b456d303e86b2c2"}
Jul 09 09:33:54.716082 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.716065    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d" event={"ID":"d5ac7be9-438f-473e-814d-c2a998a389f8","Type":"ContainerStarted","Data":"1d6435bd497825c76de9effc0a30b3ca5b3381c75d432d3868bcebf6315134ed"}
Jul 09 09:33:54.717053 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.717030    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6fsgb" event={"ID":"35d84c6e-4025-4ff5-83c5-92f505e2f9ca","Type":"ContainerStarted","Data":"d570115d6a550c6ee1cec3f9e80f3f108f4fbfc60d88fe318478849c070b8ab0"}
Jul 09 09:33:54.719659 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.719637    2577 generic.go:356] "Generic (PLEG): container finished" podID="6e8230b2-ffba-4769-888d-6fbf1a2ee9c7" containerID="00f15fa56755163eda9bee021ce15750df1a684c40fbdf133ddecf9abc1f4c85" exitCode=0
Jul 09 09:33:54.719726 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:54.719668    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9gtlt" event={"ID":"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7","Type":"ContainerDied","Data":"00f15fa56755163eda9bee021ce15750df1a684c40fbdf133ddecf9abc1f4c85"}
Jul 09 09:33:55.468612 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:55.468562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:33:55.731498 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:55.730314    2577 generic.go:356] "Generic (PLEG): container finished" podID="6e8230b2-ffba-4769-888d-6fbf1a2ee9c7" containerID="6b8771c3e37f1e70105ed0e2c287b7b3c80a06eefaa143396cecffa5668fac6e" exitCode=0
Jul 09 09:33:55.731498 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:55.730569    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9gtlt" event={"ID":"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7","Type":"ContainerDied","Data":"6b8771c3e37f1e70105ed0e2c287b7b3c80a06eefaa143396cecffa5668fac6e"}
Jul 09 09:33:56.535900 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:56.535869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:33:56.536074 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.536053    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 09:33:56.536074 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.536068    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59df9f699f-s9nfn: secret "image-registry-tls" not found
Jul 09 09:33:56.536181 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.536117    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls podName:7365b00b-2bda-4d1e-8b42-d83097184402 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:00.536101719 +0000 UTC m=+43.542270767 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls") pod "image-registry-59df9f699f-s9nfn" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402") : secret "image-registry-tls" not found
Jul 09 09:33:56.638140 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:56.636968    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:33:56.638140 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:56.637018    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-sv5mc\" (UID: \"fbaaef5e-87e5-4dae-a367-b10fc0d62031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:33:56.638140 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:56.637063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:56.638140 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:56.637120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-gcwjv\" (UID: \"960ec500-4be8-4e41-8f4b-cd8f20938c08\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:33:56.638140 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:56.637181    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:33:56.638140 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:56.637208    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:33:56.638140 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.637356    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 09 09:33:56.638140 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.637415    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs podName:0576329a-dc3f-459e-a951-f2b0aed69679 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:00.637396604 +0000 UTC m=+43.643565661 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs") pod "router-default-c7f6c5c4f-cjdpd" (UID: "0576329a-dc3f-459e-a951-f2b0aed69679") : secret "router-metrics-certs-default" not found
Jul 09 09:33:56.638140 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.637738    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle podName:0576329a-dc3f-459e-a951-f2b0aed69679 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:00.63772271 +0000 UTC m=+43.643891765 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle") pod "router-default-c7f6c5c4f-cjdpd" (UID: "0576329a-dc3f-459e-a951-f2b0aed69679") : configmap references non-existent config key: service-ca.crt
Jul 09 09:33:56.638140 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.637826    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 09:33:56.638140 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.637862    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls podName:412bfb04-3abc-46f3-b9d3-76c904b79c2a nodeName:}" failed. No retries permitted until 2026-07-09 09:34:00.637850115 +0000 UTC m=+43.644019169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls") pod "dns-default-99ggf" (UID: "412bfb04-3abc-46f3-b9d3-76c904b79c2a") : secret "dns-default-metrics-tls" not found
Jul 09 09:33:56.638140 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.637918    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 09:33:56.638888 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.637948    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls podName:fbaaef5e-87e5-4dae-a367-b10fc0d62031 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:00.637937507 +0000 UTC m=+43.644106560 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-sv5mc" (UID: "fbaaef5e-87e5-4dae-a367-b10fc0d62031") : secret "samples-operator-tls" not found
Jul 09 09:33:56.638888 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.638003    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 09:33:56.638888 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.638034    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls podName:38bfa326-e79b-447e-ae2b-c31b088b6249 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:00.638023445 +0000 UTC m=+43.644192499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-msxxf" (UID: "38bfa326-e79b-447e-ae2b-c31b088b6249") : secret "cluster-monitoring-operator-tls" not found
Jul 09 09:33:56.638888 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.638081    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 09:33:56.638888 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:33:56.638111    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert podName:960ec500-4be8-4e41-8f4b-cd8f20938c08 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:00.638101549 +0000 UTC m=+43.644270602 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-gcwjv" (UID: "960ec500-4be8-4e41-8f4b-cd8f20938c08") : secret "networking-console-plugin-cert" not found
Jul 09 09:33:56.737545 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:56.737509    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9ztxj" event={"ID":"780a8630-14fc-41e6-801c-a8271ae77ddc","Type":"ContainerStarted","Data":"42674f2e3317c51d129fb928f21a0aa3b71e9dc093280686b3ace3f06f32a7da"}
Jul 09 09:33:56.746102 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:56.746077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9gtlt" event={"ID":"6e8230b2-ffba-4769-888d-6fbf1a2ee9c7","Type":"ContainerStarted","Data":"bf2fc18c7e8d8dec259bee4da477a3a5a2852b32d101c90471f3a72383d1ba08"}
Jul 09 09:33:56.760303 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:56.760259    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9ztxj" podStartSLOduration=23.80566223 podStartE2EDuration="25.760240796s" totalImagesPullingTime="1.954578566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:33:56.758934537 +0000 UTC m=+39.765103609" watchObservedRunningTime="2026-07-09 09:33:56.760240796 +0000 UTC m=+39.766409870"
Jul 09 09:33:56.926596 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:56.926513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:33:57.025248 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:57.025216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:33:58.042590 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:58.042424    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:33:58.119756 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:58.119653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:33:59.717791 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:59.717751    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:33:59.795432 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:33:59.795396    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:00.571677 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:00.571641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:34:00.571945 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.571783    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 09:34:00.571945 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.571820    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59df9f699f-s9nfn: secret "image-registry-tls" not found
Jul 09 09:34:00.571945 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.571892    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls podName:7365b00b-2bda-4d1e-8b42-d83097184402 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:08.571871859 +0000 UTC m=+51.578040930 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls") pod "image-registry-59df9f699f-s9nfn" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402") : secret "image-registry-tls" not found
Jul 09 09:34:00.672997 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:00.672957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:34:00.673163 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:00.673038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-gcwjv\" (UID: \"960ec500-4be8-4e41-8f4b-cd8f20938c08\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:34:00.673163 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:00.673087    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:34:00.673163 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:00.673108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:34:00.673163 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.673144    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle podName:0576329a-dc3f-459e-a951-f2b0aed69679 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:08.673122438 +0000 UTC m=+51.679291491 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle") pod "router-default-c7f6c5c4f-cjdpd" (UID: "0576329a-dc3f-459e-a951-f2b0aed69679") : configmap references non-existent config key: service-ca.crt
Jul 09 09:34:00.673401 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.673187    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 09:34:00.673401 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.673197    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 09 09:34:00.673401 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.673230    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert podName:960ec500-4be8-4e41-8f4b-cd8f20938c08 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:08.673218417 +0000 UTC m=+51.679387480 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-gcwjv" (UID: "960ec500-4be8-4e41-8f4b-cd8f20938c08") : secret "networking-console-plugin-cert" not found
Jul 09 09:34:00.673401 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.673234    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 09:34:00.673401 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:00.673253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:34:00.673401 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.673261    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs podName:0576329a-dc3f-459e-a951-f2b0aed69679 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:08.673244548 +0000 UTC m=+51.679413597 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs") pod "router-default-c7f6c5c4f-cjdpd" (UID: "0576329a-dc3f-459e-a951-f2b0aed69679") : secret "router-metrics-certs-default" not found
Jul 09 09:34:00.673401 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:00.673291    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-sv5mc\" (UID: \"fbaaef5e-87e5-4dae-a367-b10fc0d62031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:34:00.673401 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.673304    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls podName:38bfa326-e79b-447e-ae2b-c31b088b6249 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:08.673287521 +0000 UTC m=+51.679456572 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-msxxf" (UID: "38bfa326-e79b-447e-ae2b-c31b088b6249") : secret "cluster-monitoring-operator-tls" not found
Jul 09 09:34:00.673401 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.673307    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 09:34:00.673401 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.673341    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls podName:412bfb04-3abc-46f3-b9d3-76c904b79c2a nodeName:}" failed. No retries permitted until 2026-07-09 09:34:08.673330803 +0000 UTC m=+51.679499852 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls") pod "dns-default-99ggf" (UID: "412bfb04-3abc-46f3-b9d3-76c904b79c2a") : secret "dns-default-metrics-tls" not found
Jul 09 09:34:00.673401 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.673350    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 09:34:00.673401 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:00.673382    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls podName:fbaaef5e-87e5-4dae-a367-b10fc0d62031 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:08.673371997 +0000 UTC m=+51.679541046 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-sv5mc" (UID: "fbaaef5e-87e5-4dae-a367-b10fc0d62031") : secret "samples-operator-tls" not found
Jul 09 09:34:02.553491 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.553449    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:02.628700 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.628658    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:02.770032 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.769497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-d5888" event={"ID":"c0f6b837-c192-44ae-8c7f-02ee427acafc","Type":"ContainerStarted","Data":"73a31507486862d47221fa2ecc7d1b94d075e1d71c1c70816fe6845ab43b0deb"}
Jul 09 09:34:02.771452 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.771435    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs" event={"ID":"24c5f723-d578-4374-8f99-2a7fa1f38727","Type":"ContainerStarted","Data":"5c0f026767e63c1f983ccbbaf8b8d4462264c4c091e315837be7765fd79aff1f"}
Jul 09 09:34:02.773479 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.773422    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v9drg" event={"ID":"b57904de-b3f2-4b07-9098-9d1fb25f91a5","Type":"ContainerStarted","Data":"ca4b97620963deec7f8f78409adc21f2dd24cf005fc320cdbc38a39330fbb546"}
Jul 09 09:34:02.775235 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.775219    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/0.log"
Jul 09 09:34:02.775376 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.775360    2577 generic.go:356] "Generic (PLEG): container finished" podID="38d5ce80-3b12-4729-9e14-510a84a1e8d0" containerID="47f3c5fb7a0af9f95a34cdc51445b6d29878051a9e84a1acdc52eff0f8e4172d" exitCode=255
Jul 09 09:34:02.775540 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.775446    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4" event={"ID":"38d5ce80-3b12-4729-9e14-510a84a1e8d0","Type":"ContainerDied","Data":"47f3c5fb7a0af9f95a34cdc51445b6d29878051a9e84a1acdc52eff0f8e4172d"}
Jul 09 09:34:02.775665 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.775587    2577 scope.go:117] "RemoveContainer" containerID="47f3c5fb7a0af9f95a34cdc51445b6d29878051a9e84a1acdc52eff0f8e4172d"
Jul 09 09:34:02.777375 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.777202    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d" event={"ID":"d5ac7be9-438f-473e-814d-c2a998a389f8","Type":"ContainerStarted","Data":"11c55bfafd80399370505ad3a91cf29ea5ee1d68c781bd119656a44d48258338"}
Jul 09 09:34:02.778715 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.778695    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6fsgb" event={"ID":"35d84c6e-4025-4ff5-83c5-92f505e2f9ca","Type":"ContainerStarted","Data":"c7032c2aa127b5504f8e6c5b3d869fe3624e992f907699990d6dbcd6e8edbab2"}
Jul 09 09:34:02.799200 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.798501    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9gtlt" podStartSLOduration=13.740679329 podStartE2EDuration="44.798484757s" totalImagesPullingTime="31.057805428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:33:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:33:56.789577351 +0000 UTC m=+39.795746439" watchObservedRunningTime="2026-07-09 09:34:02.798484757 +0000 UTC m=+45.804653829"
Jul 09 09:34:02.827811 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.827754    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-d5888" podStartSLOduration=24.344882502 podStartE2EDuration="31.827736548s" totalImagesPullingTime="7.482854046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:02.799378425 +0000 UTC m=+45.805547495" watchObservedRunningTime="2026-07-09 09:34:02.827736548 +0000 UTC m=+45.833905620"
Jul 09 09:34:02.827921 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.827871    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs" podStartSLOduration=24.386965283 podStartE2EDuration="31.827865482s" totalImagesPullingTime="7.440900199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:02.827558028 +0000 UTC m=+45.833727099" watchObservedRunningTime="2026-07-09 09:34:02.827865482 +0000 UTC m=+45.834034554"
Jul 09 09:34:02.862052 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.862016    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v9drg" podStartSLOduration=3.41812439 podStartE2EDuration="10.862002864s" totalImagesPullingTime="7.443878474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:02.860190824 +0000 UTC m=+45.866359914" watchObservedRunningTime="2026-07-09 09:34:02.862002864 +0000 UTC m=+45.868171934"
Jul 09 09:34:02.929612 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.929555    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d" podStartSLOduration=24.487564634 podStartE2EDuration="31.929533563s" totalImagesPullingTime="7.441968929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:02.888046187 +0000 UTC m=+45.894215259" watchObservedRunningTime="2026-07-09 09:34:02.929533563 +0000 UTC m=+45.935702635"
Jul 09 09:34:02.966580 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:02.966532    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6fsgb" podStartSLOduration=24.479021571 podStartE2EDuration="31.966513294s" totalImagesPullingTime="7.487491723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:02.962436645 +0000 UTC m=+45.968605716" watchObservedRunningTime="2026-07-09 09:34:02.966513294 +0000 UTC m=+45.972682368"
Jul 09 09:34:03.231893 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.231778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:03.235337 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.235317    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:34:03.235451 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.235373    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:34:03.323228 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.323182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:03.670485 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.670413    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:03.737971 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.737939    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7hkrk"]
Jul 09 09:34:03.760362 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.760335    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7hkrk"]
Jul 09 09:34:03.760501 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.760433    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:03.762627 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.762601    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 09:34:03.762960 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.762943    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 09:34:03.763380 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.763362    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zwk2d\""
Jul 09 09:34:03.765166 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.765145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:03.783163 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.783133    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/1.log"
Jul 09 09:34:03.783506 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.783489    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/0.log"
Jul 09 09:34:03.783612 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.783526    2577 generic.go:356] "Generic (PLEG): container finished" podID="38d5ce80-3b12-4729-9e14-510a84a1e8d0" containerID="5e00c1e0a9325518019794d6e56b08d97b35d47a5feb501de6a96c98c0ff8932" exitCode=255
Jul 09 09:34:03.783667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.783618    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4" event={"ID":"38d5ce80-3b12-4729-9e14-510a84a1e8d0","Type":"ContainerDied","Data":"5e00c1e0a9325518019794d6e56b08d97b35d47a5feb501de6a96c98c0ff8932"}
Jul 09 09:34:03.783715 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.783667    2577 scope.go:117] "RemoveContainer" containerID="47f3c5fb7a0af9f95a34cdc51445b6d29878051a9e84a1acdc52eff0f8e4172d"
Jul 09 09:34:03.784123 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.783912    2577 scope.go:117] "RemoveContainer" containerID="5e00c1e0a9325518019794d6e56b08d97b35d47a5feb501de6a96c98c0ff8932"
Jul 09 09:34:03.784369 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:03.784345    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-ms4r4_openshift-console-operator(38d5ce80-3b12-4729-9e14-510a84a1e8d0)\"" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4" podUID="38d5ce80-3b12-4729-9e14-510a84a1e8d0"
Jul 09 09:34:03.825314 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.825283    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:03.901938 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.901911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24feaef0-8968-4d52-86dd-3675636349ac-crio-socket\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:03.902197 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.902177    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm8v9\" (UniqueName: \"kubernetes.io/projected/24feaef0-8968-4d52-86dd-3675636349ac-kube-api-access-zm8v9\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:03.902275 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.902261    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24feaef0-8968-4d52-86dd-3675636349ac-data-volume\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:03.902372 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.902354    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:03.902427 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:03.902387    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:04.003533 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.003515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm8v9\" (UniqueName: \"kubernetes.io/projected/24feaef0-8968-4d52-86dd-3675636349ac-kube-api-access-zm8v9\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:04.003620 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.003543    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24feaef0-8968-4d52-86dd-3675636349ac-data-volume\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:04.003620 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.003571    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:04.003686 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:04.003647    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 09:34:04.003719 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.003690    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:04.003719 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:04.003704    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls podName:24feaef0-8968-4d52-86dd-3675636349ac nodeName:}" failed. No retries permitted until 2026-07-09 09:34:04.50368876 +0000 UTC m=+47.509857815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7hkrk" (UID: "24feaef0-8968-4d52-86dd-3675636349ac") : secret "insights-runtime-extractor-tls" not found
Jul 09 09:34:04.003836 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:04.003743    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 09:34:04.003836 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:04.003824    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm podName:24feaef0-8968-4d52-86dd-3675636349ac nodeName:}" failed. No retries permitted until 2026-07-09 09:34:04.503785385 +0000 UTC m=+47.509954434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7hkrk" (UID: "24feaef0-8968-4d52-86dd-3675636349ac") : configmap "kube-rbac-proxy" not found
Jul 09 09:34:04.003923 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.003865    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24feaef0-8968-4d52-86dd-3675636349ac-data-volume\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:04.003923 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.003876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24feaef0-8968-4d52-86dd-3675636349ac-crio-socket\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:04.003994 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.003974    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24feaef0-8968-4d52-86dd-3675636349ac-crio-socket\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:04.014034 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.014012    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm8v9\" (UniqueName: \"kubernetes.io/projected/24feaef0-8968-4d52-86dd-3675636349ac-kube-api-access-zm8v9\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:04.251485 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.251452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:04.357473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.357407    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:04.452252 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.452213    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:04.508886 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.508864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:04.508886 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.508893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:04.509037 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:04.508957    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 09:34:04.509037 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:04.509011    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 09:34:04.509037 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:04.509019    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm podName:24feaef0-8968-4d52-86dd-3675636349ac nodeName:}" failed. No retries permitted until 2026-07-09 09:34:05.509004442 +0000 UTC m=+48.515173500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7hkrk" (UID: "24feaef0-8968-4d52-86dd-3675636349ac") : configmap "kube-rbac-proxy" not found
Jul 09 09:34:04.509150 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:04.509065    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls podName:24feaef0-8968-4d52-86dd-3675636349ac nodeName:}" failed. No retries permitted until 2026-07-09 09:34:05.509051244 +0000 UTC m=+48.515220293 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7hkrk" (UID: "24feaef0-8968-4d52-86dd-3675636349ac") : secret "insights-runtime-extractor-tls" not found
Jul 09 09:34:04.789407 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.789381    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/1.log"
Jul 09 09:34:04.789888 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:04.789758    2577 scope.go:117] "RemoveContainer" containerID="5e00c1e0a9325518019794d6e56b08d97b35d47a5feb501de6a96c98c0ff8932"
Jul 09 09:34:04.789997 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:04.789976    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-ms4r4_openshift-console-operator(38d5ce80-3b12-4729-9e14-510a84a1e8d0)\"" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4" podUID="38d5ce80-3b12-4729-9e14-510a84a1e8d0"
Jul 09 09:34:05.114291 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:05.114233    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4nx9g_7e639bf5-c03c-46fa-a7ba-7d6770be1e10/dns-node-resolver/0.log"
Jul 09 09:34:05.516523 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:05.516494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:05.516523 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:05.516524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:05.516738 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:05.516644    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 09:34:05.516738 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:05.516646    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 09:34:05.516738 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:05.516693    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm podName:24feaef0-8968-4d52-86dd-3675636349ac nodeName:}" failed. No retries permitted until 2026-07-09 09:34:07.516680622 +0000 UTC m=+50.522849670 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7hkrk" (UID: "24feaef0-8968-4d52-86dd-3675636349ac") : configmap "kube-rbac-proxy" not found
Jul 09 09:34:05.516738 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:05.516706    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls podName:24feaef0-8968-4d52-86dd-3675636349ac nodeName:}" failed. No retries permitted until 2026-07-09 09:34:07.51670051 +0000 UTC m=+50.522869559 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7hkrk" (UID: "24feaef0-8968-4d52-86dd-3675636349ac") : secret "insights-runtime-extractor-tls" not found
Jul 09 09:34:05.818827 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:05.818744    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:05.912715 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:05.912683    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:06.009308 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:06.009283    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:06.079385 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:06.079323    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:06.515474 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:06.515455    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-skw5s_45c0bd3d-9996-4fa7-9afd-b4027c3286a0/node-ca/0.log"
Jul 09 09:34:06.557588 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:06.557558    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:06.627738 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:06.627704    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:06.920175 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:06.920117    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v9drg_b57904de-b3f2-4b07-9098-9d1fb25f91a5/serve-healthcheck-canary/0.log"
Jul 09 09:34:06.954830 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:06.954787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:07.052103 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:07.052071    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:07.368350 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:07.368093    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:07.530586 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:07.530560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:07.530723 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:07.530590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:07.530723 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:07.530689    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 09:34:07.530862 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:07.530742    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 09:34:07.530862 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:07.530764    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls podName:24feaef0-8968-4d52-86dd-3675636349ac nodeName:}" failed. No retries permitted until 2026-07-09 09:34:11.53074678 +0000 UTC m=+54.536915833 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7hkrk" (UID: "24feaef0-8968-4d52-86dd-3675636349ac") : secret "insights-runtime-extractor-tls" not found
Jul 09 09:34:07.530862 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:07.530785    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm podName:24feaef0-8968-4d52-86dd-3675636349ac nodeName:}" failed. No retries permitted until 2026-07-09 09:34:11.53077315 +0000 UTC m=+54.536942199 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7hkrk" (UID: "24feaef0-8968-4d52-86dd-3675636349ac") : configmap "kube-rbac-proxy" not found
Jul 09 09:34:08.268920 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:08.268887    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:08.328374 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:08.328344    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:08.640842 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:08.640760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:34:08.640962 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.640905    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 09:34:08.640962 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.640920    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59df9f699f-s9nfn: secret "image-registry-tls" not found
Jul 09 09:34:08.641042 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.640970    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls podName:7365b00b-2bda-4d1e-8b42-d83097184402 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:24.640955021 +0000 UTC m=+67.647124084 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls") pod "image-registry-59df9f699f-s9nfn" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402") : secret "image-registry-tls" not found
Jul 09 09:34:08.741399 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:08.741376    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:34:08.741533 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:08.741405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-sv5mc\" (UID: \"fbaaef5e-87e5-4dae-a367-b10fc0d62031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:34:08.741533 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:08.741435    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:34:08.741533 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:08.741471    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-gcwjv\" (UID: \"960ec500-4be8-4e41-8f4b-cd8f20938c08\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:34:08.741533 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:08.741505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:34:08.741533 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.741516    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 09:34:08.741760 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:08.741522    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:34:08.741760 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.741564    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls podName:412bfb04-3abc-46f3-b9d3-76c904b79c2a nodeName:}" failed. No retries permitted until 2026-07-09 09:34:24.741551685 +0000 UTC m=+67.747720737 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls") pod "dns-default-99ggf" (UID: "412bfb04-3abc-46f3-b9d3-76c904b79c2a") : secret "dns-default-metrics-tls" not found
Jul 09 09:34:08.741760 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.741585    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 09 09:34:08.741760 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.741601    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle podName:0576329a-dc3f-459e-a951-f2b0aed69679 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:24.741583597 +0000 UTC m=+67.747752646 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle") pod "router-default-c7f6c5c4f-cjdpd" (UID: "0576329a-dc3f-459e-a951-f2b0aed69679") : configmap references non-existent config key: service-ca.crt
Jul 09 09:34:08.741760 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.741606    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 09:34:08.741760 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.741635    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs podName:0576329a-dc3f-459e-a951-f2b0aed69679 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:24.741623012 +0000 UTC m=+67.747792065 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs") pod "router-default-c7f6c5c4f-cjdpd" (UID: "0576329a-dc3f-459e-a951-f2b0aed69679") : secret "router-metrics-certs-default" not found
Jul 09 09:34:08.741760 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.741652    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 09:34:08.741760 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.741668    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls podName:fbaaef5e-87e5-4dae-a367-b10fc0d62031 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:24.741653687 +0000 UTC m=+67.747822740 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-sv5mc" (UID: "fbaaef5e-87e5-4dae-a367-b10fc0d62031") : secret "samples-operator-tls" not found
Jul 09 09:34:08.741760 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.741683    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 09:34:08.741760 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.741701    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert podName:960ec500-4be8-4e41-8f4b-cd8f20938c08 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:24.741692394 +0000 UTC m=+67.747861445 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-gcwjv" (UID: "960ec500-4be8-4e41-8f4b-cd8f20938c08") : secret "networking-console-plugin-cert" not found
Jul 09 09:34:08.741760 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:08.741728    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls podName:38bfa326-e79b-447e-ae2b-c31b088b6249 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:24.741717958 +0000 UTC m=+67.747887011 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-msxxf" (UID: "38bfa326-e79b-447e-ae2b-c31b088b6249") : secret "cluster-monitoring-operator-tls" not found
Jul 09 09:34:09.398274 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:09.398233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:10.015667 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:10.015636    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:11.316167 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:11.316133    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:11.558332 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:11.558295    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:11.563330 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:11.563309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:11.563434 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:11.563419    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 09:34:11.563510 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:11.563497    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls podName:24feaef0-8968-4d52-86dd-3675636349ac nodeName:}" failed. No retries permitted until 2026-07-09 09:34:19.563477624 +0000 UTC m=+62.569646677 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7hkrk" (UID: "24feaef0-8968-4d52-86dd-3675636349ac") : secret "insights-runtime-extractor-tls" not found
Jul 09 09:34:11.563580 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:11.563422    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:11.563907 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:11.563886    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24feaef0-8968-4d52-86dd-3675636349ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:11.661215 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:11.661145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:11.732806 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:11.732777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:11.810944 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:11.810915    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:11.908885 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:11.908846    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:12.050949 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:12.050923    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:13.225281 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:13.225240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:13.235433 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:13.235401    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:34:13.235564 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:13.235462    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:34:13.235835 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:13.235820    2577 scope.go:117] "RemoveContainer" containerID="5e00c1e0a9325518019794d6e56b08d97b35d47a5feb501de6a96c98c0ff8932"
Jul 09 09:34:13.434042 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:13.434018    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:13.614189 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:13.614106    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:13.812689 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:13.812660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:13.819166 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:13.819138    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:34:13.819553 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:13.819540    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/1.log"
Jul 09 09:34:13.819605 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:13.819572    2577 generic.go:356] "Generic (PLEG): container finished" podID="38d5ce80-3b12-4729-9e14-510a84a1e8d0" containerID="40ecff50bdf5b0aa8138d071e90675f714767eb1c4e8a259433e5a42c2f586b6" exitCode=255
Jul 09 09:34:13.819605 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:13.819593    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4" event={"ID":"38d5ce80-3b12-4729-9e14-510a84a1e8d0","Type":"ContainerDied","Data":"40ecff50bdf5b0aa8138d071e90675f714767eb1c4e8a259433e5a42c2f586b6"}
Jul 09 09:34:13.819692 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:13.819629    2577 scope.go:117] "RemoveContainer" containerID="5e00c1e0a9325518019794d6e56b08d97b35d47a5feb501de6a96c98c0ff8932"
Jul 09 09:34:13.819938 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:13.819923    2577 scope.go:117] "RemoveContainer" containerID="40ecff50bdf5b0aa8138d071e90675f714767eb1c4e8a259433e5a42c2f586b6"
Jul 09 09:34:13.820106 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:13.820087    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-ms4r4_openshift-console-operator(38d5ce80-3b12-4729-9e14-510a84a1e8d0)\"" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4" podUID="38d5ce80-3b12-4729-9e14-510a84a1e8d0"
Jul 09 09:34:14.013698 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:14.013667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:14.704189 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:14.704161    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:14.807533 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:14.807496    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:14.824452 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:14.824427    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:34:15.035840 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:15.035814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:15.108451 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:15.108423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:15.360695 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:15.360615    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:15.459272 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:15.459244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:15.621155 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:15.621099    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:15.845014 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:15.844990    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:16.034169 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:16.034145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:16.142173 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:16.142145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:17.218094 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:17.218054    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:17.422676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:17.422648    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:17.530426 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:17.530371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:17.640780 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:17.640758    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:17.819905 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:17.819845    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:18.016463 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:18.016432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:18.708210 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:18.708186    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v8sxk"
Jul 09 09:34:18.842113 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:18.842088    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:18.940574 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:18.940531    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:19.205475 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:19.205443    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:19.618300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:19.618269    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:19.630911 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:19.630887    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:19.633203 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:19.633180    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24feaef0-8968-4d52-86dd-3675636349ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7hkrk\" (UID: \"24feaef0-8968-4d52-86dd-3675636349ac\") " pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:19.673242 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:19.673221    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zwk2d\""
Jul 09 09:34:19.681248 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:19.681232    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7hkrk"
Jul 09 09:34:19.813967 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:19.813943    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7hkrk"]
Jul 09 09:34:19.816856 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:34:19.816827    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24feaef0_8968_4d52_86dd_3675636349ac.slice/crio-ef69efb91f6af0a536c1061d53484bab270b097cebd2350619a7180973ac0c8e WatchSource:0}: Error finding container ef69efb91f6af0a536c1061d53484bab270b097cebd2350619a7180973ac0c8e: Status 404 returned error can't find the container with id ef69efb91f6af0a536c1061d53484bab270b097cebd2350619a7180973ac0c8e
Jul 09 09:34:19.828454 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:19.828420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:19.843377 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:19.843354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7hkrk" event={"ID":"24feaef0-8968-4d52-86dd-3675636349ac","Type":"ContainerStarted","Data":"ef69efb91f6af0a536c1061d53484bab270b097cebd2350619a7180973ac0c8e"}
Jul 09 09:34:20.405398 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:20.405361    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:20.613193 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:20.613138    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:21.030853 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:21.030816    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:21.612377 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:21.612338    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:21.821457 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:21.821420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:22.015615 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:22.015580    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:23.235534 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.235505    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:34:23.235946 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.235583    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:34:23.235946 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.235912    2577 scope.go:117] "RemoveContainer" containerID="40ecff50bdf5b0aa8138d071e90675f714767eb1c4e8a259433e5a42c2f586b6"
Jul 09 09:34:23.236124 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:23.236095    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-ms4r4_openshift-console-operator(38d5ce80-3b12-4729-9e14-510a84a1e8d0)\"" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4" podUID="38d5ce80-3b12-4729-9e14-510a84a1e8d0"
Jul 09 09:34:23.263857 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.263821    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:34:23.263982 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.263859    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs\") pod \"network-metrics-daemon-dsk4h\" (UID: \"8e94d5d6-6e3b-4c0e-b73d-e8f086218175\") " pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:34:23.266212 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.266188    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 09:34:23.266314 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.266282    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 09:34:23.277135 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.277085    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e94d5d6-6e3b-4c0e-b73d-e8f086218175-metrics-certs\") pod \"network-metrics-daemon-dsk4h\" (UID: \"8e94d5d6-6e3b-4c0e-b73d-e8f086218175\") " pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:34:23.277135 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.277096    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f492c23-72ed-4bbf-899d-3d5f588ef133-original-pull-secret\") pod \"global-pull-secret-syncer-dh44v\" (UID: \"2f492c23-72ed-4bbf-899d-3d5f588ef133\") " pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:34:23.365199 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.365171    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvl8c\" (UniqueName: \"kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c\") pod \"network-check-target-sjr8w\" (UID: \"ca9f20c6-5f63-4f75-87b1-00dc649acbcc\") " pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:34:23.367372 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.367351    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvl8c\" (UniqueName: \"kubernetes.io/projected/ca9f20c6-5f63-4f75-87b1-00dc649acbcc-kube-api-access-tvl8c\") pod \"network-check-target-sjr8w\" (UID: \"ca9f20c6-5f63-4f75-87b1-00dc649acbcc\") " pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:34:23.371187 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.371171    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh44v"
Jul 09 09:34:23.379166 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.379146    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r7trk\""
Jul 09 09:34:23.383010 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.382993    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ntnhc\""
Jul 09 09:34:23.386907 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.386890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:34:23.391243 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.391228    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dsk4h"
Jul 09 09:34:23.522318 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.522287    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dh44v"]
Jul 09 09:34:23.526856 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:34:23.526825    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f492c23_72ed_4bbf_899d_3d5f588ef133.slice/crio-937d8734987cb2ca6ad3c449e67d67a75fb04c7957684a51eac9cc2c7c81f88d WatchSource:0}: Error finding container 937d8734987cb2ca6ad3c449e67d67a75fb04c7957684a51eac9cc2c7c81f88d: Status 404 returned error can't find the container with id 937d8734987cb2ca6ad3c449e67d67a75fb04c7957684a51eac9cc2c7c81f88d
Jul 09 09:34:23.539791 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.539770    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sjr8w"]
Jul 09 09:34:23.542351 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:34:23.542322    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca9f20c6_5f63_4f75_87b1_00dc649acbcc.slice/crio-41a050114adce595fc86b48ca9aec8a2080cf63351b85f3aacd75e32abce082a WatchSource:0}: Error finding container 41a050114adce595fc86b48ca9aec8a2080cf63351b85f3aacd75e32abce082a: Status 404 returned error can't find the container with id 41a050114adce595fc86b48ca9aec8a2080cf63351b85f3aacd75e32abce082a
Jul 09 09:34:23.554660 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.554639    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dsk4h"]
Jul 09 09:34:23.557066 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:34:23.557039    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e94d5d6_6e3b_4c0e_b73d_e8f086218175.slice/crio-88742f487ab5b99b7d47925cec71d05e895ee04865088c01fa2b1163b71fe877 WatchSource:0}: Error finding container 88742f487ab5b99b7d47925cec71d05e895ee04865088c01fa2b1163b71fe877: Status 404 returned error can't find the container with id 88742f487ab5b99b7d47925cec71d05e895ee04865088c01fa2b1163b71fe877
Jul 09 09:34:23.858381 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.858319    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7hkrk" event={"ID":"24feaef0-8968-4d52-86dd-3675636349ac","Type":"ContainerStarted","Data":"4e79aaf30a2e23020a137f8cc21b0b804425fe0462f99469e32b21ff024ba7c8"}
Jul 09 09:34:23.859707 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.859686    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sjr8w" event={"ID":"ca9f20c6-5f63-4f75-87b1-00dc649acbcc","Type":"ContainerStarted","Data":"377d8ae14d24f0ff1662d79d656cb9e140c06b0a5069ecf4fa6eeb65417734ef"}
Jul 09 09:34:23.859814 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.859721    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sjr8w" event={"ID":"ca9f20c6-5f63-4f75-87b1-00dc649acbcc","Type":"ContainerStarted","Data":"41a050114adce595fc86b48ca9aec8a2080cf63351b85f3aacd75e32abce082a"}
Jul 09 09:34:23.859814 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.859758    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:34:23.860815 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.860780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dsk4h" event={"ID":"8e94d5d6-6e3b-4c0e-b73d-e8f086218175","Type":"ContainerStarted","Data":"88742f487ab5b99b7d47925cec71d05e895ee04865088c01fa2b1163b71fe877"}
Jul 09 09:34:23.861683 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.861664    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dh44v" event={"ID":"2f492c23-72ed-4bbf-899d-3d5f588ef133","Type":"ContainerStarted","Data":"937d8734987cb2ca6ad3c449e67d67a75fb04c7957684a51eac9cc2c7c81f88d"}
Jul 09 09:34:23.878578 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:23.878535    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sjr8w" podStartSLOduration=65.878523485 podStartE2EDuration="1m5.878523485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:23.877464293 +0000 UTC m=+66.883633364" watchObservedRunningTime="2026-07-09 09:34:23.878523485 +0000 UTC m=+66.884692592"
Jul 09 09:34:24.679768 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.679744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:34:24.682090 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.682069    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls\") pod \"image-registry-59df9f699f-s9nfn\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") " pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:34:24.780203 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.780121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:34:24.780203 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.780169    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:34:24.780426 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.780215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:34:24.780426 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.780242    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-sv5mc\" (UID: \"fbaaef5e-87e5-4dae-a367-b10fc0d62031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:34:24.780426 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.780280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:34:24.780426 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.780317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-gcwjv\" (UID: \"960ec500-4be8-4e41-8f4b-cd8f20938c08\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:34:24.781277 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.781245    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0576329a-dc3f-459e-a951-f2b0aed69679-service-ca-bundle\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:34:24.783262 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.783220    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/38bfa326-e79b-447e-ae2b-c31b088b6249-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-msxxf\" (UID: \"38bfa326-e79b-447e-ae2b-c31b088b6249\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:34:24.783262 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.783242    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/412bfb04-3abc-46f3-b9d3-76c904b79c2a-metrics-tls\") pod \"dns-default-99ggf\" (UID: \"412bfb04-3abc-46f3-b9d3-76c904b79c2a\") " pod="openshift-dns/dns-default-99ggf"
Jul 09 09:34:24.783419 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.783302    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/960ec500-4be8-4e41-8f4b-cd8f20938c08-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-gcwjv\" (UID: \"960ec500-4be8-4e41-8f4b-cd8f20938c08\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:34:24.783419 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.783302    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0576329a-dc3f-459e-a951-f2b0aed69679-metrics-certs\") pod \"router-default-c7f6c5c4f-cjdpd\" (UID: \"0576329a-dc3f-459e-a951-f2b0aed69679\") " pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:34:24.783419 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.783355    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fbaaef5e-87e5-4dae-a367-b10fc0d62031-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-sv5mc\" (UID: \"fbaaef5e-87e5-4dae-a367-b10fc0d62031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:34:24.867076 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.867044    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dsk4h" event={"ID":"8e94d5d6-6e3b-4c0e-b73d-e8f086218175","Type":"ContainerStarted","Data":"8a129fb7dc5d05d3ab3b8efbda04b7de05f61230826160ef8ccb3ae382422a1f"}
Jul 09 09:34:24.867245 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.867102    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dsk4h" event={"ID":"8e94d5d6-6e3b-4c0e-b73d-e8f086218175","Type":"ContainerStarted","Data":"92ae8ca192d791d8297a637592c00b6fd5028929f309af1bef1e05f5a7b8ceef"}
Jul 09 09:34:24.868997 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.868968    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7hkrk" event={"ID":"24feaef0-8968-4d52-86dd-3675636349ac","Type":"ContainerStarted","Data":"d5d6acadd8b356a96efead7887d30fbb19400645a57646eda496a42c5253392f"}
Jul 09 09:34:24.884865 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.884781    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dsk4h" podStartSLOduration=65.941094231 podStartE2EDuration="1m6.884764966s" totalImagesPullingTime="943.670735ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:24.884135204 +0000 UTC m=+67.890304267" watchObservedRunningTime="2026-07-09 09:34:24.884764966 +0000 UTC m=+67.890934040"
Jul 09 09:34:24.952647 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.952623    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pjjd7\""
Jul 09 09:34:24.960402 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.960385    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:34:24.971012 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.970990    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-69h7n\""
Jul 09 09:34:24.977495 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.977474    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"
Jul 09 09:34:24.983399 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.983382    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4rqw2\""
Jul 09 09:34:24.991630 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.991605    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:34:24.992608 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:24.992582    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cfzpw\""
Jul 09 09:34:25.000993 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.000975    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-99ggf"
Jul 09 09:34:25.006378 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.006195    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tbkpb\""
Jul 09 09:34:25.013611 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.013562    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"
Jul 09 09:34:25.051458 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.051221    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-d89w5\""
Jul 09 09:34:25.059073 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.058855    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"
Jul 09 09:34:25.211098 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.211009    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf"]
Jul 09 09:34:25.214115 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.214051    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59df9f699f-s9nfn"]
Jul 09 09:34:25.234766 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:34:25.234724    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bfa326_e79b_447e_ae2b_c31b088b6249.slice/crio-95b5ad5e0cb233d58ecf4935e2dc71cc740d8d9634d2e79343c7347dd5adb14c WatchSource:0}: Error finding container 95b5ad5e0cb233d58ecf4935e2dc71cc740d8d9634d2e79343c7347dd5adb14c: Status 404 returned error can't find the container with id 95b5ad5e0cb233d58ecf4935e2dc71cc740d8d9634d2e79343c7347dd5adb14c
Jul 09 09:34:25.252461 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.249445    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c7f6c5c4f-cjdpd"]
Jul 09 09:34:25.283541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.283479    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc"]
Jul 09 09:34:25.301960 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.301891    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-99ggf"]
Jul 09 09:34:25.338732 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.338660    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv"]
Jul 09 09:34:25.348769 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:34:25.348733    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod960ec500_4be8_4e41_8f4b_cd8f20938c08.slice/crio-81e57951088d164befea4f7ffd330c761f050bc95e410f10e68e444affb4e29f WatchSource:0}: Error finding container 81e57951088d164befea4f7ffd330c761f050bc95e410f10e68e444affb4e29f: Status 404 returned error can't find the container with id 81e57951088d164befea4f7ffd330c761f050bc95e410f10e68e444affb4e29f
Jul 09 09:34:25.883421 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.883338    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv" event={"ID":"960ec500-4be8-4e41-8f4b-cd8f20938c08","Type":"ContainerStarted","Data":"81e57951088d164befea4f7ffd330c761f050bc95e410f10e68e444affb4e29f"}
Jul 09 09:34:25.887665 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.887592    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf" event={"ID":"38bfa326-e79b-447e-ae2b-c31b088b6249","Type":"ContainerStarted","Data":"95b5ad5e0cb233d58ecf4935e2dc71cc740d8d9634d2e79343c7347dd5adb14c"}
Jul 09 09:34:25.893386 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.893308    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc" event={"ID":"fbaaef5e-87e5-4dae-a367-b10fc0d62031","Type":"ContainerStarted","Data":"4894378ac1cd44a0d7650f6510561c03eb6c4d078b87af577c9afd5c45b05484"}
Jul 09 09:34:25.898858 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.898067    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd" event={"ID":"0576329a-dc3f-459e-a951-f2b0aed69679","Type":"ContainerStarted","Data":"7e01d247348ba1fd7b3681fb73b7a41cfce32a2d9b63d4694bfa488323e14473"}
Jul 09 09:34:25.898858 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.898107    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd" event={"ID":"0576329a-dc3f-459e-a951-f2b0aed69679","Type":"ContainerStarted","Data":"fb9eaa921d2e446072acf4f4390268f8354cafffe79b7e4621d65ebef71c97ce"}
Jul 09 09:34:25.901544 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.900972    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59df9f699f-s9nfn" event={"ID":"7365b00b-2bda-4d1e-8b42-d83097184402","Type":"ContainerStarted","Data":"9794c2c6650da1128a20c59259d8f45177cf52e9e2e0bb0b2e19912dad46bcf2"}
Jul 09 09:34:25.901544 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.901011    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59df9f699f-s9nfn" event={"ID":"7365b00b-2bda-4d1e-8b42-d83097184402","Type":"ContainerStarted","Data":"aa72be9d71428d48a0cae59ea3a5333deb6f2b990725329e7f426037d9677adf"}
Jul 09 09:34:25.901544 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.901490    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:34:25.904563 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.904545    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-99ggf" event={"ID":"412bfb04-3abc-46f3-b9d3-76c904b79c2a","Type":"ContainerStarted","Data":"79104ea687b8ae4b341d1d4996fae6468617b2d43fb148d32ee8cf008e8ba55b"}
Jul 09 09:34:25.929245 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.927672    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd" podStartSLOduration=54.927654814 podStartE2EDuration="54.927654814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:25.926877879 +0000 UTC m=+68.933046961" watchObservedRunningTime="2026-07-09 09:34:25.927654814 +0000 UTC m=+68.933823888"
Jul 09 09:34:25.998316 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:25.997202    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:34:26.000352 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:26.000168    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:34:26.037004 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:26.036954    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59df9f699f-s9nfn" podStartSLOduration=67.036935225 podStartE2EDuration="1m7.036935225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:25.960189444 +0000 UTC m=+68.966358515" watchObservedRunningTime="2026-07-09 09:34:26.036935225 +0000 UTC m=+69.043104299"
Jul 09 09:34:26.533885 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:26.533733    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:26.697428 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:26.697233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:26.880122 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:26.879936    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:26.915445 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:26.915386    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:34:26.916971 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:26.916763    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-c7f6c5c4f-cjdpd"
Jul 09 09:34:27.058912 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:27.058817    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:29.564387 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:29.564302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:29.635723 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:29.635664    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:29.736251 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:29.736216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:29.809143 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:29.809103    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:31.925647 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:31.925609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:32.031866 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:32.031830    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:32.945824 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:32.945025    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf" event={"ID":"38bfa326-e79b-447e-ae2b-c31b088b6249","Type":"ContainerStarted","Data":"fe37ac1df2e13c1bdedc9cbf1ef2fe32fb75ce083274a942cf3ecfacc6203052"}
Jul 09 09:34:32.947626 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:32.947597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc" event={"ID":"fbaaef5e-87e5-4dae-a367-b10fc0d62031","Type":"ContainerStarted","Data":"222a7277c1cad33a7eb8d85d813fc45df2a0bc53a36e60620283f2ccfa901a8d"}
Jul 09 09:34:32.950399 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:32.950368    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7hkrk" event={"ID":"24feaef0-8968-4d52-86dd-3675636349ac","Type":"ContainerStarted","Data":"e9b27f692a8f01235e7fd53abdef9df18b422390110e509bbfc45c1874f0f25a"}
Jul 09 09:34:32.979328 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:32.979288    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-msxxf" podStartSLOduration=54.466134709 podStartE2EDuration="1m1.979272992s" totalImagesPullingTime="7.513138283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:32.978845064 +0000 UTC m=+75.985014162" watchObservedRunningTime="2026-07-09 09:34:32.979272992 +0000 UTC m=+75.985442080"
Jul 09 09:34:33.035179 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:33.035114    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7hkrk" podStartSLOduration=17.270914228 podStartE2EDuration="30.035090236s" totalImagesPullingTime="12.764176008s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:33.029557728 +0000 UTC m=+76.035726859" watchObservedRunningTime="2026-07-09 09:34:33.035090236 +0000 UTC m=+76.041259308"
Jul 09 09:34:33.224675 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:33.224608    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:33.334956 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:33.334917    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:33.428138 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:33.428108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:33.641636 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:33.641563    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:33.954970 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:33.954902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv" event={"ID":"960ec500-4be8-4e41-8f4b-cd8f20938c08","Type":"ContainerStarted","Data":"989c4218a117b534b22f8f29f12be510e652851743be59289143b3e2bdd94223"}
Jul 09 09:34:33.956302 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:33.956281    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dh44v" event={"ID":"2f492c23-72ed-4bbf-899d-3d5f588ef133","Type":"ContainerStarted","Data":"c6e82f826bc4361358b0d749d875166e0d79f3b8ba3bd3d27b8017b6297666d0"}
Jul 09 09:34:33.957892 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:33.957874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc" event={"ID":"fbaaef5e-87e5-4dae-a367-b10fc0d62031","Type":"ContainerStarted","Data":"f2e14e846936c6d71321de63f43acc8a1f25e549fdb272490119bee5bbd0a5c5"}
Jul 09 09:34:33.959572 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:33.959551    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-99ggf" event={"ID":"412bfb04-3abc-46f3-b9d3-76c904b79c2a","Type":"ContainerStarted","Data":"bed56de99fcd1c8eabfae6a97fef8a5a2edeb28cf72de655c9728ba8fc9cb421"}
Jul 09 09:34:33.959648 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:33.959588    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-99ggf" event={"ID":"412bfb04-3abc-46f3-b9d3-76c904b79c2a","Type":"ContainerStarted","Data":"1eedb0f3a9db33cd59f189d3139ec7349ca6a936679fbf1d861626d23547940b"}
Jul 09 09:34:33.976167 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:33.976122    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-gcwjv" podStartSLOduration=52.57175867 podStartE2EDuration="59.976108951s" totalImagesPullingTime="7.404350281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:33.974769669 +0000 UTC m=+76.980938739" watchObservedRunningTime="2026-07-09 09:34:33.976108951 +0000 UTC m=+76.982278016"
Jul 09 09:34:34.003324 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:34.003287    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-99ggf" podStartSLOduration=34.554497681 podStartE2EDuration="42.003274932s" totalImagesPullingTime="7.448777251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:34.002451827 +0000 UTC m=+77.008620898" watchObservedRunningTime="2026-07-09 09:34:34.003274932 +0000 UTC m=+77.009444002"
Jul 09 09:34:34.028218 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:34.028175    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dh44v" podStartSLOduration=65.800926175 podStartE2EDuration="1m15.028161281s" totalImagesPullingTime="9.227235106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:34.026977547 +0000 UTC m=+77.033146618" watchObservedRunningTime="2026-07-09 09:34:34.028161281 +0000 UTC m=+77.034330351"
Jul 09 09:34:34.051164 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:34.051123    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-sv5mc" podStartSLOduration=55.714118991 podStartE2EDuration="1m3.051110142s" totalImagesPullingTime="7.336991151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:34.050917527 +0000 UTC m=+77.057086599" watchObservedRunningTime="2026-07-09 09:34:34.051110142 +0000 UTC m=+77.057279255"
Jul 09 09:34:34.147317 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:34.147292    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:34.243298 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:34.243273    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:34.965530 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:34.965499    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-99ggf"
Jul 09 09:34:35.546246 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:35.546216    2577 scope.go:117] "RemoveContainer" containerID="40ecff50bdf5b0aa8138d071e90675f714767eb1c4e8a259433e5a42c2f586b6"
Jul 09 09:34:35.710180 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:35.710147    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:35.809844 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:35.809779    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:35.969680 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:35.969652    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:34:35.970129 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:35.969776    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4" event={"ID":"38d5ce80-3b12-4729-9e14-510a84a1e8d0","Type":"ContainerStarted","Data":"a8a53b754838e13c33de8ac9d761cbee9509fdaa8233b23c056960c457dc3cb8"}
Jul 09 09:34:35.970334 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:35.970315    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:34:35.975074 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:35.975057    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4"
Jul 09 09:34:35.995020 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:35.994975    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-ms4r4" podStartSLOduration=57.55230181 podStartE2EDuration="1m4.994957817s" totalImagesPullingTime="7.442656007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:35.991239028 +0000 UTC m=+78.997408103" watchObservedRunningTime="2026-07-09 09:34:35.994957817 +0000 UTC m=+79.001126889"
Jul 09 09:34:36.114318 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:36.114221    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:36.196310 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:36.196278    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:36.302187 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:36.302152    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:36.391839 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:36.391755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:36.564990 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:36.564962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:36.743106 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:36.743076    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:36.829313 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:36.829283    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:37.137922 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:37.137838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:37.239386 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:37.239358    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:37.951745 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:37.951705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:38.150371 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:38.150339    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:38.430744 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:38.430702    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:38.789119 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:38.789091    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:38.846442 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:38.846407    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:39.995270 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:39.995126    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:40.091547 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:40.091504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:40.527426 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:40.527397    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:40.628274 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:40.628242    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:40.827106 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:40.827037    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:41.316884 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:41.316856    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:41.419283 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:41.419254    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:41.517102 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:41.517078    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:41.651684 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:41.651622    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:41.751612 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:41.751589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:44.143467 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:44.143432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:44.876815 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:44.876777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:44.972634 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:44.972607    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-99ggf"
Jul 09 09:34:44.980745 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:44.980717    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:45.417996 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:45.417962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:45.572242 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:45.572212    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:46.631733 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:46.631683    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:46.852616 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:46.852581    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:46.927353 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:46.927289    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l47fk"]
Jul 09 09:34:46.931963 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:46.931944    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:46.934847 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:46.934822    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 09:34:46.934847 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:46.934823    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 09:34:46.934994 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:46.934871    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 09:34:46.934994 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:46.934938    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gclbm\""
Jul 09 09:34:46.935410 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:46.935397    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 09:34:46.945362 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:46.945343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:47.024152 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.024126    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:47.034263 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.034238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-wtmp\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.034343 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.034269    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-metrics-client-ca\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.034343 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.034287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-accelerators-collector-config\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.034343 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.034314    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-textfile\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.034439 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.034360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.034439 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.034431    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg9rz\" (UniqueName: \"kubernetes.io/projected/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-kube-api-access-vg9rz\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.034506 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.034455    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-root\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.034506 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.034479    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-tls\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.034566 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.034537    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-sys\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.135564 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.135542    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-sys\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.135682 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.135580    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-wtmp\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.135682 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.135596    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-metrics-client-ca\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.135682 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.135613    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-accelerators-collector-config\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.135682 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.135633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-textfile\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.135682 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.135659    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.135682 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.135667    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-sys\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.135991 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.135690    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg9rz\" (UniqueName: \"kubernetes.io/projected/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-kube-api-access-vg9rz\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.135991 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.135723    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-root\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.135991 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.135756    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-wtmp\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.135991 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.135761    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-tls\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.135991 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.135843    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-root\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.135991 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:47.135927    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 09 09:34:47.135991 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:34:47.135982    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-tls podName:e2de9b45-2629-4bc9-b5cc-e7553a1599f4 nodeName:}" failed. No retries permitted until 2026-07-09 09:34:47.635961935 +0000 UTC m=+90.642130998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-tls") pod "node-exporter-l47fk" (UID: "e2de9b45-2629-4bc9-b5cc-e7553a1599f4") : secret "node-exporter-tls" not found
Jul 09 09:34:47.136336 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.136123    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-textfile\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.136336 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.136278    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-accelerators-collector-config\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.136336 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.136324    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-metrics-client-ca\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.138170 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.138146    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.144518 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.144483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:47.145820 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.145789    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg9rz\" (UniqueName: \"kubernetes.io/projected/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-kube-api-access-vg9rz\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.248237 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.248206    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:47.343197 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.343170    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:47.638915 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.638859    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-tls\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.641006 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.640988    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2de9b45-2629-4bc9-b5cc-e7553a1599f4-node-exporter-tls\") pod \"node-exporter-l47fk\" (UID: \"e2de9b45-2629-4bc9-b5cc-e7553a1599f4\") " pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.840748 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.840723    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l47fk"
Jul 09 09:34:47.852429 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:34:47.852408    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2de9b45_2629_4bc9_b5cc_e7553a1599f4.slice/crio-70bdb93bdc32a1fe972cb1ab6f5c366714a539fd54fcd61f958050f1305372b3 WatchSource:0}: Error finding container 70bdb93bdc32a1fe972cb1ab6f5c366714a539fd54fcd61f958050f1305372b3: Status 404 returned error can't find the container with id 70bdb93bdc32a1fe972cb1ab6f5c366714a539fd54fcd61f958050f1305372b3
Jul 09 09:34:47.865632 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.865609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:47.923441 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.923382    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:34:47.980088 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:47.980057    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:48.014269 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:48.014243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l47fk" event={"ID":"e2de9b45-2629-4bc9-b5cc-e7553a1599f4","Type":"ContainerStarted","Data":"70bdb93bdc32a1fe972cb1ab6f5c366714a539fd54fcd61f958050f1305372b3"}
Jul 09 09:34:48.183082 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:48.182983    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:48.290189 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:48.290162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:48.590812 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:48.590760    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59df9f699f-s9nfn"]
Jul 09 09:34:48.705658 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:48.705625    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:48.801927 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:48.801903    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:48.987837 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:48.987780    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:49.019637 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:49.019603    2577 generic.go:356] "Generic (PLEG): container finished" podID="e2de9b45-2629-4bc9-b5cc-e7553a1599f4" containerID="bd2a38f48a5e82659234fec54149376540c7ffc1e1c1604b4df067b666820e6e" exitCode=0
Jul 09 09:34:49.019773 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:49.019652    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l47fk" event={"ID":"e2de9b45-2629-4bc9-b5cc-e7553a1599f4","Type":"ContainerDied","Data":"bd2a38f48a5e82659234fec54149376540c7ffc1e1c1604b4df067b666820e6e"}
Jul 09 09:34:49.058031 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:49.058010    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:49.995973 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:49.995938    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:50.026778 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:50.026749    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l47fk" event={"ID":"e2de9b45-2629-4bc9-b5cc-e7553a1599f4","Type":"ContainerStarted","Data":"db27fb083376850329de52b068350a5def47002eb7ecda1a3f4b555b22afc295"}
Jul 09 09:34:50.026920 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:50.026810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l47fk" event={"ID":"e2de9b45-2629-4bc9-b5cc-e7553a1599f4","Type":"ContainerStarted","Data":"1655f8e1f5367dd94d63b497a548c2d14e51825b36a28401aea8edc0bd6cc320"}
Jul 09 09:34:50.058098 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:50.058050    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l47fk" podStartSLOduration=3.048819392 podStartE2EDuration="4.058035009s" totalImagesPullingTime="945.948137ms" totalInitContainerRuntime="63.26748ms" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:50.057060205 +0000 UTC m=+93.063229275" watchObservedRunningTime="2026-07-09 09:34:50.058035009 +0000 UTC m=+93.064204083"
Jul 09 09:34:50.095419 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:50.095390    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:50.197119 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:50.197086    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:50.296370 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:50.296298    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:50.592565 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:50.592470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:51.645513 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:51.645479    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:51.844776 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:51.844741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:51.945712 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:51.945632    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:53.278192 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:53.278159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:54.417070 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:54.417036    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:54.608622 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:54.608574    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:54.871170 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:54.871143    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sjr8w"
Jul 09 09:34:55.398419 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.398390    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d9d8dd7f7-2dz5n"]
Jul 09 09:34:55.410270 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.410246    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.413314 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.413290    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 09 09:34:55.413432 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.413335    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 09:34:55.413492 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.413434    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qtbkd\""
Jul 09 09:34:55.414528 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.414508    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 09:34:55.414637 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.414557    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 09 09:34:55.414637 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.414594    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 09 09:34:55.414898 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.414881    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 09 09:34:55.415414 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.415390    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 09 09:34:55.416060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.416040    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d9d8dd7f7-2dz5n"]
Jul 09 09:34:55.421001 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.420981    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 09 09:34:55.495968 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.495947    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkvzh\" (UniqueName: \"kubernetes.io/projected/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-kube-api-access-fkvzh\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.496074 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.495982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-oauth-serving-cert\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.496074 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.496018    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-serving-cert\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.496074 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.496060    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-oauth-config\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.496228 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.496081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-service-ca\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.496228 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.496109    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-trusted-ca-bundle\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.496228 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.496167    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-config\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.596893 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.596867    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-oauth-config\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.597010 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.596906    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-service-ca\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.597010 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.596932    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-trusted-ca-bundle\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.597010 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.596961    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-config\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.597010 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.597005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkvzh\" (UniqueName: \"kubernetes.io/projected/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-kube-api-access-fkvzh\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.597215 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.597039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-oauth-serving-cert\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.597215 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.597095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-serving-cert\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.597674 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.597644    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-service-ca\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.597781 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.597656    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-config\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.597781 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.597761    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-oauth-serving-cert\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.597941 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.597922    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-trusted-ca-bundle\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.599788 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.599764    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-serving-cert\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.599907 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.599886    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-oauth-config\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.605706 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.605686    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkvzh\" (UniqueName: \"kubernetes.io/projected/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-kube-api-access-fkvzh\") pod \"console-5d9d8dd7f7-2dz5n\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") " pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.720701 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.720674    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:34:55.915603 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.915570    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d9d8dd7f7-2dz5n"]
Jul 09 09:34:55.918419 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:34:55.918386    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfb47c2_086b_4a2a_bc4e_f7569837cbc9.slice/crio-c20d06c0f703025199a0a60f89594c5dfb31dd7d38f9a28e38cb4a92eaffaf81 WatchSource:0}: Error finding container c20d06c0f703025199a0a60f89594c5dfb31dd7d38f9a28e38cb4a92eaffaf81: Status 404 returned error can't find the container with id c20d06c0f703025199a0a60f89594c5dfb31dd7d38f9a28e38cb4a92eaffaf81
Jul 09 09:34:55.980239 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:55.980172    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:56.047691 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:56.047664    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9d8dd7f7-2dz5n" event={"ID":"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9","Type":"ContainerStarted","Data":"c20d06c0f703025199a0a60f89594c5dfb31dd7d38f9a28e38cb4a92eaffaf81"}
Jul 09 09:34:56.080907 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:56.080878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:56.204914 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:56.204876    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:56.328481 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:56.328400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:59.731414 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:59.731380    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:34:59.795827 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:34:59.795788    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:35:00.063315 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:00.063243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9d8dd7f7-2dz5n" event={"ID":"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9","Type":"ContainerStarted","Data":"db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919"}
Jul 09 09:35:00.083945 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:00.083902    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d9d8dd7f7-2dz5n" podStartSLOduration=1.728774945 podStartE2EDuration="5.083888463s" totalImagesPullingTime="3.355113518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:00.083311596 +0000 UTC m=+103.089480686" watchObservedRunningTime="2026-07-09 09:35:00.083888463 +0000 UTC m=+103.090057579"
Jul 09 09:35:03.132652 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:03.132619    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:35:03.233347 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:03.233316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:35:04.360683 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:04.360646    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:35:04.541235 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:04.541206    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:35:05.721464 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:05.721437    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:35:05.722012 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:05.721493    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:35:05.726183 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:05.726162    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:35:06.089828 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:06.089761    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:35:06.465382 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:06.465351    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:35:06.566390 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:06.566360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:35:08.095057 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:08.094992    2577 generic.go:356] "Generic (PLEG): container finished" podID="c0f6b837-c192-44ae-8c7f-02ee427acafc" containerID="73a31507486862d47221fa2ecc7d1b94d075e1d71c1c70816fe6845ab43b0deb" exitCode=0
Jul 09 09:35:08.095370 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:08.095065    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-d5888" event={"ID":"c0f6b837-c192-44ae-8c7f-02ee427acafc","Type":"ContainerDied","Data":"73a31507486862d47221fa2ecc7d1b94d075e1d71c1c70816fe6845ab43b0deb"}
Jul 09 09:35:08.095441 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:08.095422    2577 scope.go:117] "RemoveContainer" containerID="73a31507486862d47221fa2ecc7d1b94d075e1d71c1c70816fe6845ab43b0deb"
Jul 09 09:35:09.100923 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:09.100888    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-d5888" event={"ID":"c0f6b837-c192-44ae-8c7f-02ee427acafc","Type":"ContainerStarted","Data":"296346e1e8a4050ee60ed28ca15f29f338bde3e2269c52c324756fc8b0615dd2"}
Jul 09 09:35:09.846624 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:09.846594    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:35:09.946726 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:09.946692    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:35:10.143459 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:10.143382    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-msxxf_38bfa326-e79b-447e-ae2b-c31b088b6249/cluster-monitoring-operator/0.log"
Jul 09 09:35:12.539513 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:12.539483    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l47fk_e2de9b45-2629-4bc9-b5cc-e7553a1599f4/init-textfile/0.log"
Jul 09 09:35:12.741317 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:12.741283    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l47fk_e2de9b45-2629-4bc9-b5cc-e7553a1599f4/node-exporter/0.log"
Jul 09 09:35:12.940034 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:12.939964    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l47fk_e2de9b45-2629-4bc9-b5cc-e7553a1599f4/kube-rbac-proxy/0.log"
Jul 09 09:35:13.612949 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.612905    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-59df9f699f-s9nfn" podUID="7365b00b-2bda-4d1e-8b42-d83097184402" containerName="registry" containerID="cri-o://9794c2c6650da1128a20c59259d8f45177cf52e9e2e0bb0b2e19912dad46bcf2" gracePeriod=30
Jul 09 09:35:13.848072 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.848051    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:35:13.926534 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.926466    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7365b00b-2bda-4d1e-8b42-d83097184402-registry-certificates\") pod \"7365b00b-2bda-4d1e-8b42-d83097184402\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") "
Jul 09 09:35:13.926534 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.926525    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7365b00b-2bda-4d1e-8b42-d83097184402-ca-trust-extracted\") pod \"7365b00b-2bda-4d1e-8b42-d83097184402\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") "
Jul 09 09:35:13.926679 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.926555    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls\") pod \"7365b00b-2bda-4d1e-8b42-d83097184402\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") "
Jul 09 09:35:13.926679 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.926595    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7365b00b-2bda-4d1e-8b42-d83097184402-installation-pull-secrets\") pod \"7365b00b-2bda-4d1e-8b42-d83097184402\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") "
Jul 09 09:35:13.926679 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.926634    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh4w8\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-kube-api-access-vh4w8\") pod \"7365b00b-2bda-4d1e-8b42-d83097184402\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") "
Jul 09 09:35:13.926817 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.926759    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7365b00b-2bda-4d1e-8b42-d83097184402-trusted-ca\") pod \"7365b00b-2bda-4d1e-8b42-d83097184402\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") "
Jul 09 09:35:13.926875 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.926829    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7365b00b-2bda-4d1e-8b42-d83097184402-image-registry-private-configuration\") pod \"7365b00b-2bda-4d1e-8b42-d83097184402\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") "
Jul 09 09:35:13.926875 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.926869    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-bound-sa-token\") pod \"7365b00b-2bda-4d1e-8b42-d83097184402\" (UID: \"7365b00b-2bda-4d1e-8b42-d83097184402\") "
Jul 09 09:35:13.926975 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.926921    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7365b00b-2bda-4d1e-8b42-d83097184402-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7365b00b-2bda-4d1e-8b42-d83097184402" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:35:13.927149 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.927123    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7365b00b-2bda-4d1e-8b42-d83097184402-registry-certificates\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:35:13.927266 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.927224    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7365b00b-2bda-4d1e-8b42-d83097184402-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7365b00b-2bda-4d1e-8b42-d83097184402" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:35:13.929175 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.929145    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7365b00b-2bda-4d1e-8b42-d83097184402-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7365b00b-2bda-4d1e-8b42-d83097184402" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:35:13.929281 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.929200    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-kube-api-access-vh4w8" (OuterVolumeSpecName: "kube-api-access-vh4w8") pod "7365b00b-2bda-4d1e-8b42-d83097184402" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402"). InnerVolumeSpecName "kube-api-access-vh4w8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:35:13.929341 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.929278    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7365b00b-2bda-4d1e-8b42-d83097184402" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:35:13.929396 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.929355    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7365b00b-2bda-4d1e-8b42-d83097184402" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:35:13.929396 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.929373    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7365b00b-2bda-4d1e-8b42-d83097184402-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7365b00b-2bda-4d1e-8b42-d83097184402" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:35:13.935162 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:13.935140    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7365b00b-2bda-4d1e-8b42-d83097184402-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7365b00b-2bda-4d1e-8b42-d83097184402" (UID: "7365b00b-2bda-4d1e-8b42-d83097184402"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:35:14.027619 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.027598    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh4w8\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-kube-api-access-vh4w8\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:35:14.027619 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.027618    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7365b00b-2bda-4d1e-8b42-d83097184402-trusted-ca\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:35:14.027733 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.027629    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7365b00b-2bda-4d1e-8b42-d83097184402-image-registry-private-configuration\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:35:14.027733 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.027638    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-bound-sa-token\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:35:14.027733 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.027646    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7365b00b-2bda-4d1e-8b42-d83097184402-ca-trust-extracted\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:35:14.027733 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.027654    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7365b00b-2bda-4d1e-8b42-d83097184402-registry-tls\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:35:14.027733 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.027662    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7365b00b-2bda-4d1e-8b42-d83097184402-installation-pull-secrets\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:35:14.119807 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.119769    2577 generic.go:356] "Generic (PLEG): container finished" podID="7365b00b-2bda-4d1e-8b42-d83097184402" containerID="9794c2c6650da1128a20c59259d8f45177cf52e9e2e0bb0b2e19912dad46bcf2" exitCode=0
Jul 09 09:35:14.119914 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.119833    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59df9f699f-s9nfn"
Jul 09 09:35:14.119914 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.119858    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59df9f699f-s9nfn" event={"ID":"7365b00b-2bda-4d1e-8b42-d83097184402","Type":"ContainerDied","Data":"9794c2c6650da1128a20c59259d8f45177cf52e9e2e0bb0b2e19912dad46bcf2"}
Jul 09 09:35:14.120025 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.119916    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59df9f699f-s9nfn" event={"ID":"7365b00b-2bda-4d1e-8b42-d83097184402","Type":"ContainerDied","Data":"aa72be9d71428d48a0cae59ea3a5333deb6f2b990725329e7f426037d9677adf"}
Jul 09 09:35:14.120025 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.119939    2577 scope.go:117] "RemoveContainer" containerID="9794c2c6650da1128a20c59259d8f45177cf52e9e2e0bb0b2e19912dad46bcf2"
Jul 09 09:35:14.128009 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.127991    2577 scope.go:117] "RemoveContainer" containerID="9794c2c6650da1128a20c59259d8f45177cf52e9e2e0bb0b2e19912dad46bcf2"
Jul 09 09:35:14.128256 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:35:14.128235    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9794c2c6650da1128a20c59259d8f45177cf52e9e2e0bb0b2e19912dad46bcf2\": container with ID starting with 9794c2c6650da1128a20c59259d8f45177cf52e9e2e0bb0b2e19912dad46bcf2 not found: ID does not exist" containerID="9794c2c6650da1128a20c59259d8f45177cf52e9e2e0bb0b2e19912dad46bcf2"
Jul 09 09:35:14.128330 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.128269    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9794c2c6650da1128a20c59259d8f45177cf52e9e2e0bb0b2e19912dad46bcf2"} err="failed to get container status \"9794c2c6650da1128a20c59259d8f45177cf52e9e2e0bb0b2e19912dad46bcf2\": rpc error: code = NotFound desc = could not find container \"9794c2c6650da1128a20c59259d8f45177cf52e9e2e0bb0b2e19912dad46bcf2\": container with ID starting with 9794c2c6650da1128a20c59259d8f45177cf52e9e2e0bb0b2e19912dad46bcf2 not found: ID does not exist"
Jul 09 09:35:14.141842 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.141819    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59df9f699f-s9nfn"]
Jul 09 09:35:14.146028 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.146009    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-59df9f699f-s9nfn"]
Jul 09 09:35:14.705452 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.705420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:35:14.774275 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:14.774234    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:35:15.550501 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:15.550465    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7365b00b-2bda-4d1e-8b42-d83097184402" path="/var/lib/kubelet/pods/7365b00b-2bda-4d1e-8b42-d83097184402/volumes"
Jul 09 09:35:16.941619 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:16.941587    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-gcwjv_960ec500-4be8-4e41-8f4b-cd8f20938c08/networking-console-plugin/0.log"
Jul 09 09:35:17.139184 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:17.139157    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:35:17.342295 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:17.342257    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/3.log"
Jul 09 09:35:17.739554 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:17.739513    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d9d8dd7f7-2dz5n_5dfb47c2-086b-4a2a-bc4e-f7569837cbc9/console/0.log"
Jul 09 09:35:22.146024 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:22.145988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:35:24.159310 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:24.159271    2577 generic.go:356] "Generic (PLEG): container finished" podID="24c5f723-d578-4374-8f99-2a7fa1f38727" containerID="5c0f026767e63c1f983ccbbaf8b8d4462264c4c091e315837be7765fd79aff1f" exitCode=0
Jul 09 09:35:24.159692 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:24.159354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs" event={"ID":"24c5f723-d578-4374-8f99-2a7fa1f38727","Type":"ContainerDied","Data":"5c0f026767e63c1f983ccbbaf8b8d4462264c4c091e315837be7765fd79aff1f"}
Jul 09 09:35:24.159692 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:24.159657    2577 scope.go:117] "RemoveContainer" containerID="5c0f026767e63c1f983ccbbaf8b8d4462264c4c091e315837be7765fd79aff1f"
Jul 09 09:35:25.164533 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:25.164498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-f2zvs" event={"ID":"24c5f723-d578-4374-8f99-2a7fa1f38727","Type":"ContainerStarted","Data":"a3eb2f76698af905297c09fc85404f3fdd3e13606f9b6e22c39fef0a88591ff3"}
Jul 09 09:35:33.199336 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:33.199295    2577 generic.go:356] "Generic (PLEG): container finished" podID="d5ac7be9-438f-473e-814d-c2a998a389f8" containerID="11c55bfafd80399370505ad3a91cf29ea5ee1d68c781bd119656a44d48258338" exitCode=0
Jul 09 09:35:33.199729 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:33.199370    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d" event={"ID":"d5ac7be9-438f-473e-814d-c2a998a389f8","Type":"ContainerDied","Data":"11c55bfafd80399370505ad3a91cf29ea5ee1d68c781bd119656a44d48258338"}
Jul 09 09:35:33.199777 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:33.199724    2577 scope.go:117] "RemoveContainer" containerID="11c55bfafd80399370505ad3a91cf29ea5ee1d68c781bd119656a44d48258338"
Jul 09 09:35:34.205038 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:35:34.204996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g6r9d" event={"ID":"d5ac7be9-438f-473e-814d-c2a998a389f8","Type":"ContainerStarted","Data":"0a86343fd24a424207f40f9f974ef7adaa18e6b7c7214e13937ada161284da71"}
Jul 09 09:36:06.317334 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:06.317303    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:36:06.517875 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:06.517840    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:36:11.400863 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.400772    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5d86454dd9-87pl2"]
Jul 09 09:36:11.401312 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.401081    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7365b00b-2bda-4d1e-8b42-d83097184402" containerName="registry"
Jul 09 09:36:11.401312 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.401095    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7365b00b-2bda-4d1e-8b42-d83097184402" containerName="registry"
Jul 09 09:36:11.401312 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.401167    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7365b00b-2bda-4d1e-8b42-d83097184402" containerName="registry"
Jul 09 09:36:11.404492 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.404470    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.407848 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.407822    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 09 09:36:11.407848 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.407837    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 09 09:36:11.408050 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.407891    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 09 09:36:11.408050 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.407913    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 09 09:36:11.408050 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.407832    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8jhcb\""
Jul 09 09:36:11.408050 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.407830    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 09 09:36:11.412455 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.412432    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 09 09:36:11.423641 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.423613    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d86454dd9-87pl2"]
Jul 09 09:36:11.527906 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.527881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89f07c10-f820-4636-9e73-1c71d475c421-metrics-client-ca\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.528015 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.527915    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89f07c10-f820-4636-9e73-1c71d475c421-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.528015 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.527948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/89f07c10-f820-4636-9e73-1c71d475c421-secret-telemeter-client\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.528015 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.527975    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89f07c10-f820-4636-9e73-1c71d475c421-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.528015 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.527995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/89f07c10-f820-4636-9e73-1c71d475c421-telemeter-client-tls\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.528015 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.528011    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89f07c10-f820-4636-9e73-1c71d475c421-serving-certs-ca-bundle\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.528243 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.528114    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/89f07c10-f820-4636-9e73-1c71d475c421-federate-client-tls\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.528243 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.528152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlxkg\" (UniqueName: \"kubernetes.io/projected/89f07c10-f820-4636-9e73-1c71d475c421-kube-api-access-vlxkg\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.629150 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.629126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89f07c10-f820-4636-9e73-1c71d475c421-metrics-client-ca\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.629249 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.629157    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89f07c10-f820-4636-9e73-1c71d475c421-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.629249 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.629182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/89f07c10-f820-4636-9e73-1c71d475c421-secret-telemeter-client\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.629249 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.629199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89f07c10-f820-4636-9e73-1c71d475c421-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.629249 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.629218    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/89f07c10-f820-4636-9e73-1c71d475c421-telemeter-client-tls\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.629249 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.629233    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89f07c10-f820-4636-9e73-1c71d475c421-serving-certs-ca-bundle\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.629498 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.629268    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/89f07c10-f820-4636-9e73-1c71d475c421-federate-client-tls\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.629498 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.629291    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlxkg\" (UniqueName: \"kubernetes.io/projected/89f07c10-f820-4636-9e73-1c71d475c421-kube-api-access-vlxkg\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.629987 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.629962    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89f07c10-f820-4636-9e73-1c71d475c421-metrics-client-ca\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.630088 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.630023    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89f07c10-f820-4636-9e73-1c71d475c421-serving-certs-ca-bundle\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.630088 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.630081    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89f07c10-f820-4636-9e73-1c71d475c421-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.632329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.632296    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/89f07c10-f820-4636-9e73-1c71d475c421-telemeter-client-tls\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.632414 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.632326    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/89f07c10-f820-4636-9e73-1c71d475c421-federate-client-tls\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.632414 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.632344    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/89f07c10-f820-4636-9e73-1c71d475c421-secret-telemeter-client\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.632414 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.632388    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89f07c10-f820-4636-9e73-1c71d475c421-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.637572 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.637554    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlxkg\" (UniqueName: \"kubernetes.io/projected/89f07c10-f820-4636-9e73-1c71d475c421-kube-api-access-vlxkg\") pod \"telemeter-client-5d86454dd9-87pl2\" (UID: \"89f07c10-f820-4636-9e73-1c71d475c421\") " pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.716100 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.716080    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2"
Jul 09 09:36:11.850365 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:11.850332    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d86454dd9-87pl2"]
Jul 09 09:36:11.853359 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:36:11.853333    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89f07c10_f820_4636_9e73_1c71d475c421.slice/crio-340391e7a52e97d2f185206dc4bf05f6cf3d2923c0ed0456964eed409b40fbdc WatchSource:0}: Error finding container 340391e7a52e97d2f185206dc4bf05f6cf3d2923c0ed0456964eed409b40fbdc: Status 404 returned error can't find the container with id 340391e7a52e97d2f185206dc4bf05f6cf3d2923c0ed0456964eed409b40fbdc
Jul 09 09:36:12.340316 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:12.340278    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2" event={"ID":"89f07c10-f820-4636-9e73-1c71d475c421","Type":"ContainerStarted","Data":"340391e7a52e97d2f185206dc4bf05f6cf3d2923c0ed0456964eed409b40fbdc"}
Jul 09 09:36:14.348750 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:14.348720    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2" event={"ID":"89f07c10-f820-4636-9e73-1c71d475c421","Type":"ContainerStarted","Data":"961c7a9607add346795dd032164fbd1b2bdfc5de730030582cd75c455cc4ee8e"}
Jul 09 09:36:16.359060 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:16.359030    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2" event={"ID":"89f07c10-f820-4636-9e73-1c71d475c421","Type":"ContainerStarted","Data":"1818df544045458b7dafeab323673447a97b24ebe8f6a7842185bf173d598141"}
Jul 09 09:36:16.359421 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:16.359088    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2" event={"ID":"89f07c10-f820-4636-9e73-1c71d475c421","Type":"ContainerStarted","Data":"5878f30617edd4dda94095a6debc5be33f0c75c188c55133a9e205db5e16be3c"}
Jul 09 09:36:16.383867 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:16.381882    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5d86454dd9-87pl2" podStartSLOduration=1.8449142859999998 podStartE2EDuration="5.381858113s" totalImagesPullingTime="3.536943827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:36:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:36:16.381415634 +0000 UTC m=+179.387584688" watchObservedRunningTime="2026-07-09 09:36:16.381858113 +0000 UTC m=+179.388027185"
Jul 09 09:36:17.094270 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.094236    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b8c5b4b49-pw8df"]
Jul 09 09:36:17.098073 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.098051    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.113542 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.113521    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b8c5b4b49-pw8df"]
Jul 09 09:36:17.156392 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.156368    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:36:17.222028 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.221998    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:36:17.268013 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.267989    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/850f22de-08c0-4bc5-bb0e-9668e50de116-console-serving-cert\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.268115 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.268015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-trusted-ca-bundle\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.268115 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.268034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-console-config\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.268115 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.268061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/850f22de-08c0-4bc5-bb0e-9668e50de116-console-oauth-config\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.268241 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.268137    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjbzg\" (UniqueName: \"kubernetes.io/projected/850f22de-08c0-4bc5-bb0e-9668e50de116-kube-api-access-jjbzg\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.268241 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.268190    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-service-ca\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.268241 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.268208    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-oauth-serving-cert\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.300506 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.300481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:36:17.368964 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.368904    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/850f22de-08c0-4bc5-bb0e-9668e50de116-console-oauth-config\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.368964 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.368944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjbzg\" (UniqueName: \"kubernetes.io/projected/850f22de-08c0-4bc5-bb0e-9668e50de116-kube-api-access-jjbzg\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.369338 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.368989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-service-ca\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.369338 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.369005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-oauth-serving-cert\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.369338 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.369028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/850f22de-08c0-4bc5-bb0e-9668e50de116-console-serving-cert\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.369338 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.369042    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-trusted-ca-bundle\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.369338 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.369061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-console-config\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.369814 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.369770    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-service-ca\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.369814 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.369789    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-oauth-serving-cert\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.369922 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.369905    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-console-config\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.370130 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.370112    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-trusted-ca-bundle\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.371771 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.371757    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/850f22de-08c0-4bc5-bb0e-9668e50de116-console-oauth-config\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.372036 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.372016    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/850f22de-08c0-4bc5-bb0e-9668e50de116-console-serving-cert\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.376619 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.376600    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjbzg\" (UniqueName: \"kubernetes.io/projected/850f22de-08c0-4bc5-bb0e-9668e50de116-kube-api-access-jjbzg\") pod \"console-5b8c5b4b49-pw8df\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") " pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.401476 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.401446    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:36:17.408473 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.408451    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:17.548428 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:36:17.548398    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod850f22de_08c0_4bc5_bb0e_9668e50de116.slice/crio-3b3577e95dfe5adb394c370b84066ca7c8e0932012d9ce479aa8b634353e8e62 WatchSource:0}: Error finding container 3b3577e95dfe5adb394c370b84066ca7c8e0932012d9ce479aa8b634353e8e62: Status 404 returned error can't find the container with id 3b3577e95dfe5adb394c370b84066ca7c8e0932012d9ce479aa8b634353e8e62
Jul 09 09:36:17.550974 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.550954    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b8c5b4b49-pw8df"]
Jul 09 09:36:17.905334 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.905296    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:36:17.997491 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:17.997467    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:36:18.371106 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:18.371075    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8c5b4b49-pw8df" event={"ID":"850f22de-08c0-4bc5-bb0e-9668e50de116","Type":"ContainerStarted","Data":"eab2a94893187996f9df505a97a22f3791a29b23c7c560ba29fecd0621ca4dab"}
Jul 09 09:36:18.371466 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:18.371128    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8c5b4b49-pw8df" event={"ID":"850f22de-08c0-4bc5-bb0e-9668e50de116","Type":"ContainerStarted","Data":"3b3577e95dfe5adb394c370b84066ca7c8e0932012d9ce479aa8b634353e8e62"}
Jul 09 09:36:18.390638 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:18.390571    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b8c5b4b49-pw8df" podStartSLOduration=1.3905568719999999 podStartE2EDuration="1.390556872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:36:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:36:18.390080456 +0000 UTC m=+181.396249547" watchObservedRunningTime="2026-07-09 09:36:18.390556872 +0000 UTC m=+181.396725942"
Jul 09 09:36:27.408579 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:27.408544    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:27.408990 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:27.408600    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:27.412983 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:27.412959    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:28.412163 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:28.412134    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:36:28.468341 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:28.468311    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d9d8dd7f7-2dz5n"]
Jul 09 09:36:28.579273 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:28.579240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:36:28.642036 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:28.642001    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:36:28.708170 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:28.708105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:36:39.346039 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:39.346004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:36:39.408990 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:39.408959    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:36:53.492962 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.492900    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5d9d8dd7f7-2dz5n" podUID="5dfb47c2-086b-4a2a-bc4e-f7569837cbc9" containerName="console" containerID="cri-o://db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919" gracePeriod=15
Jul 09 09:36:53.516038 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:36:53.516010    2577 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfb47c2_086b_4a2a_bc4e_f7569837cbc9.slice/crio-db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfb47c2_086b_4a2a_bc4e_f7569837cbc9.slice/crio-conmon-db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919.scope\": RecentStats: unable to find data in memory cache]"
Jul 09 09:36:53.516547 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:36:53.516522    2577 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfb47c2_086b_4a2a_bc4e_f7569837cbc9.slice/crio-db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfb47c2_086b_4a2a_bc4e_f7569837cbc9.slice/crio-conmon-db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919.scope\": RecentStats: unable to find data in memory cache]"
Jul 09 09:36:53.727527 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.727503    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:36:53.832502 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.832444    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-serving-cert\") pod \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") "
Jul 09 09:36:53.832502 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.832473    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-trusted-ca-bundle\") pod \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") "
Jul 09 09:36:53.832645 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.832502    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-oauth-serving-cert\") pod \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") "
Jul 09 09:36:53.832645 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.832537    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-service-ca\") pod \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") "
Jul 09 09:36:53.832645 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.832582    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-oauth-config\") pod \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") "
Jul 09 09:36:53.832645 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.832642    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkvzh\" (UniqueName: \"kubernetes.io/projected/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-kube-api-access-fkvzh\") pod \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") "
Jul 09 09:36:53.832834 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.832680    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-config\") pod \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\" (UID: \"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9\") "
Jul 09 09:36:53.832958 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.832924    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9" (UID: "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:36:53.833052 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.832969    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9" (UID: "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:36:53.833052 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.832974    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-service-ca" (OuterVolumeSpecName: "service-ca") pod "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9" (UID: "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:36:53.833249 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.833218    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-config" (OuterVolumeSpecName: "console-config") pod "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9" (UID: "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:36:53.834663 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.834639    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9" (UID: "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:53.834779 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.834758    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-kube-api-access-fkvzh" (OuterVolumeSpecName: "kube-api-access-fkvzh") pod "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9" (UID: "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9"). InnerVolumeSpecName "kube-api-access-fkvzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:36:53.834846 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.834788    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9" (UID: "5dfb47c2-086b-4a2a-bc4e-f7569837cbc9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:53.933232 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.933208    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-oauth-config\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:36:53.933232 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.933229    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkvzh\" (UniqueName: \"kubernetes.io/projected/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-kube-api-access-fkvzh\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:36:53.933339 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.933240    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-config\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:36:53.933339 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.933249    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-console-serving-cert\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:36:53.933339 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.933257    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-trusted-ca-bundle\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:36:53.933339 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.933265    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-oauth-serving-cert\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:36:53.933339 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:53.933273    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9-service-ca\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:36:54.503966 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:54.503938    2577 generic.go:356] "Generic (PLEG): container finished" podID="5dfb47c2-086b-4a2a-bc4e-f7569837cbc9" containerID="db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919" exitCode=0
Jul 09 09:36:54.504296 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:54.503979    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9d8dd7f7-2dz5n" event={"ID":"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9","Type":"ContainerDied","Data":"db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919"}
Jul 09 09:36:54.504296 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:54.504017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9d8dd7f7-2dz5n" event={"ID":"5dfb47c2-086b-4a2a-bc4e-f7569837cbc9","Type":"ContainerDied","Data":"c20d06c0f703025199a0a60f89594c5dfb31dd7d38f9a28e38cb4a92eaffaf81"}
Jul 09 09:36:54.504296 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:54.504036    2577 scope.go:117] "RemoveContainer" containerID="db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919"
Jul 09 09:36:54.504296 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:54.503981    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9d8dd7f7-2dz5n"
Jul 09 09:36:54.512994 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:54.512974    2577 scope.go:117] "RemoveContainer" containerID="db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919"
Jul 09 09:36:54.513231 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:36:54.513213    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919\": container with ID starting with db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919 not found: ID does not exist" containerID="db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919"
Jul 09 09:36:54.513290 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:54.513243    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919"} err="failed to get container status \"db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919\": rpc error: code = NotFound desc = could not find container \"db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919\": container with ID starting with db31f401cd75c61df20431000f598dbfcbddd5e50cb2411ca950533c4c2ac919 not found: ID does not exist"
Jul 09 09:36:54.525647 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:54.525617    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d9d8dd7f7-2dz5n"]
Jul 09 09:36:54.529425 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:54.529404    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d9d8dd7f7-2dz5n"]
Jul 09 09:36:55.549934 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:36:55.549904    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dfb47c2-086b-4a2a-bc4e-f7569837cbc9" path="/var/lib/kubelet/pods/5dfb47c2-086b-4a2a-bc4e-f7569837cbc9/volumes"
Jul 09 09:37:08.003049 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:08.003008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:21.552673 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:21.552633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:21.611322 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:21.611277    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:23.262035 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.262003    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-849f748b57-hlgbt"]
Jul 09 09:37:23.262387 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.262266    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dfb47c2-086b-4a2a-bc4e-f7569837cbc9" containerName="console"
Jul 09 09:37:23.262387 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.262280    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dfb47c2-086b-4a2a-bc4e-f7569837cbc9" containerName="console"
Jul 09 09:37:23.262387 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.262331    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dfb47c2-086b-4a2a-bc4e-f7569837cbc9" containerName="console"
Jul 09 09:37:23.266487 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.266463    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.278946 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.278919    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-849f748b57-hlgbt"]
Jul 09 09:37:23.319580 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.319555    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:23.374188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.374162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:23.429601 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.429576    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-console-config\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.429716 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.429608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-service-ca\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.429716 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.429636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-oauth-serving-cert\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.429716 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.429685    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-trusted-ca-bundle\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.429876 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.429729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d3e379c-7887-4e62-ad65-951437849a85-console-serving-cert\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.429876 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.429760    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d3e379c-7887-4e62-ad65-951437849a85-console-oauth-config\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.429876 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.429783    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7xn4\" (UniqueName: \"kubernetes.io/projected/6d3e379c-7887-4e62-ad65-951437849a85-kube-api-access-r7xn4\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.458814 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.458775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:23.513498 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.513436    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:23.530147 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.530125    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7xn4\" (UniqueName: \"kubernetes.io/projected/6d3e379c-7887-4e62-ad65-951437849a85-kube-api-access-r7xn4\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.530220 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.530158    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-console-config\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.530220 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.530179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-service-ca\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.530315 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.530300    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-oauth-serving-cert\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.530370 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.530327    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-trusted-ca-bundle\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.530370 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.530358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d3e379c-7887-4e62-ad65-951437849a85-console-serving-cert\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.530464 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.530401    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d3e379c-7887-4e62-ad65-951437849a85-console-oauth-config\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.530957 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.530935    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-console-config\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.530957 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.530950    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-oauth-serving-cert\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.531135 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.530942    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-service-ca\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.531274 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.531249    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-trusted-ca-bundle\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.532789 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.532762    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d3e379c-7887-4e62-ad65-951437849a85-console-serving-cert\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.532789 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.532772    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d3e379c-7887-4e62-ad65-951437849a85-console-oauth-config\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.538253 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.538231    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7xn4\" (UniqueName: \"kubernetes.io/projected/6d3e379c-7887-4e62-ad65-951437849a85-kube-api-access-r7xn4\") pod \"console-849f748b57-hlgbt\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") " pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.575731 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.575709    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:23.712607 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:23.712573    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-849f748b57-hlgbt"]
Jul 09 09:37:23.715664 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:37:23.715639    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d3e379c_7887_4e62_ad65_951437849a85.slice/crio-2e006e7a3606ea498f84e99b88a7e926e5fb72b528bf6af6907a84d814717969 WatchSource:0}: Error finding container 2e006e7a3606ea498f84e99b88a7e926e5fb72b528bf6af6907a84d814717969: Status 404 returned error can't find the container with id 2e006e7a3606ea498f84e99b88a7e926e5fb72b528bf6af6907a84d814717969
Jul 09 09:37:24.064539 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:24.064455    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:24.165117 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:24.165091    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:24.615054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:24.615022    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-849f748b57-hlgbt" event={"ID":"6d3e379c-7887-4e62-ad65-951437849a85","Type":"ContainerStarted","Data":"05d261a7ddfa046ed6dc38da5d409436bfe598966658a3fec4043ebb8d6a5c96"}
Jul 09 09:37:24.615444 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:24.615102    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-849f748b57-hlgbt" event={"ID":"6d3e379c-7887-4e62-ad65-951437849a85","Type":"ContainerStarted","Data":"2e006e7a3606ea498f84e99b88a7e926e5fb72b528bf6af6907a84d814717969"}
Jul 09 09:37:24.633855 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:24.633812    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-849f748b57-hlgbt" podStartSLOduration=1.633782217 podStartE2EDuration="1.633782217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:37:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:37:24.632521207 +0000 UTC m=+247.638690301" watchObservedRunningTime="2026-07-09 09:37:24.633782217 +0000 UTC m=+247.639951287"
Jul 09 09:37:33.480155 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:33.480122    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:33.576507 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:33.576485    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:33.576656 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:33.576532    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:33.580994 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:33.580961    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:33.581347 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:33.581324    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:33.649539 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:33.649517    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:37:33.694385 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:33.694354    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b8c5b4b49-pw8df"]
Jul 09 09:37:33.815906 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:33.815814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:33.916313 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:33.916280    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:33.979689 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:33.979660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:44.729284 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:44.729195    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:46.781969 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:46.781925    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:47.313886 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:47.313842    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:47.369966 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:47.369931    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:37:58.719121 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.719061    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5b8c5b4b49-pw8df" podUID="850f22de-08c0-4bc5-bb0e-9668e50de116" containerName="console" containerID="cri-o://eab2a94893187996f9df505a97a22f3791a29b23c7c560ba29fecd0621ca4dab" gracePeriod=15
Jul 09 09:37:58.956046 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.956022    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:37:58.972333 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.972282    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-console-config\") pod \"850f22de-08c0-4bc5-bb0e-9668e50de116\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") "
Jul 09 09:37:58.972333 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.972309    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-service-ca\") pod \"850f22de-08c0-4bc5-bb0e-9668e50de116\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") "
Jul 09 09:37:58.972484 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.972362    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/850f22de-08c0-4bc5-bb0e-9668e50de116-console-serving-cert\") pod \"850f22de-08c0-4bc5-bb0e-9668e50de116\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") "
Jul 09 09:37:58.972484 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.972381    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/850f22de-08c0-4bc5-bb0e-9668e50de116-console-oauth-config\") pod \"850f22de-08c0-4bc5-bb0e-9668e50de116\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") "
Jul 09 09:37:58.972484 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.972402    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-trusted-ca-bundle\") pod \"850f22de-08c0-4bc5-bb0e-9668e50de116\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") "
Jul 09 09:37:58.972484 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.972435    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-oauth-serving-cert\") pod \"850f22de-08c0-4bc5-bb0e-9668e50de116\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") "
Jul 09 09:37:58.972484 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.972469    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjbzg\" (UniqueName: \"kubernetes.io/projected/850f22de-08c0-4bc5-bb0e-9668e50de116-kube-api-access-jjbzg\") pod \"850f22de-08c0-4bc5-bb0e-9668e50de116\" (UID: \"850f22de-08c0-4bc5-bb0e-9668e50de116\") "
Jul 09 09:37:58.973162 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.972826    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-console-config" (OuterVolumeSpecName: "console-config") pod "850f22de-08c0-4bc5-bb0e-9668e50de116" (UID: "850f22de-08c0-4bc5-bb0e-9668e50de116"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:37:58.973162 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.972924    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-service-ca" (OuterVolumeSpecName: "service-ca") pod "850f22de-08c0-4bc5-bb0e-9668e50de116" (UID: "850f22de-08c0-4bc5-bb0e-9668e50de116"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:37:58.973162 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.973023    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "850f22de-08c0-4bc5-bb0e-9668e50de116" (UID: "850f22de-08c0-4bc5-bb0e-9668e50de116"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:37:58.973162 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.973121    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "850f22de-08c0-4bc5-bb0e-9668e50de116" (UID: "850f22de-08c0-4bc5-bb0e-9668e50de116"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:37:58.974505 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.974475    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850f22de-08c0-4bc5-bb0e-9668e50de116-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "850f22de-08c0-4bc5-bb0e-9668e50de116" (UID: "850f22de-08c0-4bc5-bb0e-9668e50de116"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:37:58.974600 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.974566    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850f22de-08c0-4bc5-bb0e-9668e50de116-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "850f22de-08c0-4bc5-bb0e-9668e50de116" (UID: "850f22de-08c0-4bc5-bb0e-9668e50de116"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:37:58.974897 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:58.974878    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/850f22de-08c0-4bc5-bb0e-9668e50de116-kube-api-access-jjbzg" (OuterVolumeSpecName: "kube-api-access-jjbzg") pod "850f22de-08c0-4bc5-bb0e-9668e50de116" (UID: "850f22de-08c0-4bc5-bb0e-9668e50de116"). InnerVolumeSpecName "kube-api-access-jjbzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:37:59.073733 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.073711    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-oauth-serving-cert\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:37:59.073733 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.073734    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjbzg\" (UniqueName: \"kubernetes.io/projected/850f22de-08c0-4bc5-bb0e-9668e50de116-kube-api-access-jjbzg\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:37:59.073885 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.073745    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-console-config\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:37:59.073885 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.073753    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-service-ca\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:37:59.073885 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.073761    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/850f22de-08c0-4bc5-bb0e-9668e50de116-console-serving-cert\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:37:59.073885 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.073769    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/850f22de-08c0-4bc5-bb0e-9668e50de116-console-oauth-config\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:37:59.073885 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.073776    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850f22de-08c0-4bc5-bb0e-9668e50de116-trusted-ca-bundle\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:37:59.741558 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.741510    2577 generic.go:356] "Generic (PLEG): container finished" podID="850f22de-08c0-4bc5-bb0e-9668e50de116" containerID="eab2a94893187996f9df505a97a22f3791a29b23c7c560ba29fecd0621ca4dab" exitCode=0
Jul 09 09:37:59.741558 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.741557    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8c5b4b49-pw8df"
Jul 09 09:37:59.742065 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.741563    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8c5b4b49-pw8df" event={"ID":"850f22de-08c0-4bc5-bb0e-9668e50de116","Type":"ContainerDied","Data":"eab2a94893187996f9df505a97a22f3791a29b23c7c560ba29fecd0621ca4dab"}
Jul 09 09:37:59.742065 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.741664    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8c5b4b49-pw8df" event={"ID":"850f22de-08c0-4bc5-bb0e-9668e50de116","Type":"ContainerDied","Data":"3b3577e95dfe5adb394c370b84066ca7c8e0932012d9ce479aa8b634353e8e62"}
Jul 09 09:37:59.742065 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.741684    2577 scope.go:117] "RemoveContainer" containerID="eab2a94893187996f9df505a97a22f3791a29b23c7c560ba29fecd0621ca4dab"
Jul 09 09:37:59.749522 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.749505    2577 scope.go:117] "RemoveContainer" containerID="eab2a94893187996f9df505a97a22f3791a29b23c7c560ba29fecd0621ca4dab"
Jul 09 09:37:59.749759 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:37:59.749744    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eab2a94893187996f9df505a97a22f3791a29b23c7c560ba29fecd0621ca4dab\": container with ID starting with eab2a94893187996f9df505a97a22f3791a29b23c7c560ba29fecd0621ca4dab not found: ID does not exist" containerID="eab2a94893187996f9df505a97a22f3791a29b23c7c560ba29fecd0621ca4dab"
Jul 09 09:37:59.749825 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.749766    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eab2a94893187996f9df505a97a22f3791a29b23c7c560ba29fecd0621ca4dab"} err="failed to get container status \"eab2a94893187996f9df505a97a22f3791a29b23c7c560ba29fecd0621ca4dab\": rpc error: code = NotFound desc = could not find container \"eab2a94893187996f9df505a97a22f3791a29b23c7c560ba29fecd0621ca4dab\": container with ID starting with eab2a94893187996f9df505a97a22f3791a29b23c7c560ba29fecd0621ca4dab not found: ID does not exist"
Jul 09 09:37:59.757770 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.757746    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b8c5b4b49-pw8df"]
Jul 09 09:37:59.761954 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:37:59.761932    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b8c5b4b49-pw8df"]
Jul 09 09:38:01.553974 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:38:01.553942    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="850f22de-08c0-4bc5-bb0e-9668e50de116" path="/var/lib/kubelet/pods/850f22de-08c0-4bc5-bb0e-9668e50de116/volumes"
Jul 09 09:38:17.303388 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:38:17.303357    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:38:17.309556 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:38:17.309528    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:38:17.314722 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:38:17.314701    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 09:38:35.798494 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:38:35.798456    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:00.172718 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:00.172682    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:00.265751 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:00.265704    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:00.568553 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:00.568524    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:00.669067 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:00.669035    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:00.731534 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:00.731506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:01.457123 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.457092    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"]
Jul 09 09:39:01.457484 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.457390    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="850f22de-08c0-4bc5-bb0e-9668e50de116" containerName="console"
Jul 09 09:39:01.457484 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.457405    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="850f22de-08c0-4bc5-bb0e-9668e50de116" containerName="console"
Jul 09 09:39:01.457484 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.457454    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="850f22de-08c0-4bc5-bb0e-9668e50de116" containerName="console"
Jul 09 09:39:01.460560 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.460544    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"
Jul 09 09:39:01.463259 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.463236    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gp9wj\""
Jul 09 09:39:01.463378 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.463273    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 09:39:01.463378 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.463315    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 09:39:01.469648 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.469625    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"]
Jul 09 09:39:01.577342 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.577317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn\" (UID: \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"
Jul 09 09:39:01.577452 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.577366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn\" (UID: \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"
Jul 09 09:39:01.577452 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.577420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb9vv\" (UniqueName: \"kubernetes.io/projected/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-kube-api-access-kb9vv\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn\" (UID: \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"
Jul 09 09:39:01.678669 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.678643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb9vv\" (UniqueName: \"kubernetes.io/projected/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-kube-api-access-kb9vv\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn\" (UID: \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"
Jul 09 09:39:01.678724 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.678687    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn\" (UID: \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"
Jul 09 09:39:01.678758 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.678725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn\" (UID: \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"
Jul 09 09:39:01.679056 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.679041    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn\" (UID: \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"
Jul 09 09:39:01.679103 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.679079    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn\" (UID: \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"
Jul 09 09:39:01.687614 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.687594    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb9vv\" (UniqueName: \"kubernetes.io/projected/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-kube-api-access-kb9vv\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn\" (UID: \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"
Jul 09 09:39:01.769092 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.769072    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"
Jul 09 09:39:01.897013 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.896981    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"]
Jul 09 09:39:01.899703 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:39:01.899677    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod349cd08f_0bb3_4caa_a5aa_ad69771e2fb9.slice/crio-57419658b4aba129a174be22f2b3f0a8696564171dc7ab127752bc8387be381a WatchSource:0}: Error finding container 57419658b4aba129a174be22f2b3f0a8696564171dc7ab127752bc8387be381a: Status 404 returned error can't find the container with id 57419658b4aba129a174be22f2b3f0a8696564171dc7ab127752bc8387be381a
Jul 09 09:39:01.901771 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.901754    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 09:39:01.962512 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:01.962482    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn" event={"ID":"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9","Type":"ContainerStarted","Data":"57419658b4aba129a174be22f2b3f0a8696564171dc7ab127752bc8387be381a"}
Jul 09 09:39:07.986869 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:07.986836    2577 generic.go:356] "Generic (PLEG): container finished" podID="349cd08f-0bb3-4caa-a5aa-ad69771e2fb9" containerID="fd066262aa3d11e193948226f62b30f2134f91e32f2e96e953eeab61f340b0e0" exitCode=0
Jul 09 09:39:07.987209 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:07.986898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn" event={"ID":"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9","Type":"ContainerDied","Data":"fd066262aa3d11e193948226f62b30f2134f91e32f2e96e953eeab61f340b0e0"}
Jul 09 09:39:11.000639 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:11.000608    2577 generic.go:356] "Generic (PLEG): container finished" podID="349cd08f-0bb3-4caa-a5aa-ad69771e2fb9" containerID="50d7ae347bccd49968cfa6415f9827ab9766111c29237aca6371a9fba5b6b5d2" exitCode=0
Jul 09 09:39:11.001021 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:11.000653    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn" event={"ID":"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9","Type":"ContainerDied","Data":"50d7ae347bccd49968cfa6415f9827ab9766111c29237aca6371a9fba5b6b5d2"}
Jul 09 09:39:18.032763 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:18.032733    2577 generic.go:356] "Generic (PLEG): container finished" podID="349cd08f-0bb3-4caa-a5aa-ad69771e2fb9" containerID="3f38dcb5fb6371dff9200b28c0bf50eac8c26685c36caf83d16557d46a174f02" exitCode=0
Jul 09 09:39:18.033076 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:18.032824    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn" event={"ID":"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9","Type":"ContainerDied","Data":"3f38dcb5fb6371dff9200b28c0bf50eac8c26685c36caf83d16557d46a174f02"}
Jul 09 09:39:19.159783 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:19.159764    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"
Jul 09 09:39:19.320188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:19.320132    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-util\") pod \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\" (UID: \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\") "
Jul 09 09:39:19.320188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:19.320173    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb9vv\" (UniqueName: \"kubernetes.io/projected/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-kube-api-access-kb9vv\") pod \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\" (UID: \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\") "
Jul 09 09:39:19.320329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:19.320197    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-bundle\") pod \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\" (UID: \"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9\") "
Jul 09 09:39:19.321050 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:19.321017    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-bundle" (OuterVolumeSpecName: "bundle") pod "349cd08f-0bb3-4caa-a5aa-ad69771e2fb9" (UID: "349cd08f-0bb3-4caa-a5aa-ad69771e2fb9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:39:19.322202 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:19.322182    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-kube-api-access-kb9vv" (OuterVolumeSpecName: "kube-api-access-kb9vv") pod "349cd08f-0bb3-4caa-a5aa-ad69771e2fb9" (UID: "349cd08f-0bb3-4caa-a5aa-ad69771e2fb9"). InnerVolumeSpecName "kube-api-access-kb9vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:39:19.327535 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:19.327508    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-util" (OuterVolumeSpecName: "util") pod "349cd08f-0bb3-4caa-a5aa-ad69771e2fb9" (UID: "349cd08f-0bb3-4caa-a5aa-ad69771e2fb9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:39:19.421487 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:19.421466    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-util\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:39:19.421487 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:19.421485    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb9vv\" (UniqueName: \"kubernetes.io/projected/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-kube-api-access-kb9vv\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:39:19.421615 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:19.421497    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/349cd08f-0bb3-4caa-a5aa-ad69771e2fb9-bundle\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:39:20.042745 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:20.042713    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn" event={"ID":"349cd08f-0bb3-4caa-a5aa-ad69771e2fb9","Type":"ContainerDied","Data":"57419658b4aba129a174be22f2b3f0a8696564171dc7ab127752bc8387be381a"}
Jul 09 09:39:20.042745 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:20.042733    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hcjjn"
Jul 09 09:39:20.042940 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:20.042761    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57419658b4aba129a174be22f2b3f0a8696564171dc7ab127752bc8387be381a"
Jul 09 09:39:22.575089 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:22.575047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:22.672423 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:22.672389    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:24.014897 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.014863    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8"]
Jul 09 09:39:24.015284 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.015136    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="349cd08f-0bb3-4caa-a5aa-ad69771e2fb9" containerName="pull"
Jul 09 09:39:24.015284 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.015148    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="349cd08f-0bb3-4caa-a5aa-ad69771e2fb9" containerName="pull"
Jul 09 09:39:24.015284 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.015164    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="349cd08f-0bb3-4caa-a5aa-ad69771e2fb9" containerName="util"
Jul 09 09:39:24.015284 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.015169    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="349cd08f-0bb3-4caa-a5aa-ad69771e2fb9" containerName="util"
Jul 09 09:39:24.015284 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.015180    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="349cd08f-0bb3-4caa-a5aa-ad69771e2fb9" containerName="extract"
Jul 09 09:39:24.015284 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.015186    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="349cd08f-0bb3-4caa-a5aa-ad69771e2fb9" containerName="extract"
Jul 09 09:39:24.015284 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.015238    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="349cd08f-0bb3-4caa-a5aa-ad69771e2fb9" containerName="extract"
Jul 09 09:39:24.049817 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.049778    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8"]
Jul 09 09:39:24.049946 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.049884    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8"
Jul 09 09:39:24.055342 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.055317    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 09 09:39:24.055555 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.055463    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-m82rr\""
Jul 09 09:39:24.055714 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.055695    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 09 09:39:24.055714 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.055701    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 09 09:39:24.151699 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.151672    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ffrg\" (UniqueName: \"kubernetes.io/projected/166a53d3-dfef-4294-af3d-48e0787f8695-kube-api-access-9ffrg\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-f95n8\" (UID: \"166a53d3-dfef-4294-af3d-48e0787f8695\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8"
Jul 09 09:39:24.151844 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.151706    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/166a53d3-dfef-4294-af3d-48e0787f8695-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-f95n8\" (UID: \"166a53d3-dfef-4294-af3d-48e0787f8695\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8"
Jul 09 09:39:24.252268 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.252234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ffrg\" (UniqueName: \"kubernetes.io/projected/166a53d3-dfef-4294-af3d-48e0787f8695-kube-api-access-9ffrg\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-f95n8\" (UID: \"166a53d3-dfef-4294-af3d-48e0787f8695\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8"
Jul 09 09:39:24.252375 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.252274    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/166a53d3-dfef-4294-af3d-48e0787f8695-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-f95n8\" (UID: \"166a53d3-dfef-4294-af3d-48e0787f8695\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8"
Jul 09 09:39:24.254551 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.254527    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/166a53d3-dfef-4294-af3d-48e0787f8695-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-f95n8\" (UID: \"166a53d3-dfef-4294-af3d-48e0787f8695\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8"
Jul 09 09:39:24.262568 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.262545    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ffrg\" (UniqueName: \"kubernetes.io/projected/166a53d3-dfef-4294-af3d-48e0787f8695-kube-api-access-9ffrg\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-f95n8\" (UID: \"166a53d3-dfef-4294-af3d-48e0787f8695\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8"
Jul 09 09:39:24.360210 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.360161    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8"
Jul 09 09:39:24.502419 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:24.502341    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8"]
Jul 09 09:39:24.507378 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:39:24.507348    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod166a53d3_dfef_4294_af3d_48e0787f8695.slice/crio-8ea10582e03b950f3a94922cbbcc3e725802ab6a5770d97dbd51733a8b293b14 WatchSource:0}: Error finding container 8ea10582e03b950f3a94922cbbcc3e725802ab6a5770d97dbd51733a8b293b14: Status 404 returned error can't find the container with id 8ea10582e03b950f3a94922cbbcc3e725802ab6a5770d97dbd51733a8b293b14
Jul 09 09:39:25.063954 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:25.063921    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8" event={"ID":"166a53d3-dfef-4294-af3d-48e0787f8695","Type":"ContainerStarted","Data":"8ea10582e03b950f3a94922cbbcc3e725802ab6a5770d97dbd51733a8b293b14"}
Jul 09 09:39:27.951689 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:27.951657    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:28.079033 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.078997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8" event={"ID":"166a53d3-dfef-4294-af3d-48e0787f8695","Type":"ContainerStarted","Data":"a3f8b408e7f6fb455027477fc70ceff6aec3157c1eec691b170650a47d9a34bd"}
Jul 09 09:39:28.079285 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.079155    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8"
Jul 09 09:39:28.101820 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.101749    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8" podStartSLOduration=2.184929625 podStartE2EDuration="5.101729489s" totalImagesPullingTime="2.916799864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:39:28.099725924 +0000 UTC m=+371.105895022" watchObservedRunningTime="2026-07-09 09:39:28.101729489 +0000 UTC m=+371.107898561"
Jul 09 09:39:28.182990 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.182962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:28.190374 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.190352    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-wvsk4"]
Jul 09 09:39:28.214830 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.214790    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-wvsk4"]
Jul 09 09:39:28.214922 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.214881    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:28.217616 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.217583    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 09 09:39:28.217616 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.217608    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-b5fj2\""
Jul 09 09:39:28.217791 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.217632    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 09 09:39:28.382137 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.382109    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/48a40727-2d93-490a-9e4b-b127473347df-cabundle0\") pod \"keda-operator-68bff59c-wvsk4\" (UID: \"48a40727-2d93-490a-9e4b-b127473347df\") " pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:28.382247 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.382149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np7rr\" (UniqueName: \"kubernetes.io/projected/48a40727-2d93-490a-9e4b-b127473347df-kube-api-access-np7rr\") pod \"keda-operator-68bff59c-wvsk4\" (UID: \"48a40727-2d93-490a-9e4b-b127473347df\") " pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:28.382247 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.382225    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/48a40727-2d93-490a-9e4b-b127473347df-certificates\") pod \"keda-operator-68bff59c-wvsk4\" (UID: \"48a40727-2d93-490a-9e4b-b127473347df\") " pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:28.401460 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.401427    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:28.407530 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.407510    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"]
Jul 09 09:39:28.434245 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.434224    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:28.434353 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.434230    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"]
Jul 09 09:39:28.436606 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.436587    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 09 09:39:28.483258 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.483195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/48a40727-2d93-490a-9e4b-b127473347df-cabundle0\") pod \"keda-operator-68bff59c-wvsk4\" (UID: \"48a40727-2d93-490a-9e4b-b127473347df\") " pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:28.483366 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.483257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np7rr\" (UniqueName: \"kubernetes.io/projected/48a40727-2d93-490a-9e4b-b127473347df-kube-api-access-np7rr\") pod \"keda-operator-68bff59c-wvsk4\" (UID: \"48a40727-2d93-490a-9e4b-b127473347df\") " pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:28.483366 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.483339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/48a40727-2d93-490a-9e4b-b127473347df-certificates\") pod \"keda-operator-68bff59c-wvsk4\" (UID: \"48a40727-2d93-490a-9e4b-b127473347df\") " pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:28.483480 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.483447    2577 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Jul 09 09:39:28.483480 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.483465    2577 secret.go:281] references non-existent secret key: ca.crt
Jul 09 09:39:28.483480 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.483476    2577 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 09 09:39:28.483646 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.483491    2577 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-wvsk4: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 09 09:39:28.483646 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.483551    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48a40727-2d93-490a-9e4b-b127473347df-certificates podName:48a40727-2d93-490a-9e4b-b127473347df nodeName:}" failed. No retries permitted until 2026-07-09 09:39:28.983532584 +0000 UTC m=+371.989701633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/48a40727-2d93-490a-9e4b-b127473347df-certificates") pod "keda-operator-68bff59c-wvsk4" (UID: "48a40727-2d93-490a-9e4b-b127473347df") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 09 09:39:28.484018 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.483997    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/48a40727-2d93-490a-9e4b-b127473347df-cabundle0\") pod \"keda-operator-68bff59c-wvsk4\" (UID: \"48a40727-2d93-490a-9e4b-b127473347df\") " pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:28.496932 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.496903    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-np7rr\" (UniqueName: \"kubernetes.io/projected/48a40727-2d93-490a-9e4b-b127473347df-kube-api-access-np7rr\") pod \"keda-operator-68bff59c-wvsk4\" (UID: \"48a40727-2d93-490a-9e4b-b127473347df\") " pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:28.583850 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.583823    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bd1301eb-abe7-4186-a884-5b4f9958d7f9-certificates\") pod \"keda-metrics-apiserver-559f75f947-trgf8\" (UID: \"bd1301eb-abe7-4186-a884-5b4f9958d7f9\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:28.583945 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.583858    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr5fb\" (UniqueName: \"kubernetes.io/projected/bd1301eb-abe7-4186-a884-5b4f9958d7f9-kube-api-access-zr5fb\") pod \"keda-metrics-apiserver-559f75f947-trgf8\" (UID: \"bd1301eb-abe7-4186-a884-5b4f9958d7f9\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:28.583945 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.583915    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/bd1301eb-abe7-4186-a884-5b4f9958d7f9-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-trgf8\" (UID: \"bd1301eb-abe7-4186-a884-5b4f9958d7f9\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:28.684353 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.684330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bd1301eb-abe7-4186-a884-5b4f9958d7f9-certificates\") pod \"keda-metrics-apiserver-559f75f947-trgf8\" (UID: \"bd1301eb-abe7-4186-a884-5b4f9958d7f9\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:28.684445 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.684360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr5fb\" (UniqueName: \"kubernetes.io/projected/bd1301eb-abe7-4186-a884-5b4f9958d7f9-kube-api-access-zr5fb\") pod \"keda-metrics-apiserver-559f75f947-trgf8\" (UID: \"bd1301eb-abe7-4186-a884-5b4f9958d7f9\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:28.684445 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.684388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/bd1301eb-abe7-4186-a884-5b4f9958d7f9-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-trgf8\" (UID: \"bd1301eb-abe7-4186-a884-5b4f9958d7f9\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:28.684610 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.684469    2577 secret.go:281] references non-existent secret key: tls.crt
Jul 09 09:39:28.684610 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.684486    2577 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jul 09 09:39:28.684610 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.684505    2577 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found
Jul 09 09:39:28.684610 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.684523    2577 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-559f75f947-trgf8: [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found]
Jul 09 09:39:28.684610 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.684583    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd1301eb-abe7-4186-a884-5b4f9958d7f9-certificates podName:bd1301eb-abe7-4186-a884-5b4f9958d7f9 nodeName:}" failed. No retries permitted until 2026-07-09 09:39:29.184564782 +0000 UTC m=+372.190733843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/bd1301eb-abe7-4186-a884-5b4f9958d7f9-certificates") pod "keda-metrics-apiserver-559f75f947-trgf8" (UID: "bd1301eb-abe7-4186-a884-5b4f9958d7f9") : [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found]
Jul 09 09:39:28.684824 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.684756    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/bd1301eb-abe7-4186-a884-5b4f9958d7f9-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-trgf8\" (UID: \"bd1301eb-abe7-4186-a884-5b4f9958d7f9\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:28.693860 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.693837    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr5fb\" (UniqueName: \"kubernetes.io/projected/bd1301eb-abe7-4186-a884-5b4f9958d7f9-kube-api-access-zr5fb\") pod \"keda-metrics-apiserver-559f75f947-trgf8\" (UID: \"bd1301eb-abe7-4186-a884-5b4f9958d7f9\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:28.765717 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.765653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:28.869688 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.869663    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:28.987532 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:28.987505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/48a40727-2d93-490a-9e4b-b127473347df-certificates\") pod \"keda-operator-68bff59c-wvsk4\" (UID: \"48a40727-2d93-490a-9e4b-b127473347df\") " pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:28.987963 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.987642    2577 secret.go:281] references non-existent secret key: ca.crt
Jul 09 09:39:28.987963 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.987663    2577 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 09 09:39:28.987963 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.987675    2577 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-wvsk4: references non-existent secret key: ca.crt
Jul 09 09:39:28.987963 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:28.987733    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48a40727-2d93-490a-9e4b-b127473347df-certificates podName:48a40727-2d93-490a-9e4b-b127473347df nodeName:}" failed. No retries permitted until 2026-07-09 09:39:29.987715079 +0000 UTC m=+372.993884147 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/48a40727-2d93-490a-9e4b-b127473347df-certificates") pod "keda-operator-68bff59c-wvsk4" (UID: "48a40727-2d93-490a-9e4b-b127473347df") : references non-existent secret key: ca.crt
Jul 09 09:39:29.189291 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.189228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bd1301eb-abe7-4186-a884-5b4f9958d7f9-certificates\") pod \"keda-metrics-apiserver-559f75f947-trgf8\" (UID: \"bd1301eb-abe7-4186-a884-5b4f9958d7f9\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:29.189395 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:29.189351    2577 secret.go:281] references non-existent secret key: tls.crt
Jul 09 09:39:29.189395 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:29.189368    2577 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jul 09 09:39:29.189395 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:29.189384    2577 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-559f75f947-trgf8: references non-existent secret key: tls.crt
Jul 09 09:39:29.189513 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:39:29.189427    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd1301eb-abe7-4186-a884-5b4f9958d7f9-certificates podName:bd1301eb-abe7-4186-a884-5b4f9958d7f9 nodeName:}" failed. No retries permitted until 2026-07-09 09:39:30.189415544 +0000 UTC m=+373.195584593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/bd1301eb-abe7-4186-a884-5b4f9958d7f9-certificates") pod "keda-metrics-apiserver-559f75f947-trgf8" (UID: "bd1301eb-abe7-4186-a884-5b4f9958d7f9") : references non-existent secret key: tls.crt
Jul 09 09:39:29.202363 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.202336    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:29.388104 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.388080    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-nzv4p"]
Jul 09 09:39:29.408420 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.408397    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-nzv4p"]
Jul 09 09:39:29.408520 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.408477    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-nzv4p"
Jul 09 09:39:29.410985 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.410968    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 09 09:39:29.419829 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.419791    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:29.521100 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.521079    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:29.591471 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.591445    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc9zj\" (UniqueName: \"kubernetes.io/projected/43cc0a1f-def0-480d-93ea-f531099fa3d4-kube-api-access-hc9zj\") pod \"keda-admission-5fbd5c4644-nzv4p\" (UID: \"43cc0a1f-def0-480d-93ea-f531099fa3d4\") " pod="openshift-keda/keda-admission-5fbd5c4644-nzv4p"
Jul 09 09:39:29.591571 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.591501    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/43cc0a1f-def0-480d-93ea-f531099fa3d4-certificates\") pod \"keda-admission-5fbd5c4644-nzv4p\" (UID: \"43cc0a1f-def0-480d-93ea-f531099fa3d4\") " pod="openshift-keda/keda-admission-5fbd5c4644-nzv4p"
Jul 09 09:39:29.620717 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.620690    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:29.691974 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.691946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/43cc0a1f-def0-480d-93ea-f531099fa3d4-certificates\") pod \"keda-admission-5fbd5c4644-nzv4p\" (UID: \"43cc0a1f-def0-480d-93ea-f531099fa3d4\") " pod="openshift-keda/keda-admission-5fbd5c4644-nzv4p"
Jul 09 09:39:29.692106 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.692015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9zj\" (UniqueName: \"kubernetes.io/projected/43cc0a1f-def0-480d-93ea-f531099fa3d4-kube-api-access-hc9zj\") pod \"keda-admission-5fbd5c4644-nzv4p\" (UID: \"43cc0a1f-def0-480d-93ea-f531099fa3d4\") " pod="openshift-keda/keda-admission-5fbd5c4644-nzv4p"
Jul 09 09:39:29.694150 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.694127    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/43cc0a1f-def0-480d-93ea-f531099fa3d4-certificates\") pod \"keda-admission-5fbd5c4644-nzv4p\" (UID: \"43cc0a1f-def0-480d-93ea-f531099fa3d4\") " pod="openshift-keda/keda-admission-5fbd5c4644-nzv4p"
Jul 09 09:39:29.701232 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.701214    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc9zj\" (UniqueName: \"kubernetes.io/projected/43cc0a1f-def0-480d-93ea-f531099fa3d4-kube-api-access-hc9zj\") pod \"keda-admission-5fbd5c4644-nzv4p\" (UID: \"43cc0a1f-def0-480d-93ea-f531099fa3d4\") " pod="openshift-keda/keda-admission-5fbd5c4644-nzv4p"
Jul 09 09:39:29.718588 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.718568    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-nzv4p"
Jul 09 09:39:29.865256 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.865233    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-nzv4p"]
Jul 09 09:39:29.868047 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:39:29.868023    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43cc0a1f_def0_480d_93ea_f531099fa3d4.slice/crio-79194676c5e6d1a187fdc8c3c2f5fa4e140904bb11d96cac0b739ba93637e5e4 WatchSource:0}: Error finding container 79194676c5e6d1a187fdc8c3c2f5fa4e140904bb11d96cac0b739ba93637e5e4: Status 404 returned error can't find the container with id 79194676c5e6d1a187fdc8c3c2f5fa4e140904bb11d96cac0b739ba93637e5e4
Jul 09 09:39:29.915783 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.915757    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:29.994614 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.994587    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/48a40727-2d93-490a-9e4b-b127473347df-certificates\") pod \"keda-operator-68bff59c-wvsk4\" (UID: \"48a40727-2d93-490a-9e4b-b127473347df\") " pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:29.996893 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:29.996869    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/48a40727-2d93-490a-9e4b-b127473347df-certificates\") pod \"keda-operator-68bff59c-wvsk4\" (UID: \"48a40727-2d93-490a-9e4b-b127473347df\") " pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:30.025884 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:30.025861    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:30.095961 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:30.095931    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-nzv4p" event={"ID":"43cc0a1f-def0-480d-93ea-f531099fa3d4","Type":"ContainerStarted","Data":"79194676c5e6d1a187fdc8c3c2f5fa4e140904bb11d96cac0b739ba93637e5e4"}
Jul 09 09:39:30.154072 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:30.154041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:30.187632 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:30.187603    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-wvsk4"]
Jul 09 09:39:30.190047 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:39:30.190024    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48a40727_2d93_490a_9e4b_b127473347df.slice/crio-6de081156d4ad7b318a40ec01621715bc1b4f05ed3aaf09b9a828809d9994e79 WatchSource:0}: Error finding container 6de081156d4ad7b318a40ec01621715bc1b4f05ed3aaf09b9a828809d9994e79: Status 404 returned error can't find the container with id 6de081156d4ad7b318a40ec01621715bc1b4f05ed3aaf09b9a828809d9994e79
Jul 09 09:39:30.196105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:30.196083    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bd1301eb-abe7-4186-a884-5b4f9958d7f9-certificates\") pod \"keda-metrics-apiserver-559f75f947-trgf8\" (UID: \"bd1301eb-abe7-4186-a884-5b4f9958d7f9\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:30.198428 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:30.198411    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bd1301eb-abe7-4186-a884-5b4f9958d7f9-certificates\") pod \"keda-metrics-apiserver-559f75f947-trgf8\" (UID: \"bd1301eb-abe7-4186-a884-5b4f9958d7f9\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:30.245330 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:30.245305    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:30.294013 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:30.293952    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:30.399409 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:30.399363    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:39:30.400093 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:30.400072    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"]
Jul 09 09:39:30.402536 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:39:30.402504    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd1301eb_abe7_4186_a884_5b4f9958d7f9.slice/crio-44637f05b26515fa1272a05701cc666b6d1f9fb75cad7e1f82f7c71a9f29463f WatchSource:0}: Error finding container 44637f05b26515fa1272a05701cc666b6d1f9fb75cad7e1f82f7c71a9f29463f: Status 404 returned error can't find the container with id 44637f05b26515fa1272a05701cc666b6d1f9fb75cad7e1f82f7c71a9f29463f
Jul 09 09:39:31.102175 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:31.102139    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-wvsk4" event={"ID":"48a40727-2d93-490a-9e4b-b127473347df","Type":"ContainerStarted","Data":"6de081156d4ad7b318a40ec01621715bc1b4f05ed3aaf09b9a828809d9994e79"}
Jul 09 09:39:31.103480 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:31.103450    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8" event={"ID":"bd1301eb-abe7-4186-a884-5b4f9958d7f9","Type":"ContainerStarted","Data":"44637f05b26515fa1272a05701cc666b6d1f9fb75cad7e1f82f7c71a9f29463f"}
Jul 09 09:39:34.122574 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:34.122516    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-wvsk4" event={"ID":"48a40727-2d93-490a-9e4b-b127473347df","Type":"ContainerStarted","Data":"545bf82843ef315b37a25df559dfc54d9cf2ed81b279aa3b0a850c9cd345cd7e"}
Jul 09 09:39:34.123067 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:34.122764    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:34.124627 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:34.124596    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-nzv4p" event={"ID":"43cc0a1f-def0-480d-93ea-f531099fa3d4","Type":"ContainerStarted","Data":"4cb1bd9480811fbc62d2201f8b2acceef5bd586d9a9e22568d0aecf02c40d9e7"}
Jul 09 09:39:34.124857 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:34.124837    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-nzv4p"
Jul 09 09:39:34.176176 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:34.176136    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-wvsk4" podStartSLOduration=3.024349008 podStartE2EDuration="6.1761184s" totalImagesPullingTime="3.151769392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:39:34.1532427 +0000 UTC m=+377.159411789" watchObservedRunningTime="2026-07-09 09:39:34.1761184 +0000 UTC m=+377.182287483"
Jul 09 09:39:34.176726 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:34.176678    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-nzv4p" podStartSLOduration=1.716556907 podStartE2EDuration="5.176661554s" totalImagesPullingTime="3.460104647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:39:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:39:34.175655751 +0000 UTC m=+377.181824851" watchObservedRunningTime="2026-07-09 09:39:34.176661554 +0000 UTC m=+377.182830625"
Jul 09 09:39:36.135631 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:36.135599    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8" event={"ID":"bd1301eb-abe7-4186-a884-5b4f9958d7f9","Type":"ContainerStarted","Data":"4cbe503d38d83e4f1e2de536dcac9350d84116fb96f61739530bb360c5112b2d"}
Jul 09 09:39:36.136000 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:36.135722    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:36.154055 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:36.154015    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8" podStartSLOduration=3.251769464 podStartE2EDuration="8.154001862s" totalImagesPullingTime="4.902232398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:39:36.153521645 +0000 UTC m=+379.159690718" watchObservedRunningTime="2026-07-09 09:39:36.154001862 +0000 UTC m=+379.160170933"
Jul 09 09:39:47.147041 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:47.147014    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-trgf8"
Jul 09 09:39:49.085780 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:49.085748    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-f95n8"
Jul 09 09:39:55.132257 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:55.132224    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-wvsk4"
Jul 09 09:39:55.132657 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:39:55.132512    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-nzv4p"
Jul 09 09:40:01.112137 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:01.112100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:40:01.170241 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:01.170201    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:40:37.098703 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.098669    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-5876b4f96d-hvhnk"]
Jul 09 09:40:37.101457 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.101441    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk"
Jul 09 09:40:37.103940 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.103911    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 09 09:40:37.104070 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.103945    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 09 09:40:37.104070 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.103977    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 09 09:40:37.104715 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.104697    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-fhwtq\""
Jul 09 09:40:37.111924 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.111905    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5876b4f96d-hvhnk"]
Jul 09 09:40:37.122343 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.122323    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"]
Jul 09 09:40:37.124942 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.124928    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"
Jul 09 09:40:37.127440 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.127424    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 09 09:40:37.127599 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.127584    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-wd9h7\""
Jul 09 09:40:37.144725 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.144701    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"]
Jul 09 09:40:37.157573 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.157551    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-8dccw"]
Jul 09 09:40:37.160145 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.160129    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-8dccw"
Jul 09 09:40:37.162583 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.162567    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-kq922\""
Jul 09 09:40:37.162817 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.162788    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 09 09:40:37.172725 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.172705    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-8dccw"]
Jul 09 09:40:37.177340 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.177317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fmwg\" (UniqueName: \"kubernetes.io/projected/f94cfcee-b683-43d0-90fd-a9437bfc32ee-kube-api-access-7fmwg\") pod \"kserve-controller-manager-5876b4f96d-hvhnk\" (UID: \"f94cfcee-b683-43d0-90fd-a9437bfc32ee\") " pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk"
Jul 09 09:40:37.177441 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.177364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f94cfcee-b683-43d0-90fd-a9437bfc32ee-cert\") pod \"kserve-controller-manager-5876b4f96d-hvhnk\" (UID: \"f94cfcee-b683-43d0-90fd-a9437bfc32ee\") " pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk"
Jul 09 09:40:37.277976 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.277954    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fmwg\" (UniqueName: \"kubernetes.io/projected/f94cfcee-b683-43d0-90fd-a9437bfc32ee-kube-api-access-7fmwg\") pod \"kserve-controller-manager-5876b4f96d-hvhnk\" (UID: \"f94cfcee-b683-43d0-90fd-a9437bfc32ee\") " pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk"
Jul 09 09:40:37.278071 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.277991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f94cfcee-b683-43d0-90fd-a9437bfc32ee-cert\") pod \"kserve-controller-manager-5876b4f96d-hvhnk\" (UID: \"f94cfcee-b683-43d0-90fd-a9437bfc32ee\") " pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk"
Jul 09 09:40:37.278071 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.278023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5qhg\" (UniqueName: \"kubernetes.io/projected/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-kube-api-access-l5qhg\") pod \"llmisvc-controller-manager-57d57b58b4-5rf4k\" (UID: \"1be7c91f-66f5-49a3-8c7d-8779f1e2f320\") " pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"
Jul 09 09:40:37.278071 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.278062    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f79q5\" (UniqueName: \"kubernetes.io/projected/53a33bf0-44ee-4195-9d34-ffaf0b0a357f-kube-api-access-f79q5\") pod \"seaweedfs-64568bcd49-8dccw\" (UID: \"53a33bf0-44ee-4195-9d34-ffaf0b0a357f\") " pod="kserve/seaweedfs-64568bcd49-8dccw"
Jul 09 09:40:37.278207 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.278095    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-cert\") pod \"llmisvc-controller-manager-57d57b58b4-5rf4k\" (UID: \"1be7c91f-66f5-49a3-8c7d-8779f1e2f320\") " pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"
Jul 09 09:40:37.278207 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.278109    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/53a33bf0-44ee-4195-9d34-ffaf0b0a357f-data\") pod \"seaweedfs-64568bcd49-8dccw\" (UID: \"53a33bf0-44ee-4195-9d34-ffaf0b0a357f\") " pod="kserve/seaweedfs-64568bcd49-8dccw"
Jul 09 09:40:37.278207 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:40:37.278124    2577 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 09 09:40:37.278298 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:40:37.278229    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f94cfcee-b683-43d0-90fd-a9437bfc32ee-cert podName:f94cfcee-b683-43d0-90fd-a9437bfc32ee nodeName:}" failed. No retries permitted until 2026-07-09 09:40:37.778210502 +0000 UTC m=+440.784379562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f94cfcee-b683-43d0-90fd-a9437bfc32ee-cert") pod "kserve-controller-manager-5876b4f96d-hvhnk" (UID: "f94cfcee-b683-43d0-90fd-a9437bfc32ee") : secret "kserve-webhook-server-cert" not found
Jul 09 09:40:37.286897 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.286876    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fmwg\" (UniqueName: \"kubernetes.io/projected/f94cfcee-b683-43d0-90fd-a9437bfc32ee-kube-api-access-7fmwg\") pod \"kserve-controller-manager-5876b4f96d-hvhnk\" (UID: \"f94cfcee-b683-43d0-90fd-a9437bfc32ee\") " pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk"
Jul 09 09:40:37.378981 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.378917    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-cert\") pod \"llmisvc-controller-manager-57d57b58b4-5rf4k\" (UID: \"1be7c91f-66f5-49a3-8c7d-8779f1e2f320\") " pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"
Jul 09 09:40:37.378981 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.378956    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/53a33bf0-44ee-4195-9d34-ffaf0b0a357f-data\") pod \"seaweedfs-64568bcd49-8dccw\" (UID: \"53a33bf0-44ee-4195-9d34-ffaf0b0a357f\") " pod="kserve/seaweedfs-64568bcd49-8dccw"
Jul 09 09:40:37.379105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.379011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5qhg\" (UniqueName: \"kubernetes.io/projected/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-kube-api-access-l5qhg\") pod \"llmisvc-controller-manager-57d57b58b4-5rf4k\" (UID: \"1be7c91f-66f5-49a3-8c7d-8779f1e2f320\") " pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"
Jul 09 09:40:37.379105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.379033    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f79q5\" (UniqueName: \"kubernetes.io/projected/53a33bf0-44ee-4195-9d34-ffaf0b0a357f-kube-api-access-f79q5\") pod \"seaweedfs-64568bcd49-8dccw\" (UID: \"53a33bf0-44ee-4195-9d34-ffaf0b0a357f\") " pod="kserve/seaweedfs-64568bcd49-8dccw"
Jul 09 09:40:37.379105 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:40:37.379048    2577 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 09 09:40:37.379105 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:40:37.379102    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-cert podName:1be7c91f-66f5-49a3-8c7d-8779f1e2f320 nodeName:}" failed. No retries permitted until 2026-07-09 09:40:37.87908502 +0000 UTC m=+440.885254076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-cert") pod "llmisvc-controller-manager-57d57b58b4-5rf4k" (UID: "1be7c91f-66f5-49a3-8c7d-8779f1e2f320") : secret "llmisvc-webhook-server-cert" not found
Jul 09 09:40:37.379354 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.379338    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/53a33bf0-44ee-4195-9d34-ffaf0b0a357f-data\") pod \"seaweedfs-64568bcd49-8dccw\" (UID: \"53a33bf0-44ee-4195-9d34-ffaf0b0a357f\") " pod="kserve/seaweedfs-64568bcd49-8dccw"
Jul 09 09:40:37.388754 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.388732    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5qhg\" (UniqueName: \"kubernetes.io/projected/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-kube-api-access-l5qhg\") pod \"llmisvc-controller-manager-57d57b58b4-5rf4k\" (UID: \"1be7c91f-66f5-49a3-8c7d-8779f1e2f320\") " pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"
Jul 09 09:40:37.388903 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.388887    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f79q5\" (UniqueName: \"kubernetes.io/projected/53a33bf0-44ee-4195-9d34-ffaf0b0a357f-kube-api-access-f79q5\") pod \"seaweedfs-64568bcd49-8dccw\" (UID: \"53a33bf0-44ee-4195-9d34-ffaf0b0a357f\") " pod="kserve/seaweedfs-64568bcd49-8dccw"
Jul 09 09:40:37.469270 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.469243    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-8dccw"
Jul 09 09:40:37.597326 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.597295    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-8dccw"]
Jul 09 09:40:37.600476 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:40:37.600450    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a33bf0_44ee_4195_9d34_ffaf0b0a357f.slice/crio-1284e7071a82c1f3d368affc03c420f89abb297ce4e13da265120105bc3fa04e WatchSource:0}: Error finding container 1284e7071a82c1f3d368affc03c420f89abb297ce4e13da265120105bc3fa04e: Status 404 returned error can't find the container with id 1284e7071a82c1f3d368affc03c420f89abb297ce4e13da265120105bc3fa04e
Jul 09 09:40:37.782653 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.782622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f94cfcee-b683-43d0-90fd-a9437bfc32ee-cert\") pod \"kserve-controller-manager-5876b4f96d-hvhnk\" (UID: \"f94cfcee-b683-43d0-90fd-a9437bfc32ee\") " pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk"
Jul 09 09:40:37.784944 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.784924    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f94cfcee-b683-43d0-90fd-a9437bfc32ee-cert\") pod \"kserve-controller-manager-5876b4f96d-hvhnk\" (UID: \"f94cfcee-b683-43d0-90fd-a9437bfc32ee\") " pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk"
Jul 09 09:40:37.883672 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.883644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-cert\") pod \"llmisvc-controller-manager-57d57b58b4-5rf4k\" (UID: \"1be7c91f-66f5-49a3-8c7d-8779f1e2f320\") " pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"
Jul 09 09:40:37.885736 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:37.885714    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-cert\") pod \"llmisvc-controller-manager-57d57b58b4-5rf4k\" (UID: \"1be7c91f-66f5-49a3-8c7d-8779f1e2f320\") " pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"
Jul 09 09:40:38.012139 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:38.012065    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk"
Jul 09 09:40:38.034270 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:38.034198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"
Jul 09 09:40:38.176440 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:38.176413    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5876b4f96d-hvhnk"]
Jul 09 09:40:38.202315 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:38.202049    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"]
Jul 09 09:40:38.248242 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:40:38.248210    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf94cfcee_b683_43d0_90fd_a9437bfc32ee.slice/crio-228e018c7d76ea497a93d798f199891aa2bac9e75dc792a149c1fcedfc5d6d8c WatchSource:0}: Error finding container 228e018c7d76ea497a93d798f199891aa2bac9e75dc792a149c1fcedfc5d6d8c: Status 404 returned error can't find the container with id 228e018c7d76ea497a93d798f199891aa2bac9e75dc792a149c1fcedfc5d6d8c
Jul 09 09:40:38.248893 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:40:38.248865    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1be7c91f_66f5_49a3_8c7d_8779f1e2f320.slice/crio-8ad0455c98dd28ccba193f4b332fc5cbbfb541564cb53e9399169b4ea11870ff WatchSource:0}: Error finding container 8ad0455c98dd28ccba193f4b332fc5cbbfb541564cb53e9399169b4ea11870ff: Status 404 returned error can't find the container with id 8ad0455c98dd28ccba193f4b332fc5cbbfb541564cb53e9399169b4ea11870ff
Jul 09 09:40:38.396323 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:38.396254    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-8dccw" event={"ID":"53a33bf0-44ee-4195-9d34-ffaf0b0a357f","Type":"ContainerStarted","Data":"1284e7071a82c1f3d368affc03c420f89abb297ce4e13da265120105bc3fa04e"}
Jul 09 09:40:38.397718 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:38.397689    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k" event={"ID":"1be7c91f-66f5-49a3-8c7d-8779f1e2f320","Type":"ContainerStarted","Data":"8ad0455c98dd28ccba193f4b332fc5cbbfb541564cb53e9399169b4ea11870ff"}
Jul 09 09:40:38.399080 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:38.399052    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk" event={"ID":"f94cfcee-b683-43d0-90fd-a9437bfc32ee","Type":"ContainerStarted","Data":"228e018c7d76ea497a93d798f199891aa2bac9e75dc792a149c1fcedfc5d6d8c"}
Jul 09 09:40:39.937747 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:39.937669    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:40:40.041168 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:40.041025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:40:43.428397 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:43.428364    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk" event={"ID":"f94cfcee-b683-43d0-90fd-a9437bfc32ee","Type":"ContainerStarted","Data":"36af742a187055561be63e044d14f7c7e5ab8378c2d1851292b12df2779439ea"}
Jul 09 09:40:43.428834 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:43.428456    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk"
Jul 09 09:40:43.430093 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:43.430065    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-8dccw" event={"ID":"53a33bf0-44ee-4195-9d34-ffaf0b0a357f","Type":"ContainerStarted","Data":"d48a6db541a874837710d12fcfebf82ea5c1165e8f84edea8ff2afcfa4bd77f7"}
Jul 09 09:40:43.430203 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:43.430114    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-8dccw"
Jul 09 09:40:43.431408 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:43.431387    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k" event={"ID":"1be7c91f-66f5-49a3-8c7d-8779f1e2f320","Type":"ContainerStarted","Data":"e10bf1cc016f7e977f7374620919e52e1108e98f5a11de2f99ce5e56e5dd6b61"}
Jul 09 09:40:43.431499 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:43.431436    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"
Jul 09 09:40:43.446658 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:43.446618    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk" podStartSLOduration=2.910788434 podStartE2EDuration="6.446602125s" totalImagesPullingTime="3.535813691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:40:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:40:43.444982719 +0000 UTC m=+446.451151809" watchObservedRunningTime="2026-07-09 09:40:43.446602125 +0000 UTC m=+446.452771199"
Jul 09 09:40:43.462157 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:43.462112    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k" podStartSLOduration=2.062471979 podStartE2EDuration="6.462100885s" totalImagesPullingTime="4.399628906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:40:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:40:43.461821637 +0000 UTC m=+446.467990728" watchObservedRunningTime="2026-07-09 09:40:43.462100885 +0000 UTC m=+446.468269956"
Jul 09 09:40:43.478622 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:43.478582    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-8dccw" podStartSLOduration=1.479955421 podStartE2EDuration="6.478571495s" totalImagesPullingTime="4.998616074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:40:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:40:43.477099252 +0000 UTC m=+446.483268350" watchObservedRunningTime="2026-07-09 09:40:43.478571495 +0000 UTC m=+446.484740566"
Jul 09 09:40:49.437287 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:40:49.437251    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-8dccw"
Jul 09 09:41:14.437778 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:14.437747    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"
Jul 09 09:41:14.440892 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:14.440871    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk"
Jul 09 09:41:15.871493 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:15.871462    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:15.971732 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:15.971694    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:16.152011 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.151941    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"]
Jul 09 09:41:16.216963 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.216942    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"]
Jul 09 09:41:16.217086 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.217030    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"
Jul 09 09:41:16.219738 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.219715    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 09:41:16.219868 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.219748    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 09:41:16.220571 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.220551    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gp9wj\""
Jul 09 09:41:16.352412 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.352388    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89\" (UID: \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"
Jul 09 09:41:16.352505 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.352421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89\" (UID: \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"
Jul 09 09:41:16.352505 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.352480    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggs8j\" (UniqueName: \"kubernetes.io/projected/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-kube-api-access-ggs8j\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89\" (UID: \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"
Jul 09 09:41:16.453309 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.453255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89\" (UID: \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"
Jul 09 09:41:16.453309 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.453287    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89\" (UID: \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"
Jul 09 09:41:16.453465 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.453326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggs8j\" (UniqueName: \"kubernetes.io/projected/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-kube-api-access-ggs8j\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89\" (UID: \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"
Jul 09 09:41:16.453625 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.453606    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89\" (UID: \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"
Jul 09 09:41:16.453693 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.453650    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89\" (UID: \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"
Jul 09 09:41:16.474308 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.474284    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggs8j\" (UniqueName: \"kubernetes.io/projected/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-kube-api-access-ggs8j\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89\" (UID: \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"
Jul 09 09:41:16.526100 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.526079    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"
Jul 09 09:41:16.673697 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:16.673674    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"]
Jul 09 09:41:16.675921 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:41:16.675895    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b5e53ec_5b0b_46e0_9ec8_929b6f91876c.slice/crio-55541d01add86903a993ce9952c766b5217c70bf60f50760c1e2be0bdd46fa58 WatchSource:0}: Error finding container 55541d01add86903a993ce9952c766b5217c70bf60f50760c1e2be0bdd46fa58: Status 404 returned error can't find the container with id 55541d01add86903a993ce9952c766b5217c70bf60f50760c1e2be0bdd46fa58
Jul 09 09:41:17.216198 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:17.216162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:17.578859 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:17.578826    2577 generic.go:356] "Generic (PLEG): container finished" podID="5b5e53ec-5b0b-46e0-9ec8-929b6f91876c" containerID="83b293c9edf29cab1d17b2bd8b61e761a1804e259944c62fcc5c6e42523cd9b3" exitCode=0
Jul 09 09:41:17.579003 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:17.578915    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89" event={"ID":"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c","Type":"ContainerDied","Data":"83b293c9edf29cab1d17b2bd8b61e761a1804e259944c62fcc5c6e42523cd9b3"}
Jul 09 09:41:17.579003 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:17.578967    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89" event={"ID":"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c","Type":"ContainerStarted","Data":"55541d01add86903a993ce9952c766b5217c70bf60f50760c1e2be0bdd46fa58"}
Jul 09 09:41:18.585075 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:18.585003    2577 generic.go:356] "Generic (PLEG): container finished" podID="5b5e53ec-5b0b-46e0-9ec8-929b6f91876c" containerID="faab777cebe59425b483469e49e91d37453e82d5a4a1080148eb923da9ae8a36" exitCode=0
Jul 09 09:41:18.585409 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:18.585092    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89" event={"ID":"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c","Type":"ContainerDied","Data":"faab777cebe59425b483469e49e91d37453e82d5a4a1080148eb923da9ae8a36"}
Jul 09 09:41:19.591331 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:19.591290    2577 generic.go:356] "Generic (PLEG): container finished" podID="5b5e53ec-5b0b-46e0-9ec8-929b6f91876c" containerID="a3a3fe2098e23299f054599bcb1a9d20f5465b152184d3fbc2f73e150c971f97" exitCode=0
Jul 09 09:41:19.591700 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:19.591364    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89" event={"ID":"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c","Type":"ContainerDied","Data":"a3a3fe2098e23299f054599bcb1a9d20f5465b152184d3fbc2f73e150c971f97"}
Jul 09 09:41:20.714085 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:20.714063    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"
Jul 09 09:41:20.887568 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:20.887495    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-bundle\") pod \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\" (UID: \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\") "
Jul 09 09:41:20.887568 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:20.887538    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggs8j\" (UniqueName: \"kubernetes.io/projected/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-kube-api-access-ggs8j\") pod \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\" (UID: \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\") "
Jul 09 09:41:20.887568 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:20.887565    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-util\") pod \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\" (UID: \"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c\") "
Jul 09 09:41:20.888306 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:20.888277    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-bundle" (OuterVolumeSpecName: "bundle") pod "5b5e53ec-5b0b-46e0-9ec8-929b6f91876c" (UID: "5b5e53ec-5b0b-46e0-9ec8-929b6f91876c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:41:20.889522 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:20.889491    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-kube-api-access-ggs8j" (OuterVolumeSpecName: "kube-api-access-ggs8j") pod "5b5e53ec-5b0b-46e0-9ec8-929b6f91876c" (UID: "5b5e53ec-5b0b-46e0-9ec8-929b6f91876c"). InnerVolumeSpecName "kube-api-access-ggs8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:41:20.892887 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:20.892865    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-util" (OuterVolumeSpecName: "util") pod "5b5e53ec-5b0b-46e0-9ec8-929b6f91876c" (UID: "5b5e53ec-5b0b-46e0-9ec8-929b6f91876c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:41:20.988577 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:20.988554    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-bundle\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:41:20.988577 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:20.988576    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggs8j\" (UniqueName: \"kubernetes.io/projected/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-kube-api-access-ggs8j\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:41:20.988697 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:20.988586    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b5e53ec-5b0b-46e0-9ec8-929b6f91876c-util\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:41:21.601091 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:21.601063    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89"
Jul 09 09:41:21.601091 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:21.601074    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pvr89" event={"ID":"5b5e53ec-5b0b-46e0-9ec8-929b6f91876c","Type":"ContainerDied","Data":"55541d01add86903a993ce9952c766b5217c70bf60f50760c1e2be0bdd46fa58"}
Jul 09 09:41:21.601306 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:21.601111    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55541d01add86903a993ce9952c766b5217c70bf60f50760c1e2be0bdd46fa58"
Jul 09 09:41:27.057603 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:27.057562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:27.712094 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:27.712064    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:27.849595 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:27.849568    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:28.001311 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:28.001235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:28.410947 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:28.410863    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:28.517295 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:28.517245    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:29.207819 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.207765    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp"]
Jul 09 09:41:29.208101 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.208089    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5e53ec-5b0b-46e0-9ec8-929b6f91876c" containerName="extract"
Jul 09 09:41:29.208145 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.208103    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5e53ec-5b0b-46e0-9ec8-929b6f91876c" containerName="extract"
Jul 09 09:41:29.208145 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.208118    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5e53ec-5b0b-46e0-9ec8-929b6f91876c" containerName="pull"
Jul 09 09:41:29.208145 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.208124    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5e53ec-5b0b-46e0-9ec8-929b6f91876c" containerName="pull"
Jul 09 09:41:29.208145 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.208137    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5e53ec-5b0b-46e0-9ec8-929b6f91876c" containerName="util"
Jul 09 09:41:29.208145 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.208143    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5e53ec-5b0b-46e0-9ec8-929b6f91876c" containerName="util"
Jul 09 09:41:29.208297 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.208194    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b5e53ec-5b0b-46e0-9ec8-929b6f91876c" containerName="extract"
Jul 09 09:41:29.211242 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.211227    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp"
Jul 09 09:41:29.214383 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.214364    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 09 09:41:29.214747 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.214727    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 09 09:41:29.215140 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.215121    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-gwrtb\""
Jul 09 09:41:29.230105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.230074    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp"]
Jul 09 09:41:29.247354 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.247328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c64501c-d716-4167-b282-43966798d844-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-fllnp\" (UID: \"7c64501c-d716-4167-b282-43966798d844\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp"
Jul 09 09:41:29.247438 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.247358    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm8x4\" (UniqueName: \"kubernetes.io/projected/7c64501c-d716-4167-b282-43966798d844-kube-api-access-dm8x4\") pod \"cert-manager-operator-controller-manager-684f79c9b9-fllnp\" (UID: \"7c64501c-d716-4167-b282-43966798d844\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp"
Jul 09 09:41:29.348311 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.348283    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c64501c-d716-4167-b282-43966798d844-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-fllnp\" (UID: \"7c64501c-d716-4167-b282-43966798d844\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp"
Jul 09 09:41:29.348401 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.348315    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm8x4\" (UniqueName: \"kubernetes.io/projected/7c64501c-d716-4167-b282-43966798d844-kube-api-access-dm8x4\") pod \"cert-manager-operator-controller-manager-684f79c9b9-fllnp\" (UID: \"7c64501c-d716-4167-b282-43966798d844\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp"
Jul 09 09:41:29.348644 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.348625    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c64501c-d716-4167-b282-43966798d844-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-fllnp\" (UID: \"7c64501c-d716-4167-b282-43966798d844\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp"
Jul 09 09:41:29.358748 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.358726    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm8x4\" (UniqueName: \"kubernetes.io/projected/7c64501c-d716-4167-b282-43966798d844-kube-api-access-dm8x4\") pod \"cert-manager-operator-controller-manager-684f79c9b9-fllnp\" (UID: \"7c64501c-d716-4167-b282-43966798d844\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp"
Jul 09 09:41:29.520144 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.520121    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp"
Jul 09 09:41:29.661829 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:29.661646    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp"]
Jul 09 09:41:29.664629 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:41:29.664600    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c64501c_d716_4167_b282_43966798d844.slice/crio-3bc0b1abe05da93ffe245706bcc68899e6ede9b212a828edd0ec67162eda9717 WatchSource:0}: Error finding container 3bc0b1abe05da93ffe245706bcc68899e6ede9b212a828edd0ec67162eda9717: Status 404 returned error can't find the container with id 3bc0b1abe05da93ffe245706bcc68899e6ede9b212a828edd0ec67162eda9717
Jul 09 09:41:30.648071 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:30.648036    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp" event={"ID":"7c64501c-d716-4167-b282-43966798d844","Type":"ContainerStarted","Data":"3bc0b1abe05da93ffe245706bcc68899e6ede9b212a828edd0ec67162eda9717"}
Jul 09 09:41:30.876115 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:30.876061    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:30.976697 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:30.976651    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:31.655035 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:31.654954    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp" event={"ID":"7c64501c-d716-4167-b282-43966798d844","Type":"ContainerStarted","Data":"574d85cf4c71a98f6887aa2f2a69c900b1678ace42b50d3cc703035a34051399"}
Jul 09 09:41:31.710077 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:31.710028    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fllnp" podStartSLOduration=1.12636762 podStartE2EDuration="2.710013644s" totalImagesPullingTime="1.583646024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:41:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:41:31.705771116 +0000 UTC m=+494.711940186" watchObservedRunningTime="2026-07-09 09:41:31.710013644 +0000 UTC m=+494.716182714"
Jul 09 09:41:34.630268 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:34.630226    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:34.764174 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:34.764142    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:35.027022 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:35.026978    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:35.879320 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:35.879288    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:35.950839 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:35.950812    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:37.339818 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:37.339770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:37.438444 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:37.438411    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:37.536432 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:37.536406    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:38.291292 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.291253    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"]
Jul 09 09:41:38.294923 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.294907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"
Jul 09 09:41:38.297732 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.297713    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gp9wj\""
Jul 09 09:41:38.298068 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.298054    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 09:41:38.298575 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.298559    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 09:41:38.311773 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.311753    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"]
Jul 09 09:41:38.317980 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.317960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8923182-a557-4751-9932-2f834c0e2ee2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs\" (UID: \"f8923182-a557-4751-9932-2f834c0e2ee2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"
Jul 09 09:41:38.318080 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.317991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8923182-a557-4751-9932-2f834c0e2ee2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs\" (UID: \"f8923182-a557-4751-9932-2f834c0e2ee2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"
Jul 09 09:41:38.318080 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.318031    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qstx\" (UniqueName: \"kubernetes.io/projected/f8923182-a557-4751-9932-2f834c0e2ee2-kube-api-access-6qstx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs\" (UID: \"f8923182-a557-4751-9932-2f834c0e2ee2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"
Jul 09 09:41:38.418666 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.418638    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qstx\" (UniqueName: \"kubernetes.io/projected/f8923182-a557-4751-9932-2f834c0e2ee2-kube-api-access-6qstx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs\" (UID: \"f8923182-a557-4751-9932-2f834c0e2ee2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"
Jul 09 09:41:38.419075 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.418728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8923182-a557-4751-9932-2f834c0e2ee2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs\" (UID: \"f8923182-a557-4751-9932-2f834c0e2ee2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"
Jul 09 09:41:38.419075 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.418856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8923182-a557-4751-9932-2f834c0e2ee2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs\" (UID: \"f8923182-a557-4751-9932-2f834c0e2ee2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"
Jul 09 09:41:38.419199 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.419150    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8923182-a557-4751-9932-2f834c0e2ee2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs\" (UID: \"f8923182-a557-4751-9932-2f834c0e2ee2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"
Jul 09 09:41:38.419246 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.419189    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8923182-a557-4751-9932-2f834c0e2ee2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs\" (UID: \"f8923182-a557-4751-9932-2f834c0e2ee2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"
Jul 09 09:41:38.428116 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.428091    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qstx\" (UniqueName: \"kubernetes.io/projected/f8923182-a557-4751-9932-2f834c0e2ee2-kube-api-access-6qstx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs\" (UID: \"f8923182-a557-4751-9932-2f834c0e2ee2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"
Jul 09 09:41:38.604548 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.604467    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"
Jul 09 09:41:38.765228 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.765126    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"]
Jul 09 09:41:38.769932 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:41:38.769904    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8923182_a557_4751_9932_2f834c0e2ee2.slice/crio-d85819b176f875e165a990b5d97d3f93cd17f6c1456a0cf7be2836fb6a5dafb6 WatchSource:0}: Error finding container d85819b176f875e165a990b5d97d3f93cd17f6c1456a0cf7be2836fb6a5dafb6: Status 404 returned error can't find the container with id d85819b176f875e165a990b5d97d3f93cd17f6c1456a0cf7be2836fb6a5dafb6
Jul 09 09:41:38.811446 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:38.811420    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-849f748b57-hlgbt"]
Jul 09 09:41:39.696658 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:39.696574    2577 generic.go:356] "Generic (PLEG): container finished" podID="f8923182-a557-4751-9932-2f834c0e2ee2" containerID="ec92d9e0d3c9316cd55f7886c45952c9c364e0e15b935377ded7f863be3c24ac" exitCode=0
Jul 09 09:41:39.697326 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:39.696651    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs" event={"ID":"f8923182-a557-4751-9932-2f834c0e2ee2","Type":"ContainerDied","Data":"ec92d9e0d3c9316cd55f7886c45952c9c364e0e15b935377ded7f863be3c24ac"}
Jul 09 09:41:39.697326 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:39.696703    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs" event={"ID":"f8923182-a557-4751-9932-2f834c0e2ee2","Type":"ContainerStarted","Data":"d85819b176f875e165a990b5d97d3f93cd17f6c1456a0cf7be2836fb6a5dafb6"}
Jul 09 09:41:41.408349 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:41.408303    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:41.508245 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:41.508213    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:41.708693 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:41.708663    2577 generic.go:356] "Generic (PLEG): container finished" podID="f8923182-a557-4751-9932-2f834c0e2ee2" containerID="fa61f275a338962e22c6a55c486fc9727d8f63ea9bdc5717cc094e0a2cce9c30" exitCode=0
Jul 09 09:41:41.708788 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:41.708740    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs" event={"ID":"f8923182-a557-4751-9932-2f834c0e2ee2","Type":"ContainerDied","Data":"fa61f275a338962e22c6a55c486fc9727d8f63ea9bdc5717cc094e0a2cce9c30"}
Jul 09 09:41:42.715639 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:42.715608    2577 generic.go:356] "Generic (PLEG): container finished" podID="f8923182-a557-4751-9932-2f834c0e2ee2" containerID="7b6dcc49dc085581a6e186d86e76b6813d54b0501320e9768eebb567818cfe67" exitCode=0
Jul 09 09:41:42.716031 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:42.715679    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs" event={"ID":"f8923182-a557-4751-9932-2f834c0e2ee2","Type":"ContainerDied","Data":"7b6dcc49dc085581a6e186d86e76b6813d54b0501320e9768eebb567818cfe67"}
Jul 09 09:41:42.886666 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:42.886630    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:43.848656 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:43.848630    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"
Jul 09 09:41:43.961095 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:43.961071    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qstx\" (UniqueName: \"kubernetes.io/projected/f8923182-a557-4751-9932-2f834c0e2ee2-kube-api-access-6qstx\") pod \"f8923182-a557-4751-9932-2f834c0e2ee2\" (UID: \"f8923182-a557-4751-9932-2f834c0e2ee2\") "
Jul 09 09:41:43.961220 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:43.961144    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8923182-a557-4751-9932-2f834c0e2ee2-bundle\") pod \"f8923182-a557-4751-9932-2f834c0e2ee2\" (UID: \"f8923182-a557-4751-9932-2f834c0e2ee2\") "
Jul 09 09:41:43.961220 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:43.961169    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8923182-a557-4751-9932-2f834c0e2ee2-util\") pod \"f8923182-a557-4751-9932-2f834c0e2ee2\" (UID: \"f8923182-a557-4751-9932-2f834c0e2ee2\") "
Jul 09 09:41:43.961549 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:43.961512    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8923182-a557-4751-9932-2f834c0e2ee2-bundle" (OuterVolumeSpecName: "bundle") pod "f8923182-a557-4751-9932-2f834c0e2ee2" (UID: "f8923182-a557-4751-9932-2f834c0e2ee2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:41:43.963145 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:43.963115    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8923182-a557-4751-9932-2f834c0e2ee2-kube-api-access-6qstx" (OuterVolumeSpecName: "kube-api-access-6qstx") pod "f8923182-a557-4751-9932-2f834c0e2ee2" (UID: "f8923182-a557-4751-9932-2f834c0e2ee2"). InnerVolumeSpecName "kube-api-access-6qstx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:41:43.966716 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:43.966697    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8923182-a557-4751-9932-2f834c0e2ee2-util" (OuterVolumeSpecName: "util") pod "f8923182-a557-4751-9932-2f834c0e2ee2" (UID: "f8923182-a557-4751-9932-2f834c0e2ee2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:41:44.062405 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:44.062357    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8923182-a557-4751-9932-2f834c0e2ee2-bundle\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:41:44.062405 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:44.062377    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8923182-a557-4751-9932-2f834c0e2ee2-util\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:41:44.062405 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:44.062386    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qstx\" (UniqueName: \"kubernetes.io/projected/f8923182-a557-4751-9932-2f834c0e2ee2-kube-api-access-6qstx\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:41:44.726401 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:44.726375    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs"
Jul 09 09:41:44.726401 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:44.726390    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnhqfs" event={"ID":"f8923182-a557-4751-9932-2f834c0e2ee2","Type":"ContainerDied","Data":"d85819b176f875e165a990b5d97d3f93cd17f6c1456a0cf7be2836fb6a5dafb6"}
Jul 09 09:41:44.726617 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:44.726438    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d85819b176f875e165a990b5d97d3f93cd17f6c1456a0cf7be2836fb6a5dafb6"
Jul 09 09:41:45.738043 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:45.738007    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:45.841980 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:45.841945    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:47.353189 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:47.353156    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:47.417144 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:47.417115    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:41:55.876966 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:41:55.876927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:01.476424 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.476384    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:01.480535 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.480508    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"]
Jul 09 09:42:01.480864 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.480852    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8923182-a557-4751-9932-2f834c0e2ee2" containerName="pull"
Jul 09 09:42:01.480912 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.480865    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8923182-a557-4751-9932-2f834c0e2ee2" containerName="pull"
Jul 09 09:42:01.480912 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.480889    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8923182-a557-4751-9932-2f834c0e2ee2" containerName="util"
Jul 09 09:42:01.480912 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.480894    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8923182-a557-4751-9932-2f834c0e2ee2" containerName="util"
Jul 09 09:42:01.480912 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.480901    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8923182-a557-4751-9932-2f834c0e2ee2" containerName="extract"
Jul 09 09:42:01.480912 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.480906    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8923182-a557-4751-9932-2f834c0e2ee2" containerName="extract"
Jul 09 09:42:01.481059 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.480960    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8923182-a557-4751-9932-2f834c0e2ee2" containerName="extract"
Jul 09 09:42:01.485535 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.485518    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.488051 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.488019    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 09 09:42:01.488168 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.488052    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 09 09:42:01.488168 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.488053    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 09 09:42:01.488168 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.488120    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-z777s\""
Jul 09 09:42:01.488616 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.488602    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 09 09:42:01.500846 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.500788    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"]
Jul 09 09:42:01.546931 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.546906    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:01.579112 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.579088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5581b14c-ab85-4db6-9dad-987ab3835f48-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.579210 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.579113    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5581b14c-ab85-4db6-9dad-987ab3835f48-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.579210 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.579150    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5581b14c-ab85-4db6-9dad-987ab3835f48-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.579298 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.579254    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5581b14c-ab85-4db6-9dad-987ab3835f48-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.579336 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.579295    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jd6p\" (UniqueName: \"kubernetes.io/projected/5581b14c-ab85-4db6-9dad-987ab3835f48-kube-api-access-8jd6p\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.579383 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.579333    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5581b14c-ab85-4db6-9dad-987ab3835f48-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.579428 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.579412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5581b14c-ab85-4db6-9dad-987ab3835f48-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.680507 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.680483    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5581b14c-ab85-4db6-9dad-987ab3835f48-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.680615 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.680510    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jd6p\" (UniqueName: \"kubernetes.io/projected/5581b14c-ab85-4db6-9dad-987ab3835f48-kube-api-access-8jd6p\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.680615 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.680532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5581b14c-ab85-4db6-9dad-987ab3835f48-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.680615 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.680551    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5581b14c-ab85-4db6-9dad-987ab3835f48-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.680776 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.680674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5581b14c-ab85-4db6-9dad-987ab3835f48-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.680776 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.680710    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5581b14c-ab85-4db6-9dad-987ab3835f48-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.680776 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.680762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5581b14c-ab85-4db6-9dad-987ab3835f48-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.681229 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.681204    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5581b14c-ab85-4db6-9dad-987ab3835f48-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.682869 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.682851    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5581b14c-ab85-4db6-9dad-987ab3835f48-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.682964 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.682895    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5581b14c-ab85-4db6-9dad-987ab3835f48-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.683250 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.683230    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5581b14c-ab85-4db6-9dad-987ab3835f48-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.683317 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.683247    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5581b14c-ab85-4db6-9dad-987ab3835f48-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.688446 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.688421    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5581b14c-ab85-4db6-9dad-987ab3835f48-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.688759 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.688739    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jd6p\" (UniqueName: \"kubernetes.io/projected/5581b14c-ab85-4db6-9dad-987ab3835f48-kube-api-access-8jd6p\") pod \"istiod-openshift-gateway-94bb8fbfd-m2d5h\" (UID: \"5581b14c-ab85-4db6-9dad-987ab3835f48\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.794827 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.794765    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:01.927048 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:01.927020    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"]
Jul 09 09:42:01.928961 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:42:01.928930    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5581b14c_ab85_4db6_9dad_987ab3835f48.slice/crio-e90f89042ae812630b41938b50027b108decde101f19e550a6f2b842cc64f8a2 WatchSource:0}: Error finding container e90f89042ae812630b41938b50027b108decde101f19e550a6f2b842cc64f8a2: Status 404 returned error can't find the container with id e90f89042ae812630b41938b50027b108decde101f19e550a6f2b842cc64f8a2
Jul 09 09:42:02.487850 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:02.487820    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:02.809156 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:02.809077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h" event={"ID":"5581b14c-ab85-4db6-9dad-987ab3835f48","Type":"ContainerStarted","Data":"e90f89042ae812630b41938b50027b108decde101f19e550a6f2b842cc64f8a2"}
Jul 09 09:42:03.833549 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:03.833504    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-849f748b57-hlgbt" podUID="6d3e379c-7887-4e62-ad65-951437849a85" containerName="console" containerID="cri-o://05d261a7ddfa046ed6dc38da5d409436bfe598966658a3fec4043ebb8d6a5c96" gracePeriod=15
Jul 09 09:42:04.427373 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.427336    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 09:42:04.427465 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.427405    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 09:42:04.510661 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.510638    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:42:04.605484 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.605455    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-trusted-ca-bundle\") pod \"6d3e379c-7887-4e62-ad65-951437849a85\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") "
Jul 09 09:42:04.605651 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.605493    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7xn4\" (UniqueName: \"kubernetes.io/projected/6d3e379c-7887-4e62-ad65-951437849a85-kube-api-access-r7xn4\") pod \"6d3e379c-7887-4e62-ad65-951437849a85\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") "
Jul 09 09:42:04.605651 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.605538    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-oauth-serving-cert\") pod \"6d3e379c-7887-4e62-ad65-951437849a85\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") "
Jul 09 09:42:04.605651 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.605559    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d3e379c-7887-4e62-ad65-951437849a85-console-serving-cert\") pod \"6d3e379c-7887-4e62-ad65-951437849a85\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") "
Jul 09 09:42:04.605651 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.605585    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d3e379c-7887-4e62-ad65-951437849a85-console-oauth-config\") pod \"6d3e379c-7887-4e62-ad65-951437849a85\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") "
Jul 09 09:42:04.605651 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.605609    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-console-config\") pod \"6d3e379c-7887-4e62-ad65-951437849a85\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") "
Jul 09 09:42:04.605651 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.605641    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-service-ca\") pod \"6d3e379c-7887-4e62-ad65-951437849a85\" (UID: \"6d3e379c-7887-4e62-ad65-951437849a85\") "
Jul 09 09:42:04.606035 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.605922    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6d3e379c-7887-4e62-ad65-951437849a85" (UID: "6d3e379c-7887-4e62-ad65-951437849a85"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:42:04.606035 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.605946    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6d3e379c-7887-4e62-ad65-951437849a85" (UID: "6d3e379c-7887-4e62-ad65-951437849a85"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:42:04.606238 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.606193    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-service-ca" (OuterVolumeSpecName: "service-ca") pod "6d3e379c-7887-4e62-ad65-951437849a85" (UID: "6d3e379c-7887-4e62-ad65-951437849a85"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:42:04.606364 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.606297    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-console-config" (OuterVolumeSpecName: "console-config") pod "6d3e379c-7887-4e62-ad65-951437849a85" (UID: "6d3e379c-7887-4e62-ad65-951437849a85"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:42:04.607764 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.607737    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d3e379c-7887-4e62-ad65-951437849a85-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6d3e379c-7887-4e62-ad65-951437849a85" (UID: "6d3e379c-7887-4e62-ad65-951437849a85"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:42:04.607890 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.607765    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d3e379c-7887-4e62-ad65-951437849a85-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6d3e379c-7887-4e62-ad65-951437849a85" (UID: "6d3e379c-7887-4e62-ad65-951437849a85"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:42:04.607981 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.607961    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d3e379c-7887-4e62-ad65-951437849a85-kube-api-access-r7xn4" (OuterVolumeSpecName: "kube-api-access-r7xn4") pod "6d3e379c-7887-4e62-ad65-951437849a85" (UID: "6d3e379c-7887-4e62-ad65-951437849a85"). InnerVolumeSpecName "kube-api-access-r7xn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:42:04.706818 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.706776    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-trusted-ca-bundle\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:04.706917 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.706827    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7xn4\" (UniqueName: \"kubernetes.io/projected/6d3e379c-7887-4e62-ad65-951437849a85-kube-api-access-r7xn4\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:04.706917 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.706838    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-oauth-serving-cert\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:04.706917 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.706846    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d3e379c-7887-4e62-ad65-951437849a85-console-serving-cert\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:04.706917 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.706857    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d3e379c-7887-4e62-ad65-951437849a85-console-oauth-config\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:04.706917 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.706867    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-console-config\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:04.706917 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.706875    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d3e379c-7887-4e62-ad65-951437849a85-service-ca\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:04.821438 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.821410    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h" event={"ID":"5581b14c-ab85-4db6-9dad-987ab3835f48","Type":"ContainerStarted","Data":"28a512d8c30ec4eecc7f5cd8539e9dd864562a44cc77c199e725d5903522f230"}
Jul 09 09:42:04.821554 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.821466    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:04.822766 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.822743    2577 generic.go:356] "Generic (PLEG): container finished" podID="6d3e379c-7887-4e62-ad65-951437849a85" containerID="05d261a7ddfa046ed6dc38da5d409436bfe598966658a3fec4043ebb8d6a5c96" exitCode=0
Jul 09 09:42:04.822874 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.822774    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-849f748b57-hlgbt" event={"ID":"6d3e379c-7887-4e62-ad65-951437849a85","Type":"ContainerDied","Data":"05d261a7ddfa046ed6dc38da5d409436bfe598966658a3fec4043ebb8d6a5c96"}
Jul 09 09:42:04.822874 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.822782    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-849f748b57-hlgbt"
Jul 09 09:42:04.822874 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.822814    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-849f748b57-hlgbt" event={"ID":"6d3e379c-7887-4e62-ad65-951437849a85","Type":"ContainerDied","Data":"2e006e7a3606ea498f84e99b88a7e926e5fb72b528bf6af6907a84d814717969"}
Jul 09 09:42:04.822874 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.822835    2577 scope.go:117] "RemoveContainer" containerID="05d261a7ddfa046ed6dc38da5d409436bfe598966658a3fec4043ebb8d6a5c96"
Jul 09 09:42:04.832076 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.832057    2577 scope.go:117] "RemoveContainer" containerID="05d261a7ddfa046ed6dc38da5d409436bfe598966658a3fec4043ebb8d6a5c96"
Jul 09 09:42:04.832354 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:42:04.832336    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05d261a7ddfa046ed6dc38da5d409436bfe598966658a3fec4043ebb8d6a5c96\": container with ID starting with 05d261a7ddfa046ed6dc38da5d409436bfe598966658a3fec4043ebb8d6a5c96 not found: ID does not exist" containerID="05d261a7ddfa046ed6dc38da5d409436bfe598966658a3fec4043ebb8d6a5c96"
Jul 09 09:42:04.832422 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.832360    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05d261a7ddfa046ed6dc38da5d409436bfe598966658a3fec4043ebb8d6a5c96"} err="failed to get container status \"05d261a7ddfa046ed6dc38da5d409436bfe598966658a3fec4043ebb8d6a5c96\": rpc error: code = NotFound desc = could not find container \"05d261a7ddfa046ed6dc38da5d409436bfe598966658a3fec4043ebb8d6a5c96\": container with ID starting with 05d261a7ddfa046ed6dc38da5d409436bfe598966658a3fec4043ebb8d6a5c96 not found: ID does not exist"
Jul 09 09:42:04.846869 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.846828    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h" podStartSLOduration=1.350620291 podStartE2EDuration="3.84678865s" totalImagesPullingTime="2.496168359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:42:04.845942005 +0000 UTC m=+527.852111101" watchObservedRunningTime="2026-07-09 09:42:04.84678865 +0000 UTC m=+527.852957720"
Jul 09 09:42:04.860335 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.860311    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-849f748b57-hlgbt"]
Jul 09 09:42:04.865537 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:04.865483    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-849f748b57-hlgbt"]
Jul 09 09:42:05.280301 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:05.280265    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:05.383017 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:05.382977    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:05.552118 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:05.552050    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d3e379c-7887-4e62-ad65-951437849a85" path="/var/lib/kubelet/pods/6d3e379c-7887-4e62-ad65-951437849a85/volumes"
Jul 09 09:42:05.829979 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:05.829909    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-m2d5h container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 09 09:42:05.830227 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:05.829964    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h" podUID="5581b14c-ab85-4db6-9dad-987ab3835f48" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:42:08.185988 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:08.185953    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:08.259935 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:08.259906    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:08.829676 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:08.829646    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-m2d5h"
Jul 09 09:42:09.509131 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:09.509087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:09.591076 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:09.591045    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:12.877424 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:12.877379    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:17.044311 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:17.044280    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:17.160459 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:17.160434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:19.397196 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.397161    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"]
Jul 09 09:42:19.397554 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.397531    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d3e379c-7887-4e62-ad65-951437849a85" containerName="console"
Jul 09 09:42:19.397554 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.397544    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d3e379c-7887-4e62-ad65-951437849a85" containerName="console"
Jul 09 09:42:19.397624 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.397609    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d3e379c-7887-4e62-ad65-951437849a85" containerName="console"
Jul 09 09:42:19.404232 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.404210    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"
Jul 09 09:42:19.406984 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.406961    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 09:42:19.407099 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.407046    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gp9wj\""
Jul 09 09:42:19.407920 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.407903    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 09:42:19.410058 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.410040    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"]
Jul 09 09:42:19.495484 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.495461    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"]
Jul 09 09:42:19.498912 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.498897    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"
Jul 09 09:42:19.508536 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.508514    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"]
Jul 09 09:42:19.519156 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.519136    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbzrm\" (UniqueName: \"kubernetes.io/projected/a17e7fca-e855-4959-86c9-d0f3f6298898-kube-api-access-kbzrm\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f\" (UID: \"a17e7fca-e855-4959-86c9-d0f3f6298898\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"
Jul 09 09:42:19.519250 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.519186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a17e7fca-e855-4959-86c9-d0f3f6298898-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f\" (UID: \"a17e7fca-e855-4959-86c9-d0f3f6298898\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"
Jul 09 09:42:19.519305 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.519245    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a17e7fca-e855-4959-86c9-d0f3f6298898-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f\" (UID: \"a17e7fca-e855-4959-86c9-d0f3f6298898\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"
Jul 09 09:42:19.601399 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.601378    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"]
Jul 09 09:42:19.604782 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.604759    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"
Jul 09 09:42:19.615366 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.615346    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"]
Jul 09 09:42:19.620246 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.620227    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bec0891a-18f7-4a03-a160-98ac2efc4fb1-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c\" (UID: \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"
Jul 09 09:42:19.620338 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.620265    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbzrm\" (UniqueName: \"kubernetes.io/projected/a17e7fca-e855-4959-86c9-d0f3f6298898-kube-api-access-kbzrm\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f\" (UID: \"a17e7fca-e855-4959-86c9-d0f3f6298898\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"
Jul 09 09:42:19.620338 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.620292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bec0891a-18f7-4a03-a160-98ac2efc4fb1-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c\" (UID: \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"
Jul 09 09:42:19.620338 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.620313    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a17e7fca-e855-4959-86c9-d0f3f6298898-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f\" (UID: \"a17e7fca-e855-4959-86c9-d0f3f6298898\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"
Jul 09 09:42:19.620338 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.620337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsgfx\" (UniqueName: \"kubernetes.io/projected/bec0891a-18f7-4a03-a160-98ac2efc4fb1-kube-api-access-wsgfx\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c\" (UID: \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"
Jul 09 09:42:19.620549 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.620357    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a17e7fca-e855-4959-86c9-d0f3f6298898-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f\" (UID: \"a17e7fca-e855-4959-86c9-d0f3f6298898\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"
Jul 09 09:42:19.620693 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.620673    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a17e7fca-e855-4959-86c9-d0f3f6298898-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f\" (UID: \"a17e7fca-e855-4959-86c9-d0f3f6298898\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"
Jul 09 09:42:19.620728 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.620699    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a17e7fca-e855-4959-86c9-d0f3f6298898-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f\" (UID: \"a17e7fca-e855-4959-86c9-d0f3f6298898\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"
Jul 09 09:42:19.628832 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.628792    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbzrm\" (UniqueName: \"kubernetes.io/projected/a17e7fca-e855-4959-86c9-d0f3f6298898-kube-api-access-kbzrm\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f\" (UID: \"a17e7fca-e855-4959-86c9-d0f3f6298898\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"
Jul 09 09:42:19.693693 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.693636    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"]
Jul 09 09:42:19.698144 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.698127    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"
Jul 09 09:42:19.706188 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.706059    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"]
Jul 09 09:42:19.714260 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.714243    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"
Jul 09 09:42:19.721686 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.721662    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fdgv\" (UniqueName: \"kubernetes.io/projected/d074b26a-d370-449f-af4d-081606c8436f-kube-api-access-4fdgv\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw\" (UID: \"d074b26a-d370-449f-af4d-081606c8436f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"
Jul 09 09:42:19.721812 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.721701    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsgfx\" (UniqueName: \"kubernetes.io/projected/bec0891a-18f7-4a03-a160-98ac2efc4fb1-kube-api-access-wsgfx\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c\" (UID: \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"
Jul 09 09:42:19.721812 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.721726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d074b26a-d370-449f-af4d-081606c8436f-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw\" (UID: \"d074b26a-d370-449f-af4d-081606c8436f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"
Jul 09 09:42:19.721812 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.721778    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bec0891a-18f7-4a03-a160-98ac2efc4fb1-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c\" (UID: \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"
Jul 09 09:42:19.722049 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.721962    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bec0891a-18f7-4a03-a160-98ac2efc4fb1-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c\" (UID: \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"
Jul 09 09:42:19.722049 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.721996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d074b26a-d370-449f-af4d-081606c8436f-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw\" (UID: \"d074b26a-d370-449f-af4d-081606c8436f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"
Jul 09 09:42:19.722218 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.722195    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bec0891a-18f7-4a03-a160-98ac2efc4fb1-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c\" (UID: \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"
Jul 09 09:42:19.722339 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.722320    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bec0891a-18f7-4a03-a160-98ac2efc4fb1-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c\" (UID: \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"
Jul 09 09:42:19.730302 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.730272    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsgfx\" (UniqueName: \"kubernetes.io/projected/bec0891a-18f7-4a03-a160-98ac2efc4fb1-kube-api-access-wsgfx\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c\" (UID: \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"
Jul 09 09:42:19.808143 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.808114    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"
Jul 09 09:42:19.823225 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.823193    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d074b26a-d370-449f-af4d-081606c8436f-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw\" (UID: \"d074b26a-d370-449f-af4d-081606c8436f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"
Jul 09 09:42:19.823359 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.823242    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fdgv\" (UniqueName: \"kubernetes.io/projected/d074b26a-d370-449f-af4d-081606c8436f-kube-api-access-4fdgv\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw\" (UID: \"d074b26a-d370-449f-af4d-081606c8436f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"
Jul 09 09:42:19.823359 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.823303    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d074b26a-d370-449f-af4d-081606c8436f-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw\" (UID: \"d074b26a-d370-449f-af4d-081606c8436f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"
Jul 09 09:42:19.823359 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.823345    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj9d4\" (UniqueName: \"kubernetes.io/projected/f1367cea-5a9c-403f-894b-55db9369988b-kube-api-access-zj9d4\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj\" (UID: \"f1367cea-5a9c-403f-894b-55db9369988b\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"
Jul 09 09:42:19.823523 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.823378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1367cea-5a9c-403f-894b-55db9369988b-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj\" (UID: \"f1367cea-5a9c-403f-894b-55db9369988b\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"
Jul 09 09:42:19.823523 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.823412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1367cea-5a9c-403f-894b-55db9369988b-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj\" (UID: \"f1367cea-5a9c-403f-894b-55db9369988b\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"
Jul 09 09:42:19.823876 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.823849    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d074b26a-d370-449f-af4d-081606c8436f-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw\" (UID: \"d074b26a-d370-449f-af4d-081606c8436f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"
Jul 09 09:42:19.823985 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.823954    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d074b26a-d370-449f-af4d-081606c8436f-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw\" (UID: \"d074b26a-d370-449f-af4d-081606c8436f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"
Jul 09 09:42:19.832647 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.832620    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fdgv\" (UniqueName: \"kubernetes.io/projected/d074b26a-d370-449f-af4d-081606c8436f-kube-api-access-4fdgv\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw\" (UID: \"d074b26a-d370-449f-af4d-081606c8436f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"
Jul 09 09:42:19.851151 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.851130    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"]
Jul 09 09:42:19.853774 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:42:19.853753    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda17e7fca_e855_4959_86c9_d0f3f6298898.slice/crio-4cafe35f268bc552bdd643933dc0402542125fc6be84f576380dcf956771f46c WatchSource:0}: Error finding container 4cafe35f268bc552bdd643933dc0402542125fc6be84f576380dcf956771f46c: Status 404 returned error can't find the container with id 4cafe35f268bc552bdd643933dc0402542125fc6be84f576380dcf956771f46c
Jul 09 09:42:19.894123 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.894097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f" event={"ID":"a17e7fca-e855-4959-86c9-d0f3f6298898","Type":"ContainerStarted","Data":"4cafe35f268bc552bdd643933dc0402542125fc6be84f576380dcf956771f46c"}
Jul 09 09:42:19.913541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.913519    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"
Jul 09 09:42:19.925714 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.924332    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj9d4\" (UniqueName: \"kubernetes.io/projected/f1367cea-5a9c-403f-894b-55db9369988b-kube-api-access-zj9d4\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj\" (UID: \"f1367cea-5a9c-403f-894b-55db9369988b\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"
Jul 09 09:42:19.925714 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.924695    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1367cea-5a9c-403f-894b-55db9369988b-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj\" (UID: \"f1367cea-5a9c-403f-894b-55db9369988b\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"
Jul 09 09:42:19.925714 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.925094    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1367cea-5a9c-403f-894b-55db9369988b-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj\" (UID: \"f1367cea-5a9c-403f-894b-55db9369988b\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"
Jul 09 09:42:19.925714 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.925178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1367cea-5a9c-403f-894b-55db9369988b-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj\" (UID: \"f1367cea-5a9c-403f-894b-55db9369988b\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"
Jul 09 09:42:19.925714 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.925460    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1367cea-5a9c-403f-894b-55db9369988b-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj\" (UID: \"f1367cea-5a9c-403f-894b-55db9369988b\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"
Jul 09 09:42:19.933887 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.933855    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj9d4\" (UniqueName: \"kubernetes.io/projected/f1367cea-5a9c-403f-894b-55db9369988b-kube-api-access-zj9d4\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj\" (UID: \"f1367cea-5a9c-403f-894b-55db9369988b\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"
Jul 09 09:42:19.955193 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:19.955167    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"]
Jul 09 09:42:19.956747 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:42:19.956712    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbec0891a_18f7_4a03_a160_98ac2efc4fb1.slice/crio-aa07bb388ed1e87121d6bf625887ad6e71d5bdf5e0e7d01ce34e4ca4f83ebf79 WatchSource:0}: Error finding container aa07bb388ed1e87121d6bf625887ad6e71d5bdf5e0e7d01ce34e4ca4f83ebf79: Status 404 returned error can't find the container with id aa07bb388ed1e87121d6bf625887ad6e71d5bdf5e0e7d01ce34e4ca4f83ebf79
Jul 09 09:42:20.009051 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.009027    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"
Jul 09 09:42:20.064451 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.064428    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"]
Jul 09 09:42:20.066390 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:42:20.066354    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd074b26a_d370_449f_af4d_081606c8436f.slice/crio-c525662977da8cf691f880301c58ce1a7b74866ae3b3c2ef8ec79307fd29435d WatchSource:0}: Error finding container c525662977da8cf691f880301c58ce1a7b74866ae3b3c2ef8ec79307fd29435d: Status 404 returned error can't find the container with id c525662977da8cf691f880301c58ce1a7b74866ae3b3c2ef8ec79307fd29435d
Jul 09 09:42:20.165466 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.165438    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"]
Jul 09 09:42:20.166943 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:42:20.166915    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1367cea_5a9c_403f_894b_55db9369988b.slice/crio-1328709a72da66184f78451519d7aaa64a7ec5b1584fcbca09c6b430c87667f9 WatchSource:0}: Error finding container 1328709a72da66184f78451519d7aaa64a7ec5b1584fcbca09c6b430c87667f9: Status 404 returned error can't find the container with id 1328709a72da66184f78451519d7aaa64a7ec5b1584fcbca09c6b430c87667f9
Jul 09 09:42:20.900572 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.900538    2577 generic.go:356] "Generic (PLEG): container finished" podID="bec0891a-18f7-4a03-a160-98ac2efc4fb1" containerID="a601ea7c45e6702fb823c689eccda3043296cf16a363a0b8b3b94e2578b22f8c" exitCode=0
Jul 09 09:42:20.900964 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.900638    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c" event={"ID":"bec0891a-18f7-4a03-a160-98ac2efc4fb1","Type":"ContainerDied","Data":"a601ea7c45e6702fb823c689eccda3043296cf16a363a0b8b3b94e2578b22f8c"}
Jul 09 09:42:20.900964 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.900674    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c" event={"ID":"bec0891a-18f7-4a03-a160-98ac2efc4fb1","Type":"ContainerStarted","Data":"aa07bb388ed1e87121d6bf625887ad6e71d5bdf5e0e7d01ce34e4ca4f83ebf79"}
Jul 09 09:42:20.902341 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.902314    2577 generic.go:356] "Generic (PLEG): container finished" podID="a17e7fca-e855-4959-86c9-d0f3f6298898" containerID="0f1ad910c5d8ada60382d9743147df7b474766fe3e20e20c0fc4bda43d1da13e" exitCode=0
Jul 09 09:42:20.902446 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.902414    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f" event={"ID":"a17e7fca-e855-4959-86c9-d0f3f6298898","Type":"ContainerDied","Data":"0f1ad910c5d8ada60382d9743147df7b474766fe3e20e20c0fc4bda43d1da13e"}
Jul 09 09:42:20.904107 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.904083    2577 generic.go:356] "Generic (PLEG): container finished" podID="f1367cea-5a9c-403f-894b-55db9369988b" containerID="d0d054516b6a4008e942729ecc09991d4108844696e68cc6dbc79715037e7b00" exitCode=0
Jul 09 09:42:20.904210 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.904185    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj" event={"ID":"f1367cea-5a9c-403f-894b-55db9369988b","Type":"ContainerDied","Data":"d0d054516b6a4008e942729ecc09991d4108844696e68cc6dbc79715037e7b00"}
Jul 09 09:42:20.904272 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.904233    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj" event={"ID":"f1367cea-5a9c-403f-894b-55db9369988b","Type":"ContainerStarted","Data":"1328709a72da66184f78451519d7aaa64a7ec5b1584fcbca09c6b430c87667f9"}
Jul 09 09:42:20.905816 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.905779    2577 generic.go:356] "Generic (PLEG): container finished" podID="d074b26a-d370-449f-af4d-081606c8436f" containerID="0ce3205670032913e0075c6d6e44dfcae881ae092799215f8eeab8de86bca157" exitCode=0
Jul 09 09:42:20.905888 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.905835    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw" event={"ID":"d074b26a-d370-449f-af4d-081606c8436f","Type":"ContainerDied","Data":"0ce3205670032913e0075c6d6e44dfcae881ae092799215f8eeab8de86bca157"}
Jul 09 09:42:20.905888 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:20.905881    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw" event={"ID":"d074b26a-d370-449f-af4d-081606c8436f","Type":"ContainerStarted","Data":"c525662977da8cf691f880301c58ce1a7b74866ae3b3c2ef8ec79307fd29435d"}
Jul 09 09:42:21.917711 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:21.917683    2577 generic.go:356] "Generic (PLEG): container finished" podID="d074b26a-d370-449f-af4d-081606c8436f" containerID="6c226eef7e982cd83cff8bf84699d774c6d15260e93aeb019e2b3ff3b37a332e" exitCode=0
Jul 09 09:42:21.918097 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:21.917768    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw" event={"ID":"d074b26a-d370-449f-af4d-081606c8436f","Type":"ContainerDied","Data":"6c226eef7e982cd83cff8bf84699d774c6d15260e93aeb019e2b3ff3b37a332e"}
Jul 09 09:42:21.920290 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:21.920206    2577 generic.go:356] "Generic (PLEG): container finished" podID="a17e7fca-e855-4959-86c9-d0f3f6298898" containerID="99735ac7e594e3f0c72c3d59585de276d1bcf630bf90e16e6e0d1107f42ef881" exitCode=0
Jul 09 09:42:21.920290 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:21.920251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f" event={"ID":"a17e7fca-e855-4959-86c9-d0f3f6298898","Type":"ContainerDied","Data":"99735ac7e594e3f0c72c3d59585de276d1bcf630bf90e16e6e0d1107f42ef881"}
Jul 09 09:42:22.927521 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:22.927489    2577 generic.go:356] "Generic (PLEG): container finished" podID="a17e7fca-e855-4959-86c9-d0f3f6298898" containerID="b8fd67dd823eeabcb0a683bd4ceccd7535fe34c416ea7ccb856df22007650bd8" exitCode=0
Jul 09 09:42:22.927941 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:22.927579    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f" event={"ID":"a17e7fca-e855-4959-86c9-d0f3f6298898","Type":"ContainerDied","Data":"b8fd67dd823eeabcb0a683bd4ceccd7535fe34c416ea7ccb856df22007650bd8"}
Jul 09 09:42:22.929448 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:22.929421    2577 generic.go:356] "Generic (PLEG): container finished" podID="f1367cea-5a9c-403f-894b-55db9369988b" containerID="d5e6bdb63a7855c286dd7c75a469709e7a394c8efa8b31e1c2d14f9dc8345428" exitCode=0
Jul 09 09:42:22.929576 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:22.929502    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj" event={"ID":"f1367cea-5a9c-403f-894b-55db9369988b","Type":"ContainerDied","Data":"d5e6bdb63a7855c286dd7c75a469709e7a394c8efa8b31e1c2d14f9dc8345428"}
Jul 09 09:42:22.931729 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:22.931707    2577 generic.go:356] "Generic (PLEG): container finished" podID="d074b26a-d370-449f-af4d-081606c8436f" containerID="57182f0d66d37da6039d4356f2cb2ef772ff227755f9fb3a1ca50bf6ff49041f" exitCode=0
Jul 09 09:42:22.931865 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:22.931782    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw" event={"ID":"d074b26a-d370-449f-af4d-081606c8436f","Type":"ContainerDied","Data":"57182f0d66d37da6039d4356f2cb2ef772ff227755f9fb3a1ca50bf6ff49041f"}
Jul 09 09:42:22.933415 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:22.933396    2577 generic.go:356] "Generic (PLEG): container finished" podID="bec0891a-18f7-4a03-a160-98ac2efc4fb1" containerID="bd7bfd9a477b674af6951ce380d9a2bb46e849ab1b5c418deca4e43de8afc621" exitCode=0
Jul 09 09:42:22.933501 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:22.933452    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c" event={"ID":"bec0891a-18f7-4a03-a160-98ac2efc4fb1","Type":"ContainerDied","Data":"bd7bfd9a477b674af6951ce380d9a2bb46e849ab1b5c418deca4e43de8afc621"}
Jul 09 09:42:23.940759 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:23.940725    2577 generic.go:356] "Generic (PLEG): container finished" podID="bec0891a-18f7-4a03-a160-98ac2efc4fb1" containerID="6b17feb20166c1fe2b53cfae01af51b7582716d47408ab87b7ab0deaafd375bf" exitCode=0
Jul 09 09:42:23.941173 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:23.940819    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c" event={"ID":"bec0891a-18f7-4a03-a160-98ac2efc4fb1","Type":"ContainerDied","Data":"6b17feb20166c1fe2b53cfae01af51b7582716d47408ab87b7ab0deaafd375bf"}
Jul 09 09:42:23.942756 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:23.942736    2577 generic.go:356] "Generic (PLEG): container finished" podID="f1367cea-5a9c-403f-894b-55db9369988b" containerID="ef6c2f2c28b9f9bb0b409c513a408545c665495db2cb556ad726b3f9d5f0dd83" exitCode=0
Jul 09 09:42:23.942868 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:23.942760    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj" event={"ID":"f1367cea-5a9c-403f-894b-55db9369988b","Type":"ContainerDied","Data":"ef6c2f2c28b9f9bb0b409c513a408545c665495db2cb556ad726b3f9d5f0dd83"}
Jul 09 09:42:24.082843 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.082823    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"
Jul 09 09:42:24.112287 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.112264    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"
Jul 09 09:42:24.160082 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.160057    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbzrm\" (UniqueName: \"kubernetes.io/projected/a17e7fca-e855-4959-86c9-d0f3f6298898-kube-api-access-kbzrm\") pod \"a17e7fca-e855-4959-86c9-d0f3f6298898\" (UID: \"a17e7fca-e855-4959-86c9-d0f3f6298898\") "
Jul 09 09:42:24.160207 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.160139    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a17e7fca-e855-4959-86c9-d0f3f6298898-util\") pod \"a17e7fca-e855-4959-86c9-d0f3f6298898\" (UID: \"a17e7fca-e855-4959-86c9-d0f3f6298898\") "
Jul 09 09:42:24.160244 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.160207    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a17e7fca-e855-4959-86c9-d0f3f6298898-bundle\") pod \"a17e7fca-e855-4959-86c9-d0f3f6298898\" (UID: \"a17e7fca-e855-4959-86c9-d0f3f6298898\") "
Jul 09 09:42:24.160870 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.160844    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a17e7fca-e855-4959-86c9-d0f3f6298898-bundle" (OuterVolumeSpecName: "bundle") pod "a17e7fca-e855-4959-86c9-d0f3f6298898" (UID: "a17e7fca-e855-4959-86c9-d0f3f6298898"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:42:24.162121 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.162095    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a17e7fca-e855-4959-86c9-d0f3f6298898-kube-api-access-kbzrm" (OuterVolumeSpecName: "kube-api-access-kbzrm") pod "a17e7fca-e855-4959-86c9-d0f3f6298898" (UID: "a17e7fca-e855-4959-86c9-d0f3f6298898"). InnerVolumeSpecName "kube-api-access-kbzrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:42:24.168014 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.167990    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a17e7fca-e855-4959-86c9-d0f3f6298898-util" (OuterVolumeSpecName: "util") pod "a17e7fca-e855-4959-86c9-d0f3f6298898" (UID: "a17e7fca-e855-4959-86c9-d0f3f6298898"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:42:24.260987 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.260965    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d074b26a-d370-449f-af4d-081606c8436f-bundle\") pod \"d074b26a-d370-449f-af4d-081606c8436f\" (UID: \"d074b26a-d370-449f-af4d-081606c8436f\") "
Jul 09 09:42:24.261084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.261006    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d074b26a-d370-449f-af4d-081606c8436f-util\") pod \"d074b26a-d370-449f-af4d-081606c8436f\" (UID: \"d074b26a-d370-449f-af4d-081606c8436f\") "
Jul 09 09:42:24.261084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.261032    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fdgv\" (UniqueName: \"kubernetes.io/projected/d074b26a-d370-449f-af4d-081606c8436f-kube-api-access-4fdgv\") pod \"d074b26a-d370-449f-af4d-081606c8436f\" (UID: \"d074b26a-d370-449f-af4d-081606c8436f\") "
Jul 09 09:42:24.261217 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.261156    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbzrm\" (UniqueName: \"kubernetes.io/projected/a17e7fca-e855-4959-86c9-d0f3f6298898-kube-api-access-kbzrm\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:24.261217 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.261166    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a17e7fca-e855-4959-86c9-d0f3f6298898-util\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:24.261217 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.261175    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a17e7fca-e855-4959-86c9-d0f3f6298898-bundle\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:24.261410 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.261386    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d074b26a-d370-449f-af4d-081606c8436f-bundle" (OuterVolumeSpecName: "bundle") pod "d074b26a-d370-449f-af4d-081606c8436f" (UID: "d074b26a-d370-449f-af4d-081606c8436f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:42:24.262844 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.262823    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d074b26a-d370-449f-af4d-081606c8436f-kube-api-access-4fdgv" (OuterVolumeSpecName: "kube-api-access-4fdgv") pod "d074b26a-d370-449f-af4d-081606c8436f" (UID: "d074b26a-d370-449f-af4d-081606c8436f"). InnerVolumeSpecName "kube-api-access-4fdgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:42:24.266479 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.266444    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d074b26a-d370-449f-af4d-081606c8436f-util" (OuterVolumeSpecName: "util") pod "d074b26a-d370-449f-af4d-081606c8436f" (UID: "d074b26a-d370-449f-af4d-081606c8436f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:42:24.361540 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.361521    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d074b26a-d370-449f-af4d-081606c8436f-bundle\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:24.361540 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.361540    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d074b26a-d370-449f-af4d-081606c8436f-util\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:24.361662 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.361554    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fdgv\" (UniqueName: \"kubernetes.io/projected/d074b26a-d370-449f-af4d-081606c8436f-kube-api-access-4fdgv\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:24.949701 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.949677    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw"
Jul 09 09:42:24.950106 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.949714    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efptczw" event={"ID":"d074b26a-d370-449f-af4d-081606c8436f","Type":"ContainerDied","Data":"c525662977da8cf691f880301c58ce1a7b74866ae3b3c2ef8ec79307fd29435d"}
Jul 09 09:42:24.950106 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.949761    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c525662977da8cf691f880301c58ce1a7b74866ae3b3c2ef8ec79307fd29435d"
Jul 09 09:42:24.951586 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.951563    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f"
Jul 09 09:42:24.951671 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.951581    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76tt64f" event={"ID":"a17e7fca-e855-4959-86c9-d0f3f6298898","Type":"ContainerDied","Data":"4cafe35f268bc552bdd643933dc0402542125fc6be84f576380dcf956771f46c"}
Jul 09 09:42:24.951671 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:24.951621    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cafe35f268bc552bdd643933dc0402542125fc6be84f576380dcf956771f46c"
Jul 09 09:42:25.114123 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.114094    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"
Jul 09 09:42:25.116087 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.116069    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"
Jul 09 09:42:25.267955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.267928    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1367cea-5a9c-403f-894b-55db9369988b-bundle\") pod \"f1367cea-5a9c-403f-894b-55db9369988b\" (UID: \"f1367cea-5a9c-403f-894b-55db9369988b\") "
Jul 09 09:42:25.268087 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.267960    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsgfx\" (UniqueName: \"kubernetes.io/projected/bec0891a-18f7-4a03-a160-98ac2efc4fb1-kube-api-access-wsgfx\") pod \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\" (UID: \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\") "
Jul 09 09:42:25.268087 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.267978    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bec0891a-18f7-4a03-a160-98ac2efc4fb1-bundle\") pod \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\" (UID: \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\") "
Jul 09 09:42:25.268087 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.268018    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zj9d4\" (UniqueName: \"kubernetes.io/projected/f1367cea-5a9c-403f-894b-55db9369988b-kube-api-access-zj9d4\") pod \"f1367cea-5a9c-403f-894b-55db9369988b\" (UID: \"f1367cea-5a9c-403f-894b-55db9369988b\") "
Jul 09 09:42:25.268258 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.268126    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1367cea-5a9c-403f-894b-55db9369988b-util\") pod \"f1367cea-5a9c-403f-894b-55db9369988b\" (UID: \"f1367cea-5a9c-403f-894b-55db9369988b\") "
Jul 09 09:42:25.268258 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.268226    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bec0891a-18f7-4a03-a160-98ac2efc4fb1-util\") pod \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\" (UID: \"bec0891a-18f7-4a03-a160-98ac2efc4fb1\") "
Jul 09 09:42:25.268660 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.268629    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1367cea-5a9c-403f-894b-55db9369988b-bundle" (OuterVolumeSpecName: "bundle") pod "f1367cea-5a9c-403f-894b-55db9369988b" (UID: "f1367cea-5a9c-403f-894b-55db9369988b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:42:25.268753 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.268702    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bec0891a-18f7-4a03-a160-98ac2efc4fb1-bundle" (OuterVolumeSpecName: "bundle") pod "bec0891a-18f7-4a03-a160-98ac2efc4fb1" (UID: "bec0891a-18f7-4a03-a160-98ac2efc4fb1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:42:25.270105 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.270083    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bec0891a-18f7-4a03-a160-98ac2efc4fb1-kube-api-access-wsgfx" (OuterVolumeSpecName: "kube-api-access-wsgfx") pod "bec0891a-18f7-4a03-a160-98ac2efc4fb1" (UID: "bec0891a-18f7-4a03-a160-98ac2efc4fb1"). InnerVolumeSpecName "kube-api-access-wsgfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:42:25.270404 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.270383    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1367cea-5a9c-403f-894b-55db9369988b-kube-api-access-zj9d4" (OuterVolumeSpecName: "kube-api-access-zj9d4") pod "f1367cea-5a9c-403f-894b-55db9369988b" (UID: "f1367cea-5a9c-403f-894b-55db9369988b"). InnerVolumeSpecName "kube-api-access-zj9d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:42:25.273818 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.273747    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bec0891a-18f7-4a03-a160-98ac2efc4fb1-util" (OuterVolumeSpecName: "util") pod "bec0891a-18f7-4a03-a160-98ac2efc4fb1" (UID: "bec0891a-18f7-4a03-a160-98ac2efc4fb1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:42:25.274017 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.273993    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1367cea-5a9c-403f-894b-55db9369988b-util" (OuterVolumeSpecName: "util") pod "f1367cea-5a9c-403f-894b-55db9369988b" (UID: "f1367cea-5a9c-403f-894b-55db9369988b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:42:25.369696 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.369675    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1367cea-5a9c-403f-894b-55db9369988b-bundle\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:25.369772 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.369698    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsgfx\" (UniqueName: \"kubernetes.io/projected/bec0891a-18f7-4a03-a160-98ac2efc4fb1-kube-api-access-wsgfx\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:25.369772 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.369709    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bec0891a-18f7-4a03-a160-98ac2efc4fb1-bundle\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:25.369772 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.369717    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zj9d4\" (UniqueName: \"kubernetes.io/projected/f1367cea-5a9c-403f-894b-55db9369988b-kube-api-access-zj9d4\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:25.369772 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.369725    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1367cea-5a9c-403f-894b-55db9369988b-util\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:25.369772 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.369734    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bec0891a-18f7-4a03-a160-98ac2efc4fb1-util\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:42:25.958101 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.958029    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj"
Jul 09 09:42:25.958101 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.958039    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff366c4gj" event={"ID":"f1367cea-5a9c-403f-894b-55db9369988b","Type":"ContainerDied","Data":"1328709a72da66184f78451519d7aaa64a7ec5b1584fcbca09c6b430c87667f9"}
Jul 09 09:42:25.958101 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.958091    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1328709a72da66184f78451519d7aaa64a7ec5b1584fcbca09c6b430c87667f9"
Jul 09 09:42:25.960194 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.960170    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c"
Jul 09 09:42:25.960295 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.960169    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab97v7c" event={"ID":"bec0891a-18f7-4a03-a160-98ac2efc4fb1","Type":"ContainerDied","Data":"aa07bb388ed1e87121d6bf625887ad6e71d5bdf5e0e7d01ce34e4ca4f83ebf79"}
Jul 09 09:42:25.960295 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:25.960283    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa07bb388ed1e87121d6bf625887ad6e71d5bdf5e0e7d01ce34e4ca4f83ebf79"
Jul 09 09:42:29.618069 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:29.618035    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:31.667549 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667520    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-qm9ts"]
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667850    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d074b26a-d370-449f-af4d-081606c8436f" containerName="extract"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667862    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d074b26a-d370-449f-af4d-081606c8436f" containerName="extract"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667874    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bec0891a-18f7-4a03-a160-98ac2efc4fb1" containerName="util"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667880    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bec0891a-18f7-4a03-a160-98ac2efc4fb1" containerName="util"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667889    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1367cea-5a9c-403f-894b-55db9369988b" containerName="pull"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667893    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1367cea-5a9c-403f-894b-55db9369988b" containerName="pull"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667900    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1367cea-5a9c-403f-894b-55db9369988b" containerName="extract"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667905    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1367cea-5a9c-403f-894b-55db9369988b" containerName="extract"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667914    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d074b26a-d370-449f-af4d-081606c8436f" containerName="pull"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667920    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d074b26a-d370-449f-af4d-081606c8436f" containerName="pull"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667925    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bec0891a-18f7-4a03-a160-98ac2efc4fb1" containerName="extract"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667930    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bec0891a-18f7-4a03-a160-98ac2efc4fb1" containerName="extract"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667936    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a17e7fca-e855-4959-86c9-d0f3f6298898" containerName="pull"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667941    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a17e7fca-e855-4959-86c9-d0f3f6298898" containerName="pull"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667948    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1367cea-5a9c-403f-894b-55db9369988b" containerName="util"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667954    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1367cea-5a9c-403f-894b-55db9369988b" containerName="util"
Jul 09 09:42:31.667955 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667963    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a17e7fca-e855-4959-86c9-d0f3f6298898" containerName="extract"
Jul 09 09:42:31.668526 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667968    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a17e7fca-e855-4959-86c9-d0f3f6298898" containerName="extract"
Jul 09 09:42:31.668526 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667978    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a17e7fca-e855-4959-86c9-d0f3f6298898" containerName="util"
Jul 09 09:42:31.668526 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667983    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a17e7fca-e855-4959-86c9-d0f3f6298898" containerName="util"
Jul 09 09:42:31.668526 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667990    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d074b26a-d370-449f-af4d-081606c8436f" containerName="util"
Jul 09 09:42:31.668526 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.667995    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d074b26a-d370-449f-af4d-081606c8436f" containerName="util"
Jul 09 09:42:31.668526 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.668000    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bec0891a-18f7-4a03-a160-98ac2efc4fb1" containerName="pull"
Jul 09 09:42:31.668526 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.668005    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bec0891a-18f7-4a03-a160-98ac2efc4fb1" containerName="pull"
Jul 09 09:42:31.668526 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.668057    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1367cea-5a9c-403f-894b-55db9369988b" containerName="extract"
Jul 09 09:42:31.668526 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.668065    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bec0891a-18f7-4a03-a160-98ac2efc4fb1" containerName="extract"
Jul 09 09:42:31.668526 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.668071    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d074b26a-d370-449f-af4d-081606c8436f" containerName="extract"
Jul 09 09:42:31.668526 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.668077    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a17e7fca-e855-4959-86c9-d0f3f6298898" containerName="extract"
Jul 09 09:42:31.673419 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.673391    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-qm9ts"
Jul 09 09:42:31.676020 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.675996    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 09:42:31.676110 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.676074    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 09 09:42:31.676110 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.676087    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 09:42:31.676110 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.676095    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-lg9q8\""
Jul 09 09:42:31.684641 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.684622    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-qm9ts"]
Jul 09 09:42:31.820421 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.820398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57776\" (UniqueName: \"kubernetes.io/projected/8be0106f-0797-40a0-ab86-6d47c804051a-kube-api-access-57776\") pod \"dns-operator-controller-manager-65b49595d7-qm9ts\" (UID: \"8be0106f-0797-40a0-ab86-6d47c804051a\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-qm9ts"
Jul 09 09:42:31.920906 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.920845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57776\" (UniqueName: \"kubernetes.io/projected/8be0106f-0797-40a0-ab86-6d47c804051a-kube-api-access-57776\") pod \"dns-operator-controller-manager-65b49595d7-qm9ts\" (UID: \"8be0106f-0797-40a0-ab86-6d47c804051a\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-qm9ts"
Jul 09 09:42:31.933089 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.933058    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57776\" (UniqueName: \"kubernetes.io/projected/8be0106f-0797-40a0-ab86-6d47c804051a-kube-api-access-57776\") pod \"dns-operator-controller-manager-65b49595d7-qm9ts\" (UID: \"8be0106f-0797-40a0-ab86-6d47c804051a\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-qm9ts"
Jul 09 09:42:31.984865 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:31.984845    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-qm9ts"
Jul 09 09:42:32.137108 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:32.137081    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-qm9ts"]
Jul 09 09:42:32.138524 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:42:32.138496    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8be0106f_0797_40a0_ab86_6d47c804051a.slice/crio-045543bba92aab98554e690c11a85c3563d8729c53ede7bcd4af25db713cef7e WatchSource:0}: Error finding container 045543bba92aab98554e690c11a85c3563d8729c53ede7bcd4af25db713cef7e: Status 404 returned error can't find the container with id 045543bba92aab98554e690c11a85c3563d8729c53ede7bcd4af25db713cef7e
Jul 09 09:42:32.219476 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:32.219441    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:32.622448 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:32.622364    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:32.999104 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:32.999073    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-qm9ts" event={"ID":"8be0106f-0797-40a0-ab86-6d47c804051a","Type":"ContainerStarted","Data":"045543bba92aab98554e690c11a85c3563d8729c53ede7bcd4af25db713cef7e"}
Jul 09 09:42:33.033211 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:33.033174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:34.281539 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:34.281505    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:36.019489 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.019449    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-qm9ts" event={"ID":"8be0106f-0797-40a0-ab86-6d47c804051a","Type":"ContainerStarted","Data":"911b8690a1137280762f90c95aaa25159fc3c06abdb5913a10c4b1a1c8139dc7"}
Jul 09 09:42:36.019884 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.019592    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-qm9ts"
Jul 09 09:42:36.051205 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.051160    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-qm9ts" podStartSLOduration=1.730027126 podStartE2EDuration="5.051145778s" totalImagesPullingTime="3.321118652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:42:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:42:36.049768028 +0000 UTC m=+559.055937100" watchObservedRunningTime="2026-07-09 09:42:36.051145778 +0000 UTC m=+559.057314849"
Jul 09 09:42:36.116355 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.116329    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:36.145861 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.145836    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm"]
Jul 09 09:42:36.149254 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.149238    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm"
Jul 09 09:42:36.153545 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.153526    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vg98s\""
Jul 09 09:42:36.168937 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.168910    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm"]
Jul 09 09:42:36.256280 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.256256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m845\" (UniqueName: \"kubernetes.io/projected/89197f43-f076-44a8-8583-9a86b622891d-kube-api-access-7m845\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-9btlm\" (UID: \"89197f43-f076-44a8-8583-9a86b622891d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm"
Jul 09 09:42:36.256386 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.256302    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89197f43-f076-44a8-8583-9a86b622891d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-9btlm\" (UID: \"89197f43-f076-44a8-8583-9a86b622891d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm"
Jul 09 09:42:36.357509 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.357436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m845\" (UniqueName: \"kubernetes.io/projected/89197f43-f076-44a8-8583-9a86b622891d-kube-api-access-7m845\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-9btlm\" (UID: \"89197f43-f076-44a8-8583-9a86b622891d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm"
Jul 09 09:42:36.357509 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.357481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89197f43-f076-44a8-8583-9a86b622891d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-9btlm\" (UID: \"89197f43-f076-44a8-8583-9a86b622891d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm"
Jul 09 09:42:36.357789 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.357773    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89197f43-f076-44a8-8583-9a86b622891d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-9btlm\" (UID: \"89197f43-f076-44a8-8583-9a86b622891d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm"
Jul 09 09:42:36.366136 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.366119    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m845\" (UniqueName: \"kubernetes.io/projected/89197f43-f076-44a8-8583-9a86b622891d-kube-api-access-7m845\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-9btlm\" (UID: \"89197f43-f076-44a8-8583-9a86b622891d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm"
Jul 09 09:42:36.459313 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.459290    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm"
Jul 09 09:42:36.607744 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.607725    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm"]
Jul 09 09:42:36.610127 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:42:36.610099    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89197f43_f076_44a8_8583_9a86b622891d.slice/crio-80b8c7a91ae3e9fba74df082a2aa7e8656e42d334dbee5ffb626937da80175eb WatchSource:0}: Error finding container 80b8c7a91ae3e9fba74df082a2aa7e8656e42d334dbee5ffb626937da80175eb: Status 404 returned error can't find the container with id 80b8c7a91ae3e9fba74df082a2aa7e8656e42d334dbee5ffb626937da80175eb
Jul 09 09:42:36.625002 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:36.624973    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:37.025555 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:37.025518    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm" event={"ID":"89197f43-f076-44a8-8583-9a86b622891d","Type":"ContainerStarted","Data":"80b8c7a91ae3e9fba74df082a2aa7e8656e42d334dbee5ffb626937da80175eb"}
Jul 09 09:42:38.118032 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:38.118001    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6f9f468797-g7cbp"]
Jul 09 09:42:38.128095 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:38.128064    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-g7cbp"
Jul 09 09:42:38.133055 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:38.133031    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kpr84\""
Jul 09 09:42:38.141176 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:38.141152    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6f9f468797-g7cbp"]
Jul 09 09:42:38.274856 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:38.274822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrngh\" (UniqueName: \"kubernetes.io/projected/31924cb6-d969-46ed-a196-dce7f02be8ed-kube-api-access-wrngh\") pod \"limitador-operator-controller-manager-6f9f468797-g7cbp\" (UID: \"31924cb6-d969-46ed-a196-dce7f02be8ed\") " pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-g7cbp"
Jul 09 09:42:38.376128 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:38.376052    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrngh\" (UniqueName: \"kubernetes.io/projected/31924cb6-d969-46ed-a196-dce7f02be8ed-kube-api-access-wrngh\") pod \"limitador-operator-controller-manager-6f9f468797-g7cbp\" (UID: \"31924cb6-d969-46ed-a196-dce7f02be8ed\") " pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-g7cbp"
Jul 09 09:42:38.387220 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:38.387174    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrngh\" (UniqueName: \"kubernetes.io/projected/31924cb6-d969-46ed-a196-dce7f02be8ed-kube-api-access-wrngh\") pod \"limitador-operator-controller-manager-6f9f468797-g7cbp\" (UID: \"31924cb6-d969-46ed-a196-dce7f02be8ed\") " pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-g7cbp"
Jul 09 09:42:38.443956 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:38.443928    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-g7cbp"
Jul 09 09:42:38.951918 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:38.951872    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:39.018743 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:39.018708    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6f9f468797-g7cbp"]
Jul 09 09:42:39.020200 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:42:39.020174    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31924cb6_d969_46ed_a196_dce7f02be8ed.slice/crio-bab7cf21e09812a162bb6a3c54309dd417e6fa21e77ce2fc51a285c74d3988c4 WatchSource:0}: Error finding container bab7cf21e09812a162bb6a3c54309dd417e6fa21e77ce2fc51a285c74d3988c4: Status 404 returned error can't find the container with id bab7cf21e09812a162bb6a3c54309dd417e6fa21e77ce2fc51a285c74d3988c4
Jul 09 09:42:39.041022 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:39.040993    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-g7cbp" event={"ID":"31924cb6-d969-46ed-a196-dce7f02be8ed","Type":"ContainerStarted","Data":"bab7cf21e09812a162bb6a3c54309dd417e6fa21e77ce2fc51a285c74d3988c4"}
Jul 09 09:42:39.172973 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:39.172897    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:41.054983 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:41.054942    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm" event={"ID":"89197f43-f076-44a8-8583-9a86b622891d","Type":"ContainerStarted","Data":"a87c86f9e9123409b2ff2e6be2c4952ceae624a2e3ca057bcb4c6004ade016f7"}
Jul 09 09:42:41.055456 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:41.055040    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm"
Jul 09 09:42:41.090657 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:41.090601    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm" podStartSLOduration=1.111598153 podStartE2EDuration="5.090579275s" totalImagesPullingTime="3.978981122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:42:41.088104959 +0000 UTC m=+564.094274064" watchObservedRunningTime="2026-07-09 09:42:41.090579275 +0000 UTC m=+564.096748349"
Jul 09 09:42:41.091427 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:41.091393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:42.061831 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:42.061778    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-g7cbp" event={"ID":"31924cb6-d969-46ed-a196-dce7f02be8ed","Type":"ContainerStarted","Data":"908d4f8f610b38e3a6ce83a2ab4f5fba4f45427c2d7f74f4e398292df0f41110"}
Jul 09 09:42:42.062185 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:42.062031    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-g7cbp"
Jul 09 09:42:42.081616 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:42.081564    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-g7cbp" podStartSLOduration=1.7050053090000001 podStartE2EDuration="4.081545096s" totalImagesPullingTime="2.376539787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:42:42.081277313 +0000 UTC m=+565.087446420" watchObservedRunningTime="2026-07-09 09:42:42.081545096 +0000 UTC m=+565.087714169"
Jul 09 09:42:42.217565 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:42.217530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:42.316540 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:42.316469    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:43.084178 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:43.084133    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:43.447825 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:43.447714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:47.028065 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:47.028039    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-qm9ts"
Jul 09 09:42:47.032536 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:47.032511    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:47.175702 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:47.175675    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:47.445017 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:47.444945    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:47.568834 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:47.568790    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:47.660833 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:47.660790    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:47.760564 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:47.760537    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:52.064412 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:52.064379    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-9btlm"
Jul 09 09:42:53.070242 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:53.070208    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-g7cbp"
Jul 09 09:42:53.952117 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:53.952077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:54.170876 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:54.170838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:58.649174 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:58.649135    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:42:58.749258 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:42:58.749214    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:01.139511 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:01.139476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:01.246986 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:01.246948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:05.736881 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:05.736850    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:05.839003 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:05.838975    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:05.939134 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:05.939105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:12.783593 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:12.783553    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:12.889949 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:12.889922    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:17.341002 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:17.340974    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:43:17.343253 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:17.343226    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:43:17.580811 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:17.580760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:17.681712 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:17.681644    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:20.188539 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:20.188480    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:20.293978 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:20.293946    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:24.611626 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:24.611593    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:24.724857 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:24.724823    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:24.791191 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:24.791159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:25.429693 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:25.429662    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b6c89bb85-qpqmb"]
Jul 09 09:43:25.434039 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:25.434018    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c89bb85-qpqmb"
Jul 09 09:43:25.437046 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:25.437023    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-sj47w\""
Jul 09 09:43:25.441844 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:25.441824    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b6c89bb85-qpqmb"]
Jul 09 09:43:25.491700 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:25.491676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:25.517632 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:25.517608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7lqw\" (UniqueName: \"kubernetes.io/projected/1ff2d2e3-e866-42bf-b454-6860abb75955-kube-api-access-z7lqw\") pod \"authorino-b6c89bb85-qpqmb\" (UID: \"1ff2d2e3-e866-42bf-b454-6860abb75955\") " pod="kuadrant-system/authorino-b6c89bb85-qpqmb"
Jul 09 09:43:25.616930 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:25.616896    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:25.618961 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:25.618931    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7lqw\" (UniqueName: \"kubernetes.io/projected/1ff2d2e3-e866-42bf-b454-6860abb75955-kube-api-access-z7lqw\") pod \"authorino-b6c89bb85-qpqmb\" (UID: \"1ff2d2e3-e866-42bf-b454-6860abb75955\") " pod="kuadrant-system/authorino-b6c89bb85-qpqmb"
Jul 09 09:43:25.633315 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:25.633291    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7lqw\" (UniqueName: \"kubernetes.io/projected/1ff2d2e3-e866-42bf-b454-6860abb75955-kube-api-access-z7lqw\") pod \"authorino-b6c89bb85-qpqmb\" (UID: \"1ff2d2e3-e866-42bf-b454-6860abb75955\") " pod="kuadrant-system/authorino-b6c89bb85-qpqmb"
Jul 09 09:43:25.710003 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:25.709971    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:25.751158 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:25.751126    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c89bb85-qpqmb"
Jul 09 09:43:25.880184 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:25.880151    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b6c89bb85-qpqmb"]
Jul 09 09:43:25.883010 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:43:25.882983    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ff2d2e3_e866_42bf_b454_6860abb75955.slice/crio-d4a31ff2a844dc9c24abf638c263a03eeddf4d1d056bc35c904609de8fb621c8 WatchSource:0}: Error finding container d4a31ff2a844dc9c24abf638c263a03eeddf4d1d056bc35c904609de8fb621c8: Status 404 returned error can't find the container with id d4a31ff2a844dc9c24abf638c263a03eeddf4d1d056bc35c904609de8fb621c8
Jul 09 09:43:26.005114 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:26.005032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:26.085920 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:26.085890    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:26.295655 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:26.295575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c89bb85-qpqmb" event={"ID":"1ff2d2e3-e866-42bf-b454-6860abb75955","Type":"ContainerStarted","Data":"d4a31ff2a844dc9c24abf638c263a03eeddf4d1d056bc35c904609de8fb621c8"}
Jul 09 09:43:26.389931 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:26.389898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:26.459095 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:26.459067    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:29.317617 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:29.317579    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c89bb85-qpqmb" event={"ID":"1ff2d2e3-e866-42bf-b454-6860abb75955","Type":"ContainerStarted","Data":"643f51e3870a282f4dcf9c5cee0eccf20863a913569632d6e3fd8080d2dcb112"}
Jul 09 09:43:29.336262 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:29.336208    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b6c89bb85-qpqmb" podStartSLOduration=1.619562219 podStartE2EDuration="4.336192593s" totalImagesPullingTime="2.716630374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:43:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:43:29.335879067 +0000 UTC m=+612.342048141" watchObservedRunningTime="2026-07-09 09:43:29.336192593 +0000 UTC m=+612.342361663"
Jul 09 09:43:30.426327 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:30.426300    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b6c89bb85-qpqmb"]
Jul 09 09:43:31.328371 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:31.328312    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-b6c89bb85-qpqmb" podUID="1ff2d2e3-e866-42bf-b454-6860abb75955" containerName="authorino" containerID="cri-o://643f51e3870a282f4dcf9c5cee0eccf20863a913569632d6e3fd8080d2dcb112" gracePeriod=30
Jul 09 09:43:31.571089 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:31.571065    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c89bb85-qpqmb"
Jul 09 09:43:31.671214 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:31.671143    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7lqw\" (UniqueName: \"kubernetes.io/projected/1ff2d2e3-e866-42bf-b454-6860abb75955-kube-api-access-z7lqw\") pod \"1ff2d2e3-e866-42bf-b454-6860abb75955\" (UID: \"1ff2d2e3-e866-42bf-b454-6860abb75955\") "
Jul 09 09:43:31.673155 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:31.673129    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ff2d2e3-e866-42bf-b454-6860abb75955-kube-api-access-z7lqw" (OuterVolumeSpecName: "kube-api-access-z7lqw") pod "1ff2d2e3-e866-42bf-b454-6860abb75955" (UID: "1ff2d2e3-e866-42bf-b454-6860abb75955"). InnerVolumeSpecName "kube-api-access-z7lqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:43:31.771783 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:31.771757    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7lqw\" (UniqueName: \"kubernetes.io/projected/1ff2d2e3-e866-42bf-b454-6860abb75955-kube-api-access-z7lqw\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:43:32.334661 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:32.334627    2577 generic.go:356] "Generic (PLEG): container finished" podID="1ff2d2e3-e866-42bf-b454-6860abb75955" containerID="643f51e3870a282f4dcf9c5cee0eccf20863a913569632d6e3fd8080d2dcb112" exitCode=0
Jul 09 09:43:32.334848 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:32.334679    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c89bb85-qpqmb"
Jul 09 09:43:32.334848 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:32.334714    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c89bb85-qpqmb" event={"ID":"1ff2d2e3-e866-42bf-b454-6860abb75955","Type":"ContainerDied","Data":"643f51e3870a282f4dcf9c5cee0eccf20863a913569632d6e3fd8080d2dcb112"}
Jul 09 09:43:32.334848 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:32.334755    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c89bb85-qpqmb" event={"ID":"1ff2d2e3-e866-42bf-b454-6860abb75955","Type":"ContainerDied","Data":"d4a31ff2a844dc9c24abf638c263a03eeddf4d1d056bc35c904609de8fb621c8"}
Jul 09 09:43:32.334848 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:32.334774    2577 scope.go:117] "RemoveContainer" containerID="643f51e3870a282f4dcf9c5cee0eccf20863a913569632d6e3fd8080d2dcb112"
Jul 09 09:43:32.343622 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:32.343608    2577 scope.go:117] "RemoveContainer" containerID="643f51e3870a282f4dcf9c5cee0eccf20863a913569632d6e3fd8080d2dcb112"
Jul 09 09:43:32.343889 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:43:32.343871    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"643f51e3870a282f4dcf9c5cee0eccf20863a913569632d6e3fd8080d2dcb112\": container with ID starting with 643f51e3870a282f4dcf9c5cee0eccf20863a913569632d6e3fd8080d2dcb112 not found: ID does not exist" containerID="643f51e3870a282f4dcf9c5cee0eccf20863a913569632d6e3fd8080d2dcb112"
Jul 09 09:43:32.343939 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:32.343896    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"643f51e3870a282f4dcf9c5cee0eccf20863a913569632d6e3fd8080d2dcb112"} err="failed to get container status \"643f51e3870a282f4dcf9c5cee0eccf20863a913569632d6e3fd8080d2dcb112\": rpc error: code = NotFound desc = could not find container \"643f51e3870a282f4dcf9c5cee0eccf20863a913569632d6e3fd8080d2dcb112\": container with ID starting with 643f51e3870a282f4dcf9c5cee0eccf20863a913569632d6e3fd8080d2dcb112 not found: ID does not exist"
Jul 09 09:43:32.354957 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:32.354937    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b6c89bb85-qpqmb"]
Jul 09 09:43:32.361304 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:32.361281    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b6c89bb85-qpqmb"]
Jul 09 09:43:33.551104 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:33.551071    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ff2d2e3-e866-42bf-b454-6860abb75955" path="/var/lib/kubelet/pods/1ff2d2e3-e866-42bf-b454-6860abb75955/volumes"
Jul 09 09:43:33.791004 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:33.790971    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:33.890826 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:33.890739    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:36.069596 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:36.069562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:36.170670 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:36.170644    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:38.611524 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:38.611485    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:38.709925 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:38.709873    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:38.825513 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:38.825482    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:38.924671 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:38.924583    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:39.278509 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:39.278460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:39.398573 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:39.398533    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:46.501989 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:46.501906    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:48.021200 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:48.021166    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:43:56.945063 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:56.944980    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-5876b4f96d-hvhnk"]
Jul 09 09:43:56.945554 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:56.945242    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk" podUID="f94cfcee-b683-43d0-90fd-a9437bfc32ee" containerName="manager" containerID="cri-o://36af742a187055561be63e044d14f7c7e5ab8378c2d1851292b12df2779439ea" gracePeriod=10
Jul 09 09:43:56.989627 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:56.989598    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-5876b4f96d-pgx27"]
Jul 09 09:43:56.989992 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:56.989978    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ff2d2e3-e866-42bf-b454-6860abb75955" containerName="authorino"
Jul 09 09:43:56.990044 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:56.989994    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ff2d2e3-e866-42bf-b454-6860abb75955" containerName="authorino"
Jul 09 09:43:56.990080 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:56.990062    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ff2d2e3-e866-42bf-b454-6860abb75955" containerName="authorino"
Jul 09 09:43:56.993049 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:56.993034    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5876b4f96d-pgx27"
Jul 09 09:43:57.017294 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.017274    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5876b4f96d-pgx27"]
Jul 09 09:43:57.159394 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.159364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvhxv\" (UniqueName: \"kubernetes.io/projected/6e35e008-95da-4230-a3f2-ed5ec4d25217-kube-api-access-hvhxv\") pod \"kserve-controller-manager-5876b4f96d-pgx27\" (UID: \"6e35e008-95da-4230-a3f2-ed5ec4d25217\") " pod="kserve/kserve-controller-manager-5876b4f96d-pgx27"
Jul 09 09:43:57.159540 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.159427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e35e008-95da-4230-a3f2-ed5ec4d25217-cert\") pod \"kserve-controller-manager-5876b4f96d-pgx27\" (UID: \"6e35e008-95da-4230-a3f2-ed5ec4d25217\") " pod="kserve/kserve-controller-manager-5876b4f96d-pgx27"
Jul 09 09:43:57.207294 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.207238    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk"
Jul 09 09:43:57.260264 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.260232    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e35e008-95da-4230-a3f2-ed5ec4d25217-cert\") pod \"kserve-controller-manager-5876b4f96d-pgx27\" (UID: \"6e35e008-95da-4230-a3f2-ed5ec4d25217\") " pod="kserve/kserve-controller-manager-5876b4f96d-pgx27"
Jul 09 09:43:57.260420 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.260302    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvhxv\" (UniqueName: \"kubernetes.io/projected/6e35e008-95da-4230-a3f2-ed5ec4d25217-kube-api-access-hvhxv\") pod \"kserve-controller-manager-5876b4f96d-pgx27\" (UID: \"6e35e008-95da-4230-a3f2-ed5ec4d25217\") " pod="kserve/kserve-controller-manager-5876b4f96d-pgx27"
Jul 09 09:43:57.262514 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.262487    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e35e008-95da-4230-a3f2-ed5ec4d25217-cert\") pod \"kserve-controller-manager-5876b4f96d-pgx27\" (UID: \"6e35e008-95da-4230-a3f2-ed5ec4d25217\") " pod="kserve/kserve-controller-manager-5876b4f96d-pgx27"
Jul 09 09:43:57.271046 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.271019    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvhxv\" (UniqueName: \"kubernetes.io/projected/6e35e008-95da-4230-a3f2-ed5ec4d25217-kube-api-access-hvhxv\") pod \"kserve-controller-manager-5876b4f96d-pgx27\" (UID: \"6e35e008-95da-4230-a3f2-ed5ec4d25217\") " pod="kserve/kserve-controller-manager-5876b4f96d-pgx27"
Jul 09 09:43:57.324067 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.324045    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5876b4f96d-pgx27"
Jul 09 09:43:57.361012 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.360989    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f94cfcee-b683-43d0-90fd-a9437bfc32ee-cert\") pod \"f94cfcee-b683-43d0-90fd-a9437bfc32ee\" (UID: \"f94cfcee-b683-43d0-90fd-a9437bfc32ee\") "
Jul 09 09:43:57.361147 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.361087    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fmwg\" (UniqueName: \"kubernetes.io/projected/f94cfcee-b683-43d0-90fd-a9437bfc32ee-kube-api-access-7fmwg\") pod \"f94cfcee-b683-43d0-90fd-a9437bfc32ee\" (UID: \"f94cfcee-b683-43d0-90fd-a9437bfc32ee\") "
Jul 09 09:43:57.363469 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.363436    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f94cfcee-b683-43d0-90fd-a9437bfc32ee-cert" (OuterVolumeSpecName: "cert") pod "f94cfcee-b683-43d0-90fd-a9437bfc32ee" (UID: "f94cfcee-b683-43d0-90fd-a9437bfc32ee"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:43:57.363555 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.363523    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f94cfcee-b683-43d0-90fd-a9437bfc32ee-kube-api-access-7fmwg" (OuterVolumeSpecName: "kube-api-access-7fmwg") pod "f94cfcee-b683-43d0-90fd-a9437bfc32ee" (UID: "f94cfcee-b683-43d0-90fd-a9437bfc32ee"). InnerVolumeSpecName "kube-api-access-7fmwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:43:57.462379 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.462356    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fmwg\" (UniqueName: \"kubernetes.io/projected/f94cfcee-b683-43d0-90fd-a9437bfc32ee-kube-api-access-7fmwg\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:43:57.462509 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.462380    2577 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f94cfcee-b683-43d0-90fd-a9437bfc32ee-cert\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:43:57.466103 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.466051    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5876b4f96d-pgx27"]
Jul 09 09:43:57.468284 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:43:57.468261    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e35e008_95da_4230_a3f2_ed5ec4d25217.slice/crio-7da2575b9cb5f46ab56de31850e63e30642f2f87829bdddb11e89232a951c527 WatchSource:0}: Error finding container 7da2575b9cb5f46ab56de31850e63e30642f2f87829bdddb11e89232a951c527: Status 404 returned error can't find the container with id 7da2575b9cb5f46ab56de31850e63e30642f2f87829bdddb11e89232a951c527
Jul 09 09:43:57.474077 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.474053    2577 generic.go:356] "Generic (PLEG): container finished" podID="f94cfcee-b683-43d0-90fd-a9437bfc32ee" containerID="36af742a187055561be63e044d14f7c7e5ab8378c2d1851292b12df2779439ea" exitCode=0
Jul 09 09:43:57.474163 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.474105    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk"
Jul 09 09:43:57.474163 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.474133    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk" event={"ID":"f94cfcee-b683-43d0-90fd-a9437bfc32ee","Type":"ContainerDied","Data":"36af742a187055561be63e044d14f7c7e5ab8378c2d1851292b12df2779439ea"}
Jul 09 09:43:57.474252 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.474182    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5876b4f96d-hvhnk" event={"ID":"f94cfcee-b683-43d0-90fd-a9437bfc32ee","Type":"ContainerDied","Data":"228e018c7d76ea497a93d798f199891aa2bac9e75dc792a149c1fcedfc5d6d8c"}
Jul 09 09:43:57.474252 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.474204    2577 scope.go:117] "RemoveContainer" containerID="36af742a187055561be63e044d14f7c7e5ab8378c2d1851292b12df2779439ea"
Jul 09 09:43:57.483325 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.483270    2577 scope.go:117] "RemoveContainer" containerID="36af742a187055561be63e044d14f7c7e5ab8378c2d1851292b12df2779439ea"
Jul 09 09:43:57.483572 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:43:57.483554    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36af742a187055561be63e044d14f7c7e5ab8378c2d1851292b12df2779439ea\": container with ID starting with 36af742a187055561be63e044d14f7c7e5ab8378c2d1851292b12df2779439ea not found: ID does not exist" containerID="36af742a187055561be63e044d14f7c7e5ab8378c2d1851292b12df2779439ea"
Jul 09 09:43:57.483634 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.483584    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36af742a187055561be63e044d14f7c7e5ab8378c2d1851292b12df2779439ea"} err="failed to get container status \"36af742a187055561be63e044d14f7c7e5ab8378c2d1851292b12df2779439ea\": rpc error: code = NotFound desc = could not find container \"36af742a187055561be63e044d14f7c7e5ab8378c2d1851292b12df2779439ea\": container with ID starting with 36af742a187055561be63e044d14f7c7e5ab8378c2d1851292b12df2779439ea not found: ID does not exist"
Jul 09 09:43:57.513374 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.513350    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-5876b4f96d-hvhnk"]
Jul 09 09:43:57.522737 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.522714    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-5876b4f96d-hvhnk"]
Jul 09 09:43:57.551474 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:57.551450    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f94cfcee-b683-43d0-90fd-a9437bfc32ee" path="/var/lib/kubelet/pods/f94cfcee-b683-43d0-90fd-a9437bfc32ee/volumes"
Jul 09 09:43:58.481877 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:58.481840    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5876b4f96d-pgx27" event={"ID":"6e35e008-95da-4230-a3f2-ed5ec4d25217","Type":"ContainerStarted","Data":"b2385a17d84b575b51e29d019b887a580944be321085597c007b84a1b6597086"}
Jul 09 09:43:58.482320 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:58.481900    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5876b4f96d-pgx27" event={"ID":"6e35e008-95da-4230-a3f2-ed5ec4d25217","Type":"ContainerStarted","Data":"7da2575b9cb5f46ab56de31850e63e30642f2f87829bdddb11e89232a951c527"}
Jul 09 09:43:58.482320 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:58.481932    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-5876b4f96d-pgx27"
Jul 09 09:43:58.520293 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:43:58.520251    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-5876b4f96d-pgx27" podStartSLOduration=2.065248378 podStartE2EDuration="2.520235047s" totalImagesPullingTime="454.986669ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:43:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:43:58.518333104 +0000 UTC m=+641.524502174" watchObservedRunningTime="2026-07-09 09:43:58.520235047 +0000 UTC m=+641.526404117"
Jul 09 09:44:15.492189 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:15.492157    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:44:29.492383 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:29.492353    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-5876b4f96d-pgx27"
Jul 09 09:44:30.457492 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.457456    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:44:30.527735 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.527707    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-4wtpk"]
Jul 09 09:44:30.528099 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.528080    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f94cfcee-b683-43d0-90fd-a9437bfc32ee" containerName="manager"
Jul 09 09:44:30.528099 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.528091    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f94cfcee-b683-43d0-90fd-a9437bfc32ee" containerName="manager"
Jul 09 09:44:30.528175 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.528143    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f94cfcee-b683-43d0-90fd-a9437bfc32ee" containerName="manager"
Jul 09 09:44:30.532615 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.532592    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-4wtpk"
Jul 09 09:44:30.535290 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.535267    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Jul 09 09:44:30.535426 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.535363    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-x5fr9\""
Jul 09 09:44:30.552268 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.552249    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-4wtpk"]
Jul 09 09:44:30.571272 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.571244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:44:30.664947 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.664912    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:44:30.702569 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.702542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dcd9f86-f183-4e63-94d8-65b48b7977f3-cert\") pod \"odh-model-controller-84b54b7f97-4wtpk\" (UID: \"5dcd9f86-f183-4e63-94d8-65b48b7977f3\") " pod="kserve/odh-model-controller-84b54b7f97-4wtpk"
Jul 09 09:44:30.702675 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.702600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d8bt\" (UniqueName: \"kubernetes.io/projected/5dcd9f86-f183-4e63-94d8-65b48b7977f3-kube-api-access-6d8bt\") pod \"odh-model-controller-84b54b7f97-4wtpk\" (UID: \"5dcd9f86-f183-4e63-94d8-65b48b7977f3\") " pod="kserve/odh-model-controller-84b54b7f97-4wtpk"
Jul 09 09:44:30.803364 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.803335    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dcd9f86-f183-4e63-94d8-65b48b7977f3-cert\") pod \"odh-model-controller-84b54b7f97-4wtpk\" (UID: \"5dcd9f86-f183-4e63-94d8-65b48b7977f3\") " pod="kserve/odh-model-controller-84b54b7f97-4wtpk"
Jul 09 09:44:30.803476 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.803427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d8bt\" (UniqueName: \"kubernetes.io/projected/5dcd9f86-f183-4e63-94d8-65b48b7977f3-kube-api-access-6d8bt\") pod \"odh-model-controller-84b54b7f97-4wtpk\" (UID: \"5dcd9f86-f183-4e63-94d8-65b48b7977f3\") " pod="kserve/odh-model-controller-84b54b7f97-4wtpk"
Jul 09 09:44:30.805698 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.805674    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dcd9f86-f183-4e63-94d8-65b48b7977f3-cert\") pod \"odh-model-controller-84b54b7f97-4wtpk\" (UID: \"5dcd9f86-f183-4e63-94d8-65b48b7977f3\") " pod="kserve/odh-model-controller-84b54b7f97-4wtpk"
Jul 09 09:44:30.815899 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.815873    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d8bt\" (UniqueName: \"kubernetes.io/projected/5dcd9f86-f183-4e63-94d8-65b48b7977f3-kube-api-access-6d8bt\") pod \"odh-model-controller-84b54b7f97-4wtpk\" (UID: \"5dcd9f86-f183-4e63-94d8-65b48b7977f3\") " pod="kserve/odh-model-controller-84b54b7f97-4wtpk"
Jul 09 09:44:30.842724 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.842697    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-4wtpk"
Jul 09 09:44:30.991497 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.991470    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-4wtpk"]
Jul 09 09:44:30.993473 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:44:30.993438    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dcd9f86_f183_4e63_94d8_65b48b7977f3.slice/crio-3319d0c80d07760262892c315b950aaf63ebce886c1ccf3b738f3d2d6f63fe11 WatchSource:0}: Error finding container 3319d0c80d07760262892c315b950aaf63ebce886c1ccf3b738f3d2d6f63fe11: Status 404 returned error can't find the container with id 3319d0c80d07760262892c315b950aaf63ebce886c1ccf3b738f3d2d6f63fe11
Jul 09 09:44:30.994756 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:30.994740    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 09:44:31.354043 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:31.354016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:44:31.419405 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:31.419376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:44:31.648431 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:31.648354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-4wtpk" event={"ID":"5dcd9f86-f183-4e63-94d8-65b48b7977f3","Type":"ContainerStarted","Data":"3319d0c80d07760262892c315b950aaf63ebce886c1ccf3b738f3d2d6f63fe11"}
Jul 09 09:44:34.671027 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:34.670992    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-4wtpk" event={"ID":"5dcd9f86-f183-4e63-94d8-65b48b7977f3","Type":"ContainerStarted","Data":"28d41fa62633c96a5dd2f353fbe359a9fba8c55b90185b1d3a8f921c98948498"}
Jul 09 09:44:34.671410 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:34.671126    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-4wtpk"
Jul 09 09:44:34.696318 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:34.696262    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-4wtpk" podStartSLOduration=1.834887733 podStartE2EDuration="4.696244075s" totalImagesPullingTime="2.861356342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:44:34.692547605 +0000 UTC m=+677.698716676" watchObservedRunningTime="2026-07-09 09:44:34.696244075 +0000 UTC m=+677.702413147"
Jul 09 09:44:40.495530 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:40.495485    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:44:45.678948 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:44:45.678919    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-4wtpk"
Jul 09 09:45:12.527549 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:45:12.527454    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:03.272142 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:03.272105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:05.348977 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:05.348938    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:05.440403 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:05.440370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:17.742083 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:17.742039    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:17.833949 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:17.833921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:17.905150 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:17.905121    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"]
Jul 09 09:46:17.908594 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:17.908578    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:17.912372 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:17.912350    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-gw-ca-root-cert\""
Jul 09 09:46:17.912737 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:17.912719    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 09:46:17.912983 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:17.912966    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-1-openshift-default-dockercfg-f9j98\""
Jul 09 09:46:17.913918 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:17.913901    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 09:46:17.928670 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:17.928648    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"]
Jul 09 09:46:18.075054 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.075024    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-data\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.075246 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.075068    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-workload-socket\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.075246 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.075144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8pcg\" (UniqueName: \"kubernetes.io/projected/165b63a4-91bb-4be2-905c-fd4c076957dc-kube-api-access-q8pcg\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.075246 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.075177    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-envoy\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.075444 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.075252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/165b63a4-91bb-4be2-905c-fd4c076957dc-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.075444 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.075276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-podinfo\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.075444 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.075314    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-workload-certs\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.075444 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.075377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-token\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.075444 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.075415    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-credential-socket\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.176558 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.176527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8pcg\" (UniqueName: \"kubernetes.io/projected/165b63a4-91bb-4be2-905c-fd4c076957dc-kube-api-access-q8pcg\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.176558 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.176559    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-envoy\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.176772 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.176689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/165b63a4-91bb-4be2-905c-fd4c076957dc-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.176772 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.176727    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-podinfo\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.176772 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.176763    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-workload-certs\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.176992 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.176826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-token\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.176992 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.176876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-credential-socket\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.176992 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.176918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-data\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.176992 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.176970    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-workload-socket\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.177214 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.177116    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-workload-certs\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.177346 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.177327    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-credential-socket\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.177443 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.177351    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-data\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.177443 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.177391    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-workload-socket\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.177848 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.177826    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/165b63a4-91bb-4be2-905c-fd4c076957dc-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.178904 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.178887    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-envoy\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.179867 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.179843    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-podinfo\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.185052 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.185023    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-token\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.185210 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.185193    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8pcg\" (UniqueName: \"kubernetes.io/projected/165b63a4-91bb-4be2-905c-fd4c076957dc-kube-api-access-q8pcg\") pod \"router-gateway-1-openshift-default-b49948d49-qcsxp\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.219069 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.219045    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:18.361325 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:18.361302    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"]
Jul 09 09:46:18.363384 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:46:18.363357    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod165b63a4_91bb_4be2_905c_fd4c076957dc.slice/crio-31bd144207b519f9ef7a4132991f3d924396c29cdeb5ec466fced0c725a38173 WatchSource:0}: Error finding container 31bd144207b519f9ef7a4132991f3d924396c29cdeb5ec466fced0c725a38173: Status 404 returned error can't find the container with id 31bd144207b519f9ef7a4132991f3d924396c29cdeb5ec466fced0c725a38173
Jul 09 09:46:19.228562 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:19.228521    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp" event={"ID":"165b63a4-91bb-4be2-905c-fd4c076957dc","Type":"ContainerStarted","Data":"31bd144207b519f9ef7a4132991f3d924396c29cdeb5ec466fced0c725a38173"}
Jul 09 09:46:21.178075 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:21.178041    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 09:46:21.178329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:21.178115    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 09:46:21.178329 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:21.178145    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 09:46:21.244656 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:21.244599    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:22.250260 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:22.250223    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp" event={"ID":"165b63a4-91bb-4be2-905c-fd4c076957dc","Type":"ContainerStarted","Data":"98814f48133a3dd5d199292ebeb7c6fe87270ce186cfb7843cc3a3c26360843e"}
Jul 09 09:46:22.273083 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:22.273020    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp" podStartSLOduration=2.460973466 podStartE2EDuration="5.272993408s" totalImagesPullingTime="2.812019942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:46:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:46:22.269115358 +0000 UTC m=+785.275284443" watchObservedRunningTime="2026-07-09 09:46:22.272993408 +0000 UTC m=+785.279162480"
Jul 09 09:46:23.219878 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:23.219844    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:23.224527 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:23.224502    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:23.256467 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:23.256438    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:23.257334 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:23.257318    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:24.863442 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:24.863402    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:32.220355 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:32.220324    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:37.091187 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:37.091151    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"]
Jul 09 09:46:37.091606 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:37.091345    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp" podUID="165b63a4-91bb-4be2-905c-fd4c076957dc" containerName="istio-proxy" containerID="cri-o://98814f48133a3dd5d199292ebeb7c6fe87270ce186cfb7843cc3a3c26360843e" gracePeriod=30
Jul 09 09:46:38.258715 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:38.258665    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp" podUID="165b63a4-91bb-4be2-905c-fd4c076957dc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:46:39.622526 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:39.622476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:39.722730 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:39.722684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:41.552536 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.552499    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:41.653641 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.653571    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-credential-socket\") pod \"165b63a4-91bb-4be2-905c-fd4c076957dc\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") "
Jul 09 09:46:41.653641 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.653609    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-envoy\") pod \"165b63a4-91bb-4be2-905c-fd4c076957dc\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") "
Jul 09 09:46:41.653882 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.653644    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-data\") pod \"165b63a4-91bb-4be2-905c-fd4c076957dc\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") "
Jul 09 09:46:41.653882 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.653661    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-podinfo\") pod \"165b63a4-91bb-4be2-905c-fd4c076957dc\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") "
Jul 09 09:46:41.653882 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.653697    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8pcg\" (UniqueName: \"kubernetes.io/projected/165b63a4-91bb-4be2-905c-fd4c076957dc-kube-api-access-q8pcg\") pod \"165b63a4-91bb-4be2-905c-fd4c076957dc\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") "
Jul 09 09:46:41.653882 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.653721    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/165b63a4-91bb-4be2-905c-fd4c076957dc-istiod-ca-cert\") pod \"165b63a4-91bb-4be2-905c-fd4c076957dc\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") "
Jul 09 09:46:41.653882 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.653744    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-workload-certs\") pod \"165b63a4-91bb-4be2-905c-fd4c076957dc\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") "
Jul 09 09:46:41.653882 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.653835    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-token\") pod \"165b63a4-91bb-4be2-905c-fd4c076957dc\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") "
Jul 09 09:46:41.654203 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.653889    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-workload-socket\") pod \"165b63a4-91bb-4be2-905c-fd4c076957dc\" (UID: \"165b63a4-91bb-4be2-905c-fd4c076957dc\") "
Jul 09 09:46:41.654203 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.653896    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "165b63a4-91bb-4be2-905c-fd4c076957dc" (UID: "165b63a4-91bb-4be2-905c-fd4c076957dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:46:41.654203 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.654042    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-data" (OuterVolumeSpecName: "istio-data") pod "165b63a4-91bb-4be2-905c-fd4c076957dc" (UID: "165b63a4-91bb-4be2-905c-fd4c076957dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:46:41.654203 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.654120    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "165b63a4-91bb-4be2-905c-fd4c076957dc" (UID: "165b63a4-91bb-4be2-905c-fd4c076957dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:46:41.654203 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.654148    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/165b63a4-91bb-4be2-905c-fd4c076957dc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "165b63a4-91bb-4be2-905c-fd4c076957dc" (UID: "165b63a4-91bb-4be2-905c-fd4c076957dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:46:41.654203 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.654161    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-credential-socket\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:46:41.654203 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.654180    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-data\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:46:41.654563 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.654287    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "165b63a4-91bb-4be2-905c-fd4c076957dc" (UID: "165b63a4-91bb-4be2-905c-fd4c076957dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:46:41.656128 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.656099    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/165b63a4-91bb-4be2-905c-fd4c076957dc-kube-api-access-q8pcg" (OuterVolumeSpecName: "kube-api-access-q8pcg") pod "165b63a4-91bb-4be2-905c-fd4c076957dc" (UID: "165b63a4-91bb-4be2-905c-fd4c076957dc"). InnerVolumeSpecName "kube-api-access-q8pcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:46:41.656128 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.656108    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "165b63a4-91bb-4be2-905c-fd4c076957dc" (UID: "165b63a4-91bb-4be2-905c-fd4c076957dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 09:46:41.656276 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.656205    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "165b63a4-91bb-4be2-905c-fd4c076957dc" (UID: "165b63a4-91bb-4be2-905c-fd4c076957dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:46:41.656399 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.656373    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-token" (OuterVolumeSpecName: "istio-token") pod "165b63a4-91bb-4be2-905c-fd4c076957dc" (UID: "165b63a4-91bb-4be2-905c-fd4c076957dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:46:41.755002 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.754980    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-podinfo\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:46:41.755002 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.755001    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8pcg\" (UniqueName: \"kubernetes.io/projected/165b63a4-91bb-4be2-905c-fd4c076957dc-kube-api-access-q8pcg\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:46:41.755131 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.755011    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/165b63a4-91bb-4be2-905c-fd4c076957dc-istiod-ca-cert\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:46:41.755131 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.755019    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-workload-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:46:41.755131 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.755028    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-token\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:46:41.755131 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.755036    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-workload-socket\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:46:41.755131 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:41.755044    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/165b63a4-91bb-4be2-905c-fd4c076957dc-istio-envoy\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:46:42.360225 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:42.360188    2577 generic.go:356] "Generic (PLEG): container finished" podID="165b63a4-91bb-4be2-905c-fd4c076957dc" containerID="98814f48133a3dd5d199292ebeb7c6fe87270ce186cfb7843cc3a3c26360843e" exitCode=0
Jul 09 09:46:42.360394 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:42.360239    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"
Jul 09 09:46:42.360394 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:42.360275    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp" event={"ID":"165b63a4-91bb-4be2-905c-fd4c076957dc","Type":"ContainerDied","Data":"98814f48133a3dd5d199292ebeb7c6fe87270ce186cfb7843cc3a3c26360843e"}
Jul 09 09:46:42.360394 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:42.360330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp" event={"ID":"165b63a4-91bb-4be2-905c-fd4c076957dc","Type":"ContainerDied","Data":"31bd144207b519f9ef7a4132991f3d924396c29cdeb5ec466fced0c725a38173"}
Jul 09 09:46:42.360394 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:42.360354    2577 scope.go:117] "RemoveContainer" containerID="98814f48133a3dd5d199292ebeb7c6fe87270ce186cfb7843cc3a3c26360843e"
Jul 09 09:46:42.371129 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:42.371110    2577 scope.go:117] "RemoveContainer" containerID="98814f48133a3dd5d199292ebeb7c6fe87270ce186cfb7843cc3a3c26360843e"
Jul 09 09:46:42.371395 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:46:42.371376    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98814f48133a3dd5d199292ebeb7c6fe87270ce186cfb7843cc3a3c26360843e\": container with ID starting with 98814f48133a3dd5d199292ebeb7c6fe87270ce186cfb7843cc3a3c26360843e not found: ID does not exist" containerID="98814f48133a3dd5d199292ebeb7c6fe87270ce186cfb7843cc3a3c26360843e"
Jul 09 09:46:42.371442 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:42.371402    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98814f48133a3dd5d199292ebeb7c6fe87270ce186cfb7843cc3a3c26360843e"} err="failed to get container status \"98814f48133a3dd5d199292ebeb7c6fe87270ce186cfb7843cc3a3c26360843e\": rpc error: code = NotFound desc = could not find container \"98814f48133a3dd5d199292ebeb7c6fe87270ce186cfb7843cc3a3c26360843e\": container with ID starting with 98814f48133a3dd5d199292ebeb7c6fe87270ce186cfb7843cc3a3c26360843e not found: ID does not exist"
Jul 09 09:46:42.382446 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:42.382427    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"]
Jul 09 09:46:42.386756 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:42.386737    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-b49948d49-qcsxp"]
Jul 09 09:46:43.552007 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:43.551975    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="165b63a4-91bb-4be2-905c-fd4c076957dc" path="/var/lib/kubelet/pods/165b63a4-91bb-4be2-905c-fd4c076957dc/volumes"
Jul 09 09:46:47.147719 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.147685    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"]
Jul 09 09:46:47.148274 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.148251    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="165b63a4-91bb-4be2-905c-fd4c076957dc" containerName="istio-proxy"
Jul 09 09:46:47.148354 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.148279    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="165b63a4-91bb-4be2-905c-fd4c076957dc" containerName="istio-proxy"
Jul 09 09:46:47.148414 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.148402    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="165b63a4-91bb-4be2-905c-fd4c076957dc" containerName="istio-proxy"
Jul 09 09:46:47.154746 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.154719    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.157483 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.157458    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 09:46:47.157587 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.157468    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-kserve-self-signed-certs\""
Jul 09 09:46:47.158224 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.158204    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 09:46:47.158297 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.158224    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 09:46:47.158297 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.158217    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 09:46:47.170368 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.170344    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"]
Jul 09 09:46:47.198314 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.198285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q9fx\" (UniqueName: \"kubernetes.io/projected/b016e6c6-9195-420a-9b34-b8f0ab2eab23-kube-api-access-7q9fx\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.198429 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.198404    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b016e6c6-9195-420a-9b34-b8f0ab2eab23-tls-certs\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.198510 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.198491    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-home\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.198576 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.198525    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.198576 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.198560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-model-cache\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.198685 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.198603    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-dshm\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.198685 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.198628    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-tmp-dir\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.221097 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.221065    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:47.299301 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.299273    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b016e6c6-9195-420a-9b34-b8f0ab2eab23-tls-certs\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.299404 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.299331    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-home\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.299404 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.299356    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.299404 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.299386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-model-cache\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.299553 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.299422    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-dshm\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.299553 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.299450    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-tmp-dir\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.299639 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.299585    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q9fx\" (UniqueName: \"kubernetes.io/projected/b016e6c6-9195-420a-9b34-b8f0ab2eab23-kube-api-access-7q9fx\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.299728 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.299710    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-home\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.299793 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.299752    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.299872 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.299792    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-tmp-dir\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.299958 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.299940    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-model-cache\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.301589 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.301570    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-dshm\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.301680 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.301666    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b016e6c6-9195-420a-9b34-b8f0ab2eab23-tls-certs\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.312917 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.312900    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q9fx\" (UniqueName: \"kubernetes.io/projected/b016e6c6-9195-420a-9b34-b8f0ab2eab23-kube-api-access-7q9fx\") pod \"precise-prefix-cache-test-kserve-b49d646d8-8sr5g\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.469914 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.469890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:46:47.512375 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.512335    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"]
Jul 09 09:46:47.518429 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.518404    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.521021 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.520996    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-epp-sa-dockercfg-m5jjz\""
Jul 09 09:46:47.528022 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.527999    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"]
Jul 09 09:46:47.553820 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.553762    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:47.601962 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.601930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.602101 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.601978    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.602101 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.602084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b05216b8-1307-4716-847a-f82015d2c93d-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.602235 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.602141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.602235 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.602186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mfm9\" (UniqueName: \"kubernetes.io/projected/b05216b8-1307-4716-847a-f82015d2c93d-kube-api-access-6mfm9\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.602336 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.602267    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.629763 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.629741    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"]
Jul 09 09:46:47.631359 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:46:47.631330    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb016e6c6_9195_420a_9b34_b8f0ab2eab23.slice/crio-e84817cca0b9e3f02bc89dc0d5e70b271b111e98933c357c53a532cc89c2a767 WatchSource:0}: Error finding container e84817cca0b9e3f02bc89dc0d5e70b271b111e98933c357c53a532cc89c2a767: Status 404 returned error can't find the container with id e84817cca0b9e3f02bc89dc0d5e70b271b111e98933c357c53a532cc89c2a767
Jul 09 09:46:47.703285 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.703261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.703383 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.703298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.703383 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.703339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b05216b8-1307-4716-847a-f82015d2c93d-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.703383 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.703361    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.703383 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.703380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mfm9\" (UniqueName: \"kubernetes.io/projected/b05216b8-1307-4716-847a-f82015d2c93d-kube-api-access-6mfm9\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.703558 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.703446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.703707 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.703683    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.703785 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.703707    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.703785 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.703754    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.703894 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.703837    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.705665 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.705646    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b05216b8-1307-4716-847a-f82015d2c93d-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.712770 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.712748    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mfm9\" (UniqueName: \"kubernetes.io/projected/b05216b8-1307-4716-847a-f82015d2c93d-kube-api-access-6mfm9\") pod \"precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.832404 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.832330    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:46:47.985407 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:47.985383    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"]
Jul 09 09:46:47.986715 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:46:47.986694    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb05216b8_1307_4716_847a_f82015d2c93d.slice/crio-1ce360b7916c0221c302d0605683a6dfbb03ec890abeb48e83afaf397afcda3a WatchSource:0}: Error finding container 1ce360b7916c0221c302d0605683a6dfbb03ec890abeb48e83afaf397afcda3a: Status 404 returned error can't find the container with id 1ce360b7916c0221c302d0605683a6dfbb03ec890abeb48e83afaf397afcda3a
Jul 09 09:46:48.402310 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:48.402272    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g" event={"ID":"b016e6c6-9195-420a-9b34-b8f0ab2eab23","Type":"ContainerStarted","Data":"e84817cca0b9e3f02bc89dc0d5e70b271b111e98933c357c53a532cc89c2a767"}
Jul 09 09:46:48.403582 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:48.403552    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" event={"ID":"b05216b8-1307-4716-847a-f82015d2c93d","Type":"ContainerStarted","Data":"1ce360b7916c0221c302d0605683a6dfbb03ec890abeb48e83afaf397afcda3a"}
Jul 09 09:46:50.321835 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:50.321771    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:51.424319 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:51.424284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g" event={"ID":"b016e6c6-9195-420a-9b34-b8f0ab2eab23","Type":"ContainerStarted","Data":"486c9efbfd47d823e9a4707ab99603c7c91fe978b0f470add13d3cac900eeeaa"}
Jul 09 09:46:51.426013 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:51.425975    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" event={"ID":"b05216b8-1307-4716-847a-f82015d2c93d","Type":"ContainerStarted","Data":"5f528871008b06d5bd3995c4ddcd54fefaed84b273b21b758b1994fe8a4b026c"}
Jul 09 09:46:52.433861 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:52.433824    2577 generic.go:356] "Generic (PLEG): container finished" podID="b05216b8-1307-4716-847a-f82015d2c93d" containerID="5f528871008b06d5bd3995c4ddcd54fefaed84b273b21b758b1994fe8a4b026c" exitCode=0
Jul 09 09:46:52.434273 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:52.433908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" event={"ID":"b05216b8-1307-4716-847a-f82015d2c93d","Type":"ContainerDied","Data":"5f528871008b06d5bd3995c4ddcd54fefaed84b273b21b758b1994fe8a4b026c"}
Jul 09 09:46:54.454277 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:54.454138    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" event={"ID":"b05216b8-1307-4716-847a-f82015d2c93d","Type":"ContainerStarted","Data":"a0827caf760cc1c071584d90f73aaf4c64a59cdeeda541e0fd93d645413f4b24"}
Jul 09 09:46:54.516905 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:54.516783    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:46:56.472666 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:56.472627    2577 generic.go:356] "Generic (PLEG): container finished" podID="b016e6c6-9195-420a-9b34-b8f0ab2eab23" containerID="486c9efbfd47d823e9a4707ab99603c7c91fe978b0f470add13d3cac900eeeaa" exitCode=0
Jul 09 09:46:56.473215 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:56.472701    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g" event={"ID":"b016e6c6-9195-420a-9b34-b8f0ab2eab23","Type":"ContainerDied","Data":"486c9efbfd47d823e9a4707ab99603c7c91fe978b0f470add13d3cac900eeeaa"}
Jul 09 09:46:58.488770 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:58.488733    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g" event={"ID":"b016e6c6-9195-420a-9b34-b8f0ab2eab23","Type":"ContainerStarted","Data":"2ae86a15507ffd6613d022ae1a08ceaa8b82dc4014f544ed0c1fa7a0d8511660"}
Jul 09 09:46:58.512304 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:58.512256    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g" podStartSLOduration=2.164739915 podStartE2EDuration="11.512242454s" totalImagesPullingTime="4.501343406s" totalInitContainerRuntime="4.846159133s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:46:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:46:58.510170002 +0000 UTC m=+821.516339097" watchObservedRunningTime="2026-07-09 09:46:58.512242454 +0000 UTC m=+821.518411572"
Jul 09 09:46:59.470338 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:46:59.470307    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:47:04.528556 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:04.528525    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk_b05216b8-1307-4716-847a-f82015d2c93d/main/0.log"
Jul 09 09:47:04.529035 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:04.528965    2577 generic.go:356] "Generic (PLEG): container finished" podID="b05216b8-1307-4716-847a-f82015d2c93d" containerID="a0827caf760cc1c071584d90f73aaf4c64a59cdeeda541e0fd93d645413f4b24" exitCode=1
Jul 09 09:47:04.529100 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:04.529028    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" event={"ID":"b05216b8-1307-4716-847a-f82015d2c93d","Type":"ContainerDied","Data":"a0827caf760cc1c071584d90f73aaf4c64a59cdeeda541e0fd93d645413f4b24"}
Jul 09 09:47:07.470654 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:07.470619    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:47:07.486234 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:07.486208    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:47:07.564775 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:07.564746    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:47:24.653040 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:24.653008    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk_b05216b8-1307-4716-847a-f82015d2c93d/main/0.log"
Jul 09 09:47:24.653533 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:24.653420    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" event={"ID":"b05216b8-1307-4716-847a-f82015d2c93d","Type":"ContainerStarted","Data":"7ca61219be98e0e5a73667359690c607838c103858d6af88b9826fe2ade22c66"}
Jul 09 09:47:24.653762 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:24.653747    2577 scope.go:117] "RemoveContainer" containerID="a0827caf760cc1c071584d90f73aaf4c64a59cdeeda541e0fd93d645413f4b24"
Jul 09 09:47:25.661169 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:25.661139    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk_b05216b8-1307-4716-847a-f82015d2c93d/main/0.log"
Jul 09 09:47:25.661665 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:25.661639    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" event={"ID":"b05216b8-1307-4716-847a-f82015d2c93d","Type":"ContainerStarted","Data":"21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b"}
Jul 09 09:47:25.661844 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:25.661819    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:47:25.686991 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:25.686946    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" podStartSLOduration=3.4191876 podStartE2EDuration="38.686930893s" totalImagesPullingTime="34.613556703s" totalInitContainerRuntime="654.18659ms" isStatefulPod=false podCreationTimestamp="2026-07-09 09:46:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:47:25.683025042 +0000 UTC m=+848.689194117" watchObservedRunningTime="2026-07-09 09:47:25.686930893 +0000 UTC m=+848.693099964"
Jul 09 09:47:27.832882 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:27.832851    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:47:27.833353 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:27.832907    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:47:32.526521 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:32.526467    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:47:34.127563 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:34.127518    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:47:37.834851 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:47:37.834819    2577 logging.go:55] [core] [Channel #17 SubChannel #18]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.48:9003", ServerName: "10.132.0.48:9003", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.48:9003: connect: connection refused"
Jul 09 09:47:37.836132 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:37.836106    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:47:37.837253 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:37.837225    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:47:38.833489 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:38.833451    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.132.0.48:9003\" within 1s: context deadline exceeded"
Jul 09 09:47:46.906385 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:46.906340    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:47:47.128077 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:47.128043    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:47:58.736689 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:47:58.736660    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:48:07.333840 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:07.333778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:48:10.362490 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:10.362414    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"]
Jul 09 09:48:10.362975 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:10.362641    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="tokenizer" containerID="cri-o://7ca61219be98e0e5a73667359690c607838c103858d6af88b9826fe2ade22c66" gracePeriod=60
Jul 09 09:48:10.366333 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:10.366296    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:48:10.374086 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:10.374060    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"]
Jul 09 09:48:10.374823 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:48:10.374770    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 09:48:10.374823 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:48:10.374823 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:48:10.374823 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b" cmd=["/bin/sleep","15"]
Jul 09 09:48:10.375099 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:10.374844    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" message="" err=<
Jul 09 09:48:10.375099 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:48:10.375099 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:48:10.375099 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 09:48:10.375099 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:48:10.374860    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 09:48:10.375099 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:48:10.375099 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:48:10.375099 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="main" containerID="cri-o://21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b"
Jul 09 09:48:10.375099 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:10.374901    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="main" containerID="cri-o://21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b" gracePeriod=60
Jul 09 09:48:10.432818 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:10.432770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:48:10.911337 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:10.911306    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk_b05216b8-1307-4716-847a-f82015d2c93d/main/0.log"
Jul 09 09:48:10.911703 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:10.911680    2577 generic.go:356] "Generic (PLEG): container finished" podID="b05216b8-1307-4716-847a-f82015d2c93d" containerID="21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b" exitCode=0
Jul 09 09:48:10.911776 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:10.911752    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" event={"ID":"b05216b8-1307-4716-847a-f82015d2c93d","Type":"ContainerDied","Data":"21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b"}
Jul 09 09:48:10.911861 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:10.911846    2577 scope.go:117] "RemoveContainer" containerID="a0827caf760cc1c071584d90f73aaf4c64a59cdeeda541e0fd93d645413f4b24"
Jul 09 09:48:11.713852 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.713820    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:48:11.827871 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.827789    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-uds\") pod \"b05216b8-1307-4716-847a-f82015d2c93d\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") "
Jul 09 09:48:11.827871 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.827855    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b05216b8-1307-4716-847a-f82015d2c93d-tls-certs\") pod \"b05216b8-1307-4716-847a-f82015d2c93d\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") "
Jul 09 09:48:11.828037 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.827874    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-kserve-provision-location\") pod \"b05216b8-1307-4716-847a-f82015d2c93d\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") "
Jul 09 09:48:11.828037 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.827935    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-tmp\") pod \"b05216b8-1307-4716-847a-f82015d2c93d\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") "
Jul 09 09:48:11.828037 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.827970    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-cache\") pod \"b05216b8-1307-4716-847a-f82015d2c93d\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") "
Jul 09 09:48:11.828037 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.828016    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mfm9\" (UniqueName: \"kubernetes.io/projected/b05216b8-1307-4716-847a-f82015d2c93d-kube-api-access-6mfm9\") pod \"b05216b8-1307-4716-847a-f82015d2c93d\" (UID: \"b05216b8-1307-4716-847a-f82015d2c93d\") "
Jul 09 09:48:11.828256 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.828060    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "b05216b8-1307-4716-847a-f82015d2c93d" (UID: "b05216b8-1307-4716-847a-f82015d2c93d"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:48:11.828256 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.828232    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:48:11.828365 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.828317    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "b05216b8-1307-4716-847a-f82015d2c93d" (UID: "b05216b8-1307-4716-847a-f82015d2c93d"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:48:11.828411 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.828371    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "b05216b8-1307-4716-847a-f82015d2c93d" (UID: "b05216b8-1307-4716-847a-f82015d2c93d"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:48:11.828683 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.828660    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b05216b8-1307-4716-847a-f82015d2c93d" (UID: "b05216b8-1307-4716-847a-f82015d2c93d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:48:11.830124 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.830092    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b05216b8-1307-4716-847a-f82015d2c93d-kube-api-access-6mfm9" (OuterVolumeSpecName: "kube-api-access-6mfm9") pod "b05216b8-1307-4716-847a-f82015d2c93d" (UID: "b05216b8-1307-4716-847a-f82015d2c93d"). InnerVolumeSpecName "kube-api-access-6mfm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:48:11.830209 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.830145    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b05216b8-1307-4716-847a-f82015d2c93d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b05216b8-1307-4716-847a-f82015d2c93d" (UID: "b05216b8-1307-4716-847a-f82015d2c93d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:48:11.919005 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.918973    2577 generic.go:356] "Generic (PLEG): container finished" podID="b05216b8-1307-4716-847a-f82015d2c93d" containerID="7ca61219be98e0e5a73667359690c607838c103858d6af88b9826fe2ade22c66" exitCode=0
Jul 09 09:48:11.919120 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.919038    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"
Jul 09 09:48:11.919120 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.919060    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" event={"ID":"b05216b8-1307-4716-847a-f82015d2c93d","Type":"ContainerDied","Data":"7ca61219be98e0e5a73667359690c607838c103858d6af88b9826fe2ade22c66"}
Jul 09 09:48:11.919120 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.919110    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk" event={"ID":"b05216b8-1307-4716-847a-f82015d2c93d","Type":"ContainerDied","Data":"1ce360b7916c0221c302d0605683a6dfbb03ec890abeb48e83afaf397afcda3a"}
Jul 09 09:48:11.919292 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.919134    2577 scope.go:117] "RemoveContainer" containerID="21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b"
Jul 09 09:48:11.928634 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.928552    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b05216b8-1307-4716-847a-f82015d2c93d-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:48:11.928634 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.928572    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:48:11.928634 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.928586    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-tmp\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:48:11.928634 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.928601    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b05216b8-1307-4716-847a-f82015d2c93d-tokenizer-cache\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:48:11.928634 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.928614    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mfm9\" (UniqueName: \"kubernetes.io/projected/b05216b8-1307-4716-847a-f82015d2c93d-kube-api-access-6mfm9\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:48:11.928634 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.928613    2577 scope.go:117] "RemoveContainer" containerID="7ca61219be98e0e5a73667359690c607838c103858d6af88b9826fe2ade22c66"
Jul 09 09:48:11.936242 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.936227    2577 scope.go:117] "RemoveContainer" containerID="5f528871008b06d5bd3995c4ddcd54fefaed84b273b21b758b1994fe8a4b026c"
Jul 09 09:48:11.942458 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.942438    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"]
Jul 09 09:48:11.944437 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.944410    2577 scope.go:117] "RemoveContainer" containerID="21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b"
Jul 09 09:48:11.944767 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:48:11.944746    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b\": container with ID starting with 21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b not found: ID does not exist" containerID="21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b"
Jul 09 09:48:11.944888 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.944773    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b"} err="failed to get container status \"21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b\": rpc error: code = NotFound desc = could not find container \"21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b\": container with ID starting with 21273d7b0b586df9b2fbf3c6ca778a5ceeba4df8d351e1ee1b0bb794a5078c9b not found: ID does not exist"
Jul 09 09:48:11.944888 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.944790    2577 scope.go:117] "RemoveContainer" containerID="7ca61219be98e0e5a73667359690c607838c103858d6af88b9826fe2ade22c66"
Jul 09 09:48:11.945131 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:48:11.945107    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ca61219be98e0e5a73667359690c607838c103858d6af88b9826fe2ade22c66\": container with ID starting with 7ca61219be98e0e5a73667359690c607838c103858d6af88b9826fe2ade22c66 not found: ID does not exist" containerID="7ca61219be98e0e5a73667359690c607838c103858d6af88b9826fe2ade22c66"
Jul 09 09:48:11.945192 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.945141    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ca61219be98e0e5a73667359690c607838c103858d6af88b9826fe2ade22c66"} err="failed to get container status \"7ca61219be98e0e5a73667359690c607838c103858d6af88b9826fe2ade22c66\": rpc error: code = NotFound desc = could not find container \"7ca61219be98e0e5a73667359690c607838c103858d6af88b9826fe2ade22c66\": container with ID starting with 7ca61219be98e0e5a73667359690c607838c103858d6af88b9826fe2ade22c66 not found: ID does not exist"
Jul 09 09:48:11.945192 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.945161    2577 scope.go:117] "RemoveContainer" containerID="5f528871008b06d5bd3995c4ddcd54fefaed84b273b21b758b1994fe8a4b026c"
Jul 09 09:48:11.945373 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:48:11.945356    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f528871008b06d5bd3995c4ddcd54fefaed84b273b21b758b1994fe8a4b026c\": container with ID starting with 5f528871008b06d5bd3995c4ddcd54fefaed84b273b21b758b1994fe8a4b026c not found: ID does not exist" containerID="5f528871008b06d5bd3995c4ddcd54fefaed84b273b21b758b1994fe8a4b026c"
Jul 09 09:48:11.945435 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.945387    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f528871008b06d5bd3995c4ddcd54fefaed84b273b21b758b1994fe8a4b026c"} err="failed to get container status \"5f528871008b06d5bd3995c4ddcd54fefaed84b273b21b758b1994fe8a4b026c\": rpc error: code = NotFound desc = could not find container \"5f528871008b06d5bd3995c4ddcd54fefaed84b273b21b758b1994fe8a4b026c\": container with ID starting with 5f528871008b06d5bd3995c4ddcd54fefaed84b273b21b758b1994fe8a4b026c not found: ID does not exist"
Jul 09 09:48:11.946939 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:11.946919    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-78bbd8bb77thk"]
Jul 09 09:48:13.551465 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:13.551428    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b05216b8-1307-4716-847a-f82015d2c93d" path="/var/lib/kubelet/pods/b05216b8-1307-4716-847a-f82015d2c93d/volumes"
Jul 09 09:48:17.378023 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:17.377996    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:48:17.378492 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:17.378181    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:48:23.099683 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:23.099619    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:48:25.397484 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.397447    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g" podUID="b016e6c6-9195-420a-9b34-b8f0ab2eab23" containerName="main" containerID="cri-o://2ae86a15507ffd6613d022ae1a08ceaa8b82dc4014f544ed0c1fa7a0d8511660" gracePeriod=45
Jul 09 09:48:25.551056 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.551019    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g" podUID="b016e6c6-9195-420a-9b34-b8f0ab2eab23" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/health\": dial tcp 10.132.0.47:8000: connect: connection refused"
Jul 09 09:48:25.679053 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.679029    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:48:25.718684 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.718654    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-model-cache\") pod \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") "
Jul 09 09:48:25.718862 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.718699    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-tmp-dir\") pod \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") "
Jul 09 09:48:25.718862 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.718771    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q9fx\" (UniqueName: \"kubernetes.io/projected/b016e6c6-9195-420a-9b34-b8f0ab2eab23-kube-api-access-7q9fx\") pod \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") "
Jul 09 09:48:25.718862 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.718828    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-kserve-provision-location\") pod \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") "
Jul 09 09:48:25.719064 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.718862    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-model-cache" (OuterVolumeSpecName: "model-cache") pod "b016e6c6-9195-420a-9b34-b8f0ab2eab23" (UID: "b016e6c6-9195-420a-9b34-b8f0ab2eab23"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:48:25.719064 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.718881    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-home\") pod \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") "
Jul 09 09:48:25.719064 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.718945    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b016e6c6-9195-420a-9b34-b8f0ab2eab23" (UID: "b016e6c6-9195-420a-9b34-b8f0ab2eab23"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:48:25.719219 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.719121    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-home" (OuterVolumeSpecName: "home") pod "b016e6c6-9195-420a-9b34-b8f0ab2eab23" (UID: "b016e6c6-9195-420a-9b34-b8f0ab2eab23"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:48:25.719219 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.719123    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-model-cache\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:48:25.719219 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.719157    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-tmp-dir\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:48:25.720659 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.720637    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b016e6c6-9195-420a-9b34-b8f0ab2eab23-kube-api-access-7q9fx" (OuterVolumeSpecName: "kube-api-access-7q9fx") pod "b016e6c6-9195-420a-9b34-b8f0ab2eab23" (UID: "b016e6c6-9195-420a-9b34-b8f0ab2eab23"). InnerVolumeSpecName "kube-api-access-7q9fx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:48:25.773487 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.773458    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b016e6c6-9195-420a-9b34-b8f0ab2eab23" (UID: "b016e6c6-9195-420a-9b34-b8f0ab2eab23"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:48:25.820084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.820063    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-dshm\") pod \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") "
Jul 09 09:48:25.820219 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.820096    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b016e6c6-9195-420a-9b34-b8f0ab2eab23-tls-certs\") pod \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\" (UID: \"b016e6c6-9195-420a-9b34-b8f0ab2eab23\") "
Jul 09 09:48:25.820219 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.820206    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q9fx\" (UniqueName: \"kubernetes.io/projected/b016e6c6-9195-420a-9b34-b8f0ab2eab23-kube-api-access-7q9fx\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:48:25.820219 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.820217    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:48:25.820383 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.820226    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-home\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:48:25.821928 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.821907    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-dshm" (OuterVolumeSpecName: "dshm") pod "b016e6c6-9195-420a-9b34-b8f0ab2eab23" (UID: "b016e6c6-9195-420a-9b34-b8f0ab2eab23"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:48:25.822023 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.821997    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b016e6c6-9195-420a-9b34-b8f0ab2eab23-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b016e6c6-9195-420a-9b34-b8f0ab2eab23" (UID: "b016e6c6-9195-420a-9b34-b8f0ab2eab23"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:48:25.921403 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.921354    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b016e6c6-9195-420a-9b34-b8f0ab2eab23-dshm\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:48:25.921403 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:25.921373    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b016e6c6-9195-420a-9b34-b8f0ab2eab23-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:48:26.002538 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:26.002511    2577 generic.go:356] "Generic (PLEG): container finished" podID="b016e6c6-9195-420a-9b34-b8f0ab2eab23" containerID="2ae86a15507ffd6613d022ae1a08ceaa8b82dc4014f544ed0c1fa7a0d8511660" exitCode=0
Jul 09 09:48:26.002614 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:26.002566    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"
Jul 09 09:48:26.002614 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:26.002593    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g" event={"ID":"b016e6c6-9195-420a-9b34-b8f0ab2eab23","Type":"ContainerDied","Data":"2ae86a15507ffd6613d022ae1a08ceaa8b82dc4014f544ed0c1fa7a0d8511660"}
Jul 09 09:48:26.002703 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:26.002645    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g" event={"ID":"b016e6c6-9195-420a-9b34-b8f0ab2eab23","Type":"ContainerDied","Data":"e84817cca0b9e3f02bc89dc0d5e70b271b111e98933c357c53a532cc89c2a767"}
Jul 09 09:48:26.002703 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:26.002671    2577 scope.go:117] "RemoveContainer" containerID="2ae86a15507ffd6613d022ae1a08ceaa8b82dc4014f544ed0c1fa7a0d8511660"
Jul 09 09:48:26.011280 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:26.011256    2577 scope.go:117] "RemoveContainer" containerID="486c9efbfd47d823e9a4707ab99603c7c91fe978b0f470add13d3cac900eeeaa"
Jul 09 09:48:26.019308 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:26.019288    2577 scope.go:117] "RemoveContainer" containerID="2ae86a15507ffd6613d022ae1a08ceaa8b82dc4014f544ed0c1fa7a0d8511660"
Jul 09 09:48:26.019549 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:48:26.019532    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ae86a15507ffd6613d022ae1a08ceaa8b82dc4014f544ed0c1fa7a0d8511660\": container with ID starting with 2ae86a15507ffd6613d022ae1a08ceaa8b82dc4014f544ed0c1fa7a0d8511660 not found: ID does not exist" containerID="2ae86a15507ffd6613d022ae1a08ceaa8b82dc4014f544ed0c1fa7a0d8511660"
Jul 09 09:48:26.019596 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:26.019558    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ae86a15507ffd6613d022ae1a08ceaa8b82dc4014f544ed0c1fa7a0d8511660"} err="failed to get container status \"2ae86a15507ffd6613d022ae1a08ceaa8b82dc4014f544ed0c1fa7a0d8511660\": rpc error: code = NotFound desc = could not find container \"2ae86a15507ffd6613d022ae1a08ceaa8b82dc4014f544ed0c1fa7a0d8511660\": container with ID starting with 2ae86a15507ffd6613d022ae1a08ceaa8b82dc4014f544ed0c1fa7a0d8511660 not found: ID does not exist"
Jul 09 09:48:26.019596 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:26.019574    2577 scope.go:117] "RemoveContainer" containerID="486c9efbfd47d823e9a4707ab99603c7c91fe978b0f470add13d3cac900eeeaa"
Jul 09 09:48:26.019854 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:48:26.019831    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"486c9efbfd47d823e9a4707ab99603c7c91fe978b0f470add13d3cac900eeeaa\": container with ID starting with 486c9efbfd47d823e9a4707ab99603c7c91fe978b0f470add13d3cac900eeeaa not found: ID does not exist" containerID="486c9efbfd47d823e9a4707ab99603c7c91fe978b0f470add13d3cac900eeeaa"
Jul 09 09:48:26.019908 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:26.019863    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"486c9efbfd47d823e9a4707ab99603c7c91fe978b0f470add13d3cac900eeeaa"} err="failed to get container status \"486c9efbfd47d823e9a4707ab99603c7c91fe978b0f470add13d3cac900eeeaa\": rpc error: code = NotFound desc = could not find container \"486c9efbfd47d823e9a4707ab99603c7c91fe978b0f470add13d3cac900eeeaa\": container with ID starting with 486c9efbfd47d823e9a4707ab99603c7c91fe978b0f470add13d3cac900eeeaa not found: ID does not exist"
Jul 09 09:48:26.025750 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:26.025729    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"]
Jul 09 09:48:26.030184 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:26.030161    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-b49d646d8-8sr5g"]
Jul 09 09:48:27.554850 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:27.554819    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b016e6c6-9195-420a-9b34-b8f0ab2eab23" path="/var/lib/kubelet/pods/b016e6c6-9195-420a-9b34-b8f0ab2eab23/volumes"
Jul 09 09:48:37.421368 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.421332    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:48:37.830759 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.830723    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"]
Jul 09 09:48:37.831099 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831087    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b016e6c6-9195-420a-9b34-b8f0ab2eab23" containerName="storage-initializer"
Jul 09 09:48:37.831141 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831101    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b016e6c6-9195-420a-9b34-b8f0ab2eab23" containerName="storage-initializer"
Jul 09 09:48:37.831141 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831109    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="storage-initializer"
Jul 09 09:48:37.831141 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831115    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="storage-initializer"
Jul 09 09:48:37.831141 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831129    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="main"
Jul 09 09:48:37.831141 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831135    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="main"
Jul 09 09:48:37.831141 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831141    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="main"
Jul 09 09:48:37.831332 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831147    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="main"
Jul 09 09:48:37.831332 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831156    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="tokenizer"
Jul 09 09:48:37.831332 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831161    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="tokenizer"
Jul 09 09:48:37.831332 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831173    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b016e6c6-9195-420a-9b34-b8f0ab2eab23" containerName="main"
Jul 09 09:48:37.831332 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831179    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b016e6c6-9195-420a-9b34-b8f0ab2eab23" containerName="main"
Jul 09 09:48:37.831332 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831233    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="main"
Jul 09 09:48:37.831332 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831242    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="tokenizer"
Jul 09 09:48:37.831332 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831249    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b05216b8-1307-4716-847a-f82015d2c93d" containerName="main"
Jul 09 09:48:37.831332 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.831255    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b016e6c6-9195-420a-9b34-b8f0ab2eab23" containerName="main"
Jul 09 09:48:37.835200 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.835182    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:48:37.840244 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.840226    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 09:48:37.841076 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.841055    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs\""
Jul 09 09:48:37.841480 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.841461    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 09:48:37.843134 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.843115    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-router-managed-test-llm-e45d1f79-epp-sa-dockercfg-85wjj\""
Jul 09 09:48:37.852073 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.852056    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 09:48:37.869727 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.869704    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"]
Jul 09 09:48:37.870233 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.870208    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:48:37.900019 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.899995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2glhm\" (UniqueName: \"kubernetes.io/projected/964fa76f-963e-4d26-be74-24f543105bcf-kube-api-access-2glhm\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7\" (UID: \"964fa76f-963e-4d26-be74-24f543105bcf\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:48:37.900124 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.900029    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964fa76f-963e-4d26-be74-24f543105bcf-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7\" (UID: \"964fa76f-963e-4d26-be74-24f543105bcf\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:48:37.900170 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:37.900117    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/964fa76f-963e-4d26-be74-24f543105bcf-tokenizer-uds\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7\" (UID: \"964fa76f-963e-4d26-be74-24f543105bcf\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:48:38.000653 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:38.000624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2glhm\" (UniqueName: \"kubernetes.io/projected/964fa76f-963e-4d26-be74-24f543105bcf-kube-api-access-2glhm\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7\" (UID: \"964fa76f-963e-4d26-be74-24f543105bcf\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:48:38.000767 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:38.000658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964fa76f-963e-4d26-be74-24f543105bcf-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7\" (UID: \"964fa76f-963e-4d26-be74-24f543105bcf\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:48:38.000767 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:38.000710    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/964fa76f-963e-4d26-be74-24f543105bcf-tokenizer-uds\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7\" (UID: \"964fa76f-963e-4d26-be74-24f543105bcf\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:48:38.001142 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:38.001118    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/964fa76f-963e-4d26-be74-24f543105bcf-tokenizer-uds\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7\" (UID: \"964fa76f-963e-4d26-be74-24f543105bcf\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:48:38.003220 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:38.003199    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964fa76f-963e-4d26-be74-24f543105bcf-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7\" (UID: \"964fa76f-963e-4d26-be74-24f543105bcf\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:48:38.011689 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:38.011664    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2glhm\" (UniqueName: \"kubernetes.io/projected/964fa76f-963e-4d26-be74-24f543105bcf-kube-api-access-2glhm\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7\" (UID: \"964fa76f-963e-4d26-be74-24f543105bcf\") " pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:48:38.145480 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:38.145412    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:48:38.315909 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:38.315876    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"]
Jul 09 09:48:38.334619 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:48:38.334589    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod964fa76f_963e_4d26_be74_24f543105bcf.slice/crio-b909c434bcf0ebb27d1aca8c3a0b004ffcc018777ed701a7a392c1c49efd9914 WatchSource:0}: Error finding container b909c434bcf0ebb27d1aca8c3a0b004ffcc018777ed701a7a392c1c49efd9914: Status 404 returned error can't find the container with id b909c434bcf0ebb27d1aca8c3a0b004ffcc018777ed701a7a392c1c49efd9914
Jul 09 09:48:39.080435 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:39.080402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7" event={"ID":"964fa76f-963e-4d26-be74-24f543105bcf","Type":"ContainerStarted","Data":"fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a"}
Jul 09 09:48:39.080890 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:39.080452    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7" event={"ID":"964fa76f-963e-4d26-be74-24f543105bcf","Type":"ContainerStarted","Data":"b909c434bcf0ebb27d1aca8c3a0b004ffcc018777ed701a7a392c1c49efd9914"}
Jul 09 09:48:39.080890 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:39.080481    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:48:39.103252 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:39.103210    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7" podStartSLOduration=2.103195076 podStartE2EDuration="2.103195076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:48:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:48:39.100510573 +0000 UTC m=+922.106679643" watchObservedRunningTime="2026-07-09 09:48:39.103195076 +0000 UTC m=+922.109364146"
Jul 09 09:48:43.385445 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:48:43.385402    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:49:10.086548 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:10.086522    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:49:11.337432 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.337401    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"]
Jul 09 09:49:11.347515 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:49:11.347481    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 09:49:11.347515 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:49:11.347515 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:49:11.347515 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a" cmd=["/bin/sleep","15"]
Jul 09 09:49:11.347754 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.347528    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7" message="" err=<
Jul 09 09:49:11.347754 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:49:11.347754 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:49:11.347754 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 09:49:11.347754 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:49:11.347539    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 09:49:11.347754 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:49:11.347754 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:49:11.347754 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7" podUID="964fa76f-963e-4d26-be74-24f543105bcf" containerName="main" containerID="cri-o://fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a"
Jul 09 09:49:11.347754 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.347582    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7" podUID="964fa76f-963e-4d26-be74-24f543105bcf" containerName="main" containerID="cri-o://fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a" gracePeriod=60
Jul 09 09:49:11.397080 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.397052    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:49:11.598048 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.597990    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:49:11.653099 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.653059    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964fa76f-963e-4d26-be74-24f543105bcf-tls-certs\") pod \"964fa76f-963e-4d26-be74-24f543105bcf\" (UID: \"964fa76f-963e-4d26-be74-24f543105bcf\") "
Jul 09 09:49:11.653252 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.653153    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/964fa76f-963e-4d26-be74-24f543105bcf-tokenizer-uds\") pod \"964fa76f-963e-4d26-be74-24f543105bcf\" (UID: \"964fa76f-963e-4d26-be74-24f543105bcf\") "
Jul 09 09:49:11.653252 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.653231    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2glhm\" (UniqueName: \"kubernetes.io/projected/964fa76f-963e-4d26-be74-24f543105bcf-kube-api-access-2glhm\") pod \"964fa76f-963e-4d26-be74-24f543105bcf\" (UID: \"964fa76f-963e-4d26-be74-24f543105bcf\") "
Jul 09 09:49:11.653428 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.653405    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/964fa76f-963e-4d26-be74-24f543105bcf-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "964fa76f-963e-4d26-be74-24f543105bcf" (UID: "964fa76f-963e-4d26-be74-24f543105bcf"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:49:11.655114 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.655086    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/964fa76f-963e-4d26-be74-24f543105bcf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "964fa76f-963e-4d26-be74-24f543105bcf" (UID: "964fa76f-963e-4d26-be74-24f543105bcf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:49:11.655215 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.655140    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/964fa76f-963e-4d26-be74-24f543105bcf-kube-api-access-2glhm" (OuterVolumeSpecName: "kube-api-access-2glhm") pod "964fa76f-963e-4d26-be74-24f543105bcf" (UID: "964fa76f-963e-4d26-be74-24f543105bcf"). InnerVolumeSpecName "kube-api-access-2glhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:49:11.754762 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.754733    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/964fa76f-963e-4d26-be74-24f543105bcf-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:49:11.754762 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.754759    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2glhm\" (UniqueName: \"kubernetes.io/projected/964fa76f-963e-4d26-be74-24f543105bcf-kube-api-access-2glhm\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:49:11.754942 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:11.754768    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964fa76f-963e-4d26-be74-24f543105bcf-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:49:12.265638 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:12.265601    2577 generic.go:356] "Generic (PLEG): container finished" podID="964fa76f-963e-4d26-be74-24f543105bcf" containerID="fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a" exitCode=0
Jul 09 09:49:12.265854 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:12.265643    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"
Jul 09 09:49:12.265854 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:12.265687    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7" event={"ID":"964fa76f-963e-4d26-be74-24f543105bcf","Type":"ContainerDied","Data":"fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a"}
Jul 09 09:49:12.265854 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:12.265734    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7" event={"ID":"964fa76f-963e-4d26-be74-24f543105bcf","Type":"ContainerDied","Data":"b909c434bcf0ebb27d1aca8c3a0b004ffcc018777ed701a7a392c1c49efd9914"}
Jul 09 09:49:12.265854 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:12.265758    2577 scope.go:117] "RemoveContainer" containerID="fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a"
Jul 09 09:49:12.275602 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:12.275585    2577 scope.go:117] "RemoveContainer" containerID="fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a"
Jul 09 09:49:12.275869 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:49:12.275851    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a\": container with ID starting with fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a not found: ID does not exist" containerID="fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a"
Jul 09 09:49:12.275941 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:12.275881    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a"} err="failed to get container status \"fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a\": rpc error: code = NotFound desc = could not find container \"fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a\": container with ID starting with fc5f121e4ab86d8e803fd1013cfca06cc6758fc59bfe0a702b22750a54f3391a not found: ID does not exist"
Jul 09 09:49:12.287102 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:12.287081    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"]
Jul 09 09:49:12.291097 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:12.291075    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schefq7b7"]
Jul 09 09:49:13.326649 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:13.326599    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:49:13.558019 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:13.557983    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="964fa76f-963e-4d26-be74-24f543105bcf" path="/var/lib/kubelet/pods/964fa76f-963e-4d26-be74-24f543105bcf/volumes"
Jul 09 09:49:45.929704 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:45.929607    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:49:46.126553 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.126521    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"]
Jul 09 09:49:46.126991 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.126968    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="964fa76f-963e-4d26-be74-24f543105bcf" containerName="main"
Jul 09 09:49:46.127142 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.127035    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="964fa76f-963e-4d26-be74-24f543105bcf" containerName="main"
Jul 09 09:49:46.127215 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.127174    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="964fa76f-963e-4d26-be74-24f543105bcf" containerName="main"
Jul 09 09:49:46.130621 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.130600    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:49:46.134008 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.133986    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 09:49:46.134675 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.134654    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 09:49:46.134833 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.134814    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 09:49:46.134909 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.134875    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs\""
Jul 09 09:49:46.134909 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.134814    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-router-managed-test-llm-5b1e8f15-epp-sa-dockercfg-dnc2b\""
Jul 09 09:49:46.145469 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.145449    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"]
Jul 09 09:49:46.163745 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.163721    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:49:46.309916 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.309886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sz7b\" (UniqueName: \"kubernetes.io/projected/f0149903-58c4-45b8-84ca-df2d4f1745b1-kube-api-access-8sz7b\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x\" (UID: \"f0149903-58c4-45b8-84ca-df2d4f1745b1\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:49:46.310078 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.309922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0149903-58c4-45b8-84ca-df2d4f1745b1-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x\" (UID: \"f0149903-58c4-45b8-84ca-df2d4f1745b1\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:49:46.310078 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.310004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f0149903-58c4-45b8-84ca-df2d4f1745b1-tokenizer-uds\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x\" (UID: \"f0149903-58c4-45b8-84ca-df2d4f1745b1\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:49:46.410685 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.410656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sz7b\" (UniqueName: \"kubernetes.io/projected/f0149903-58c4-45b8-84ca-df2d4f1745b1-kube-api-access-8sz7b\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x\" (UID: \"f0149903-58c4-45b8-84ca-df2d4f1745b1\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:49:46.410853 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.410689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0149903-58c4-45b8-84ca-df2d4f1745b1-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x\" (UID: \"f0149903-58c4-45b8-84ca-df2d4f1745b1\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:49:46.410853 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.410726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f0149903-58c4-45b8-84ca-df2d4f1745b1-tokenizer-uds\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x\" (UID: \"f0149903-58c4-45b8-84ca-df2d4f1745b1\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:49:46.411071 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.411054    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f0149903-58c4-45b8-84ca-df2d4f1745b1-tokenizer-uds\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x\" (UID: \"f0149903-58c4-45b8-84ca-df2d4f1745b1\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:49:46.413235 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.413211    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0149903-58c4-45b8-84ca-df2d4f1745b1-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x\" (UID: \"f0149903-58c4-45b8-84ca-df2d4f1745b1\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:49:46.418559 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.418537    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sz7b\" (UniqueName: \"kubernetes.io/projected/f0149903-58c4-45b8-84ca-df2d4f1745b1-kube-api-access-8sz7b\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x\" (UID: \"f0149903-58c4-45b8-84ca-df2d4f1745b1\") " pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:49:46.441369 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.441347    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:49:46.578782 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:46.578758    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"]
Jul 09 09:49:46.580921 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:49:46.580893    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0149903_58c4_45b8_84ca_df2d4f1745b1.slice/crio-0050ebd3be621fe067cb4ea99adefe8d2e75febb99695c4ee6b139ab91ab4c04 WatchSource:0}: Error finding container 0050ebd3be621fe067cb4ea99adefe8d2e75febb99695c4ee6b139ab91ab4c04: Status 404 returned error can't find the container with id 0050ebd3be621fe067cb4ea99adefe8d2e75febb99695c4ee6b139ab91ab4c04
Jul 09 09:49:47.457269 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:47.457234    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x" event={"ID":"f0149903-58c4-45b8-84ca-df2d4f1745b1","Type":"ContainerStarted","Data":"2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9"}
Jul 09 09:49:47.457707 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:47.457285    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x" event={"ID":"f0149903-58c4-45b8-84ca-df2d4f1745b1","Type":"ContainerStarted","Data":"0050ebd3be621fe067cb4ea99adefe8d2e75febb99695c4ee6b139ab91ab4c04"}
Jul 09 09:49:47.457707 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:47.457320    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:49:47.481438 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:47.481394    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x" podStartSLOduration=1.481378477 podStartE2EDuration="1.481378477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:49:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:49:47.479443406 +0000 UTC m=+990.485612504" watchObservedRunningTime="2026-07-09 09:49:47.481378477 +0000 UTC m=+990.487547570"
Jul 09 09:49:56.933301 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:49:56.933220    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:50:18.465572 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:18.465543    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:50:21.729389 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:21.729340    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:50:21.793493 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:21.793455    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:50:30.113553 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.113517    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:50:30.287627 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.287582    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"]
Jul 09 09:50:30.291669 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.291644    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:50:30.295062 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.295034    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs\""
Jul 09 09:50:30.295187 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.295035    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-sa-dockercfg-6x7hq\""
Jul 09 09:50:30.301240 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.301215    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"]
Jul 09 09:50:30.324775 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.324752    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:50:30.330901 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.330878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmx2n\" (UniqueName: \"kubernetes.io/projected/1e7e627c-9513-4b99-ab07-618cd023ee0d-kube-api-access-wmx2n\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk\" (UID: \"1e7e627c-9513-4b99-ab07-618cd023ee0d\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:50:30.331009 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.330931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1e7e627c-9513-4b99-ab07-618cd023ee0d-tokenizer-uds\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk\" (UID: \"1e7e627c-9513-4b99-ab07-618cd023ee0d\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:50:30.331009 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.330998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e7e627c-9513-4b99-ab07-618cd023ee0d-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk\" (UID: \"1e7e627c-9513-4b99-ab07-618cd023ee0d\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:50:30.431555 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.431488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1e7e627c-9513-4b99-ab07-618cd023ee0d-tokenizer-uds\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk\" (UID: \"1e7e627c-9513-4b99-ab07-618cd023ee0d\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:50:30.431555 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.431530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e7e627c-9513-4b99-ab07-618cd023ee0d-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk\" (UID: \"1e7e627c-9513-4b99-ab07-618cd023ee0d\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:50:30.431731 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.431708    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmx2n\" (UniqueName: \"kubernetes.io/projected/1e7e627c-9513-4b99-ab07-618cd023ee0d-kube-api-access-wmx2n\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk\" (UID: \"1e7e627c-9513-4b99-ab07-618cd023ee0d\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:50:30.431849 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.431832    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1e7e627c-9513-4b99-ab07-618cd023ee0d-tokenizer-uds\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk\" (UID: \"1e7e627c-9513-4b99-ab07-618cd023ee0d\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:50:30.433839 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.433823    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e7e627c-9513-4b99-ab07-618cd023ee0d-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk\" (UID: \"1e7e627c-9513-4b99-ab07-618cd023ee0d\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:50:30.440953 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.440933    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmx2n\" (UniqueName: \"kubernetes.io/projected/1e7e627c-9513-4b99-ab07-618cd023ee0d-kube-api-access-wmx2n\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk\" (UID: \"1e7e627c-9513-4b99-ab07-618cd023ee0d\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:50:30.603568 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.603538    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:50:30.740813 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:30.740765    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"]
Jul 09 09:50:30.743828 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:50:30.743764    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e7e627c_9513_4b99_ab07_618cd023ee0d.slice/crio-648bbba30ecb118a6e72bfd168f5554daf55de3759338eeec001fccec1e56492 WatchSource:0}: Error finding container 648bbba30ecb118a6e72bfd168f5554daf55de3759338eeec001fccec1e56492: Status 404 returned error can't find the container with id 648bbba30ecb118a6e72bfd168f5554daf55de3759338eeec001fccec1e56492
Jul 09 09:50:31.703554 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:31.703522    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk" event={"ID":"1e7e627c-9513-4b99-ab07-618cd023ee0d","Type":"ContainerStarted","Data":"7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be"}
Jul 09 09:50:31.703983 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:31.703569    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk" event={"ID":"1e7e627c-9513-4b99-ab07-618cd023ee0d","Type":"ContainerStarted","Data":"648bbba30ecb118a6e72bfd168f5554daf55de3759338eeec001fccec1e56492"}
Jul 09 09:50:31.703983 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:31.703628    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:50:31.721424 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:50:31.721380    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk" podStartSLOduration=1.721362347 podStartE2EDuration="1.721362347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:50:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:50:31.720981219 +0000 UTC m=+1034.727150290" watchObservedRunningTime="2026-07-09 09:50:31.721362347 +0000 UTC m=+1034.727531440"
Jul 09 09:51:00.082605 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:00.082573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:51:00.480043 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:00.480007    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:51:00.579307 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:00.579279    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:51:02.710938 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:02.710907    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:51:20.641620 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:20.641584    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:51:20.866681 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:20.866645    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:51:26.692140 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:26.692082    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:51:57.096971 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:57.096921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:51:59.102642 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.102605    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"]
Jul 09 09:51:59.111548 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:51:59.111518    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 09:51:59.111548 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:51:59.111548 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:51:59.111548 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be" cmd=["/bin/sleep","15"]
Jul 09 09:51:59.111781 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.111562    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk" message="" err=<
Jul 09 09:51:59.111781 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:51:59.111781 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:51:59.111781 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 09:51:59.111781 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:51:59.111574    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 09:51:59.111781 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:51:59.111781 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:51:59.111781 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk" podUID="1e7e627c-9513-4b99-ab07-618cd023ee0d" containerName="main" containerID="cri-o://7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be"
Jul 09 09:51:59.111781 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.111613    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk" podUID="1e7e627c-9513-4b99-ab07-618cd023ee0d" containerName="main" containerID="cri-o://7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be" gracePeriod=60
Jul 09 09:51:59.121445 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.121417    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:51:59.220135 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.220105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:51:59.375357 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.375335    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:51:59.508056 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.508027    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmx2n\" (UniqueName: \"kubernetes.io/projected/1e7e627c-9513-4b99-ab07-618cd023ee0d-kube-api-access-wmx2n\") pod \"1e7e627c-9513-4b99-ab07-618cd023ee0d\" (UID: \"1e7e627c-9513-4b99-ab07-618cd023ee0d\") "
Jul 09 09:51:59.508201 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.508069    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1e7e627c-9513-4b99-ab07-618cd023ee0d-tokenizer-uds\") pod \"1e7e627c-9513-4b99-ab07-618cd023ee0d\" (UID: \"1e7e627c-9513-4b99-ab07-618cd023ee0d\") "
Jul 09 09:51:59.508201 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.508106    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e7e627c-9513-4b99-ab07-618cd023ee0d-tls-certs\") pod \"1e7e627c-9513-4b99-ab07-618cd023ee0d\" (UID: \"1e7e627c-9513-4b99-ab07-618cd023ee0d\") "
Jul 09 09:51:59.508385 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.508358    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e7e627c-9513-4b99-ab07-618cd023ee0d-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "1e7e627c-9513-4b99-ab07-618cd023ee0d" (UID: "1e7e627c-9513-4b99-ab07-618cd023ee0d"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:51:59.510096 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.510073    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e7e627c-9513-4b99-ab07-618cd023ee0d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1e7e627c-9513-4b99-ab07-618cd023ee0d" (UID: "1e7e627c-9513-4b99-ab07-618cd023ee0d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:51:59.510163 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.510139    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e7e627c-9513-4b99-ab07-618cd023ee0d-kube-api-access-wmx2n" (OuterVolumeSpecName: "kube-api-access-wmx2n") pod "1e7e627c-9513-4b99-ab07-618cd023ee0d" (UID: "1e7e627c-9513-4b99-ab07-618cd023ee0d"). InnerVolumeSpecName "kube-api-access-wmx2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:51:59.609201 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.609143    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e7e627c-9513-4b99-ab07-618cd023ee0d-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:51:59.609201 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.609166    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmx2n\" (UniqueName: \"kubernetes.io/projected/1e7e627c-9513-4b99-ab07-618cd023ee0d-kube-api-access-wmx2n\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:51:59.609201 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:51:59.609175    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1e7e627c-9513-4b99-ab07-618cd023ee0d-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:52:00.168969 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:00.168940    2577 generic.go:356] "Generic (PLEG): container finished" podID="1e7e627c-9513-4b99-ab07-618cd023ee0d" containerID="7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be" exitCode=0
Jul 09 09:52:00.169403 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:00.168979    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"
Jul 09 09:52:00.169403 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:00.169021    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk" event={"ID":"1e7e627c-9513-4b99-ab07-618cd023ee0d","Type":"ContainerDied","Data":"7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be"}
Jul 09 09:52:00.169403 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:00.169067    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk" event={"ID":"1e7e627c-9513-4b99-ab07-618cd023ee0d","Type":"ContainerDied","Data":"648bbba30ecb118a6e72bfd168f5554daf55de3759338eeec001fccec1e56492"}
Jul 09 09:52:00.169403 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:00.169090    2577 scope.go:117] "RemoveContainer" containerID="7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be"
Jul 09 09:52:00.178544 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:00.178526    2577 scope.go:117] "RemoveContainer" containerID="7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be"
Jul 09 09:52:00.178784 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:52:00.178764    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be\": container with ID starting with 7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be not found: ID does not exist" containerID="7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be"
Jul 09 09:52:00.178883 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:00.178791    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be"} err="failed to get container status \"7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be\": rpc error: code = NotFound desc = could not find container \"7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be\": container with ID starting with 7dd80ed499119009732627f7a45cfdbb8f208b7ea515129122cd12a6f4efe1be not found: ID does not exist"
Jul 09 09:52:00.187123 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:00.187101    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"]
Jul 09 09:52:00.193100 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:00.193078    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche8n9dk"]
Jul 09 09:52:01.552562 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:01.552532    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e7e627c-9513-4b99-ab07-618cd023ee0d" path="/var/lib/kubelet/pods/1e7e627c-9513-4b99-ab07-618cd023ee0d/volumes"
Jul 09 09:52:15.860732 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:15.860698    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:52:15.862334 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:15.862309    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"]
Jul 09 09:52:15.871625 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:52:15.871592    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 09:52:15.871625 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:52:15.871625 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:52:15.871625 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9" cmd=["/bin/sleep","15"]
Jul 09 09:52:15.871911 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:15.871643    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x" message="" err=<
Jul 09 09:52:15.871911 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:52:15.871911 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:52:15.871911 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 09:52:15.871911 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:52:15.871659    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 09:52:15.871911 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:52:15.871911 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:52:15.871911 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x" podUID="f0149903-58c4-45b8-84ca-df2d4f1745b1" containerName="main" containerID="cri-o://2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9"
Jul 09 09:52:15.871911 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:15.871713    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x" podUID="f0149903-58c4-45b8-84ca-df2d4f1745b1" containerName="main" containerID="cri-o://2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9" gracePeriod=60
Jul 09 09:52:15.924138 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:15.924084    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:52:16.123415 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.123354    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:52:16.239937 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.239910    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f0149903-58c4-45b8-84ca-df2d4f1745b1-tokenizer-uds\") pod \"f0149903-58c4-45b8-84ca-df2d4f1745b1\" (UID: \"f0149903-58c4-45b8-84ca-df2d4f1745b1\") "
Jul 09 09:52:16.240094 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.239952    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0149903-58c4-45b8-84ca-df2d4f1745b1-tls-certs\") pod \"f0149903-58c4-45b8-84ca-df2d4f1745b1\" (UID: \"f0149903-58c4-45b8-84ca-df2d4f1745b1\") "
Jul 09 09:52:16.240094 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.239987    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sz7b\" (UniqueName: \"kubernetes.io/projected/f0149903-58c4-45b8-84ca-df2d4f1745b1-kube-api-access-8sz7b\") pod \"f0149903-58c4-45b8-84ca-df2d4f1745b1\" (UID: \"f0149903-58c4-45b8-84ca-df2d4f1745b1\") "
Jul 09 09:52:16.240226 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.240161    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0149903-58c4-45b8-84ca-df2d4f1745b1-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "f0149903-58c4-45b8-84ca-df2d4f1745b1" (UID: "f0149903-58c4-45b8-84ca-df2d4f1745b1"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:52:16.241939 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.241919    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0149903-58c4-45b8-84ca-df2d4f1745b1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0149903-58c4-45b8-84ca-df2d4f1745b1" (UID: "f0149903-58c4-45b8-84ca-df2d4f1745b1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:52:16.242114 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.242091    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0149903-58c4-45b8-84ca-df2d4f1745b1-kube-api-access-8sz7b" (OuterVolumeSpecName: "kube-api-access-8sz7b") pod "f0149903-58c4-45b8-84ca-df2d4f1745b1" (UID: "f0149903-58c4-45b8-84ca-df2d4f1745b1"). InnerVolumeSpecName "kube-api-access-8sz7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:52:16.256533 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.256504    2577 generic.go:356] "Generic (PLEG): container finished" podID="f0149903-58c4-45b8-84ca-df2d4f1745b1" containerID="2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9" exitCode=0
Jul 09 09:52:16.256644 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.256539    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x" event={"ID":"f0149903-58c4-45b8-84ca-df2d4f1745b1","Type":"ContainerDied","Data":"2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9"}
Jul 09 09:52:16.256644 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.256549    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"
Jul 09 09:52:16.256644 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.256588    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x" event={"ID":"f0149903-58c4-45b8-84ca-df2d4f1745b1","Type":"ContainerDied","Data":"0050ebd3be621fe067cb4ea99adefe8d2e75febb99695c4ee6b139ab91ab4c04"}
Jul 09 09:52:16.256644 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.256617    2577 scope.go:117] "RemoveContainer" containerID="2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9"
Jul 09 09:52:16.265621 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.265601    2577 scope.go:117] "RemoveContainer" containerID="2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9"
Jul 09 09:52:16.265889 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:52:16.265867    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9\": container with ID starting with 2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9 not found: ID does not exist" containerID="2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9"
Jul 09 09:52:16.265970 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.265901    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9"} err="failed to get container status \"2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9\": rpc error: code = NotFound desc = could not find container \"2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9\": container with ID starting with 2c75c3bef5b3b549cacf551ee947b89dfb9bf6c5d6fd1bccaf6459abdf73bce9 not found: ID does not exist"
Jul 09 09:52:16.280403 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.280372    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"]
Jul 09 09:52:16.282326 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.282307    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schehd28x"]
Jul 09 09:52:16.340751 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.340725    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f0149903-58c4-45b8-84ca-df2d4f1745b1-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:52:16.340751 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.340746    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0149903-58c4-45b8-84ca-df2d4f1745b1-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:52:16.340751 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:16.340755    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sz7b\" (UniqueName: \"kubernetes.io/projected/f0149903-58c4-45b8-84ca-df2d4f1745b1-kube-api-access-8sz7b\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:52:17.552542 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:17.552509    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0149903-58c4-45b8-84ca-df2d4f1745b1" path="/var/lib/kubelet/pods/f0149903-58c4-45b8-84ca-df2d4f1745b1/volumes"
Jul 09 09:52:39.781406 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:39.781321    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:52:40.206036 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.205937    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"]
Jul 09 09:52:40.206408 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.206387    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0149903-58c4-45b8-84ca-df2d4f1745b1" containerName="main"
Jul 09 09:52:40.206514 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.206410    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0149903-58c4-45b8-84ca-df2d4f1745b1" containerName="main"
Jul 09 09:52:40.206514 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.206433    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e7e627c-9513-4b99-ab07-618cd023ee0d" containerName="main"
Jul 09 09:52:40.206514 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.206440    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e7e627c-9513-4b99-ab07-618cd023ee0d" containerName="main"
Jul 09 09:52:40.206514 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.206497    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0149903-58c4-45b8-84ca-df2d4f1745b1" containerName="main"
Jul 09 09:52:40.206514 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.206505    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e7e627c-9513-4b99-ab07-618cd023ee0d" containerName="main"
Jul 09 09:52:40.211038 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.211017    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:52:40.213579 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.213560    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 09:52:40.214504 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.214486    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 09:52:40.214618 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.214507    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 09:52:40.214618 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.214513    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs\""
Jul 09 09:52:40.214618 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.214549    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-router-managed-test-llm-4b931143-epp-sa-dockercfg-tsgm5\""
Jul 09 09:52:40.221555 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.221532    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"]
Jul 09 09:52:40.242073 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.242046    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:52:40.310295 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.310271    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mx64\" (UniqueName: \"kubernetes.io/projected/28a23720-5eda-4d30-9e53-c91e4e4efb2d-kube-api-access-6mx64\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv\" (UID: \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:52:40.310405 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.310311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/28a23720-5eda-4d30-9e53-c91e4e4efb2d-tokenizer-uds\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv\" (UID: \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:52:40.310405 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.310373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a23720-5eda-4d30-9e53-c91e4e4efb2d-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv\" (UID: \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:52:40.410865 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.410843    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mx64\" (UniqueName: \"kubernetes.io/projected/28a23720-5eda-4d30-9e53-c91e4e4efb2d-kube-api-access-6mx64\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv\" (UID: \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:52:40.410971 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.410875    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/28a23720-5eda-4d30-9e53-c91e4e4efb2d-tokenizer-uds\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv\" (UID: \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:52:40.410971 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.410922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a23720-5eda-4d30-9e53-c91e4e4efb2d-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv\" (UID: \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:52:40.411337 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.411317    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/28a23720-5eda-4d30-9e53-c91e4e4efb2d-tokenizer-uds\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv\" (UID: \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:52:40.413590 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.413565    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a23720-5eda-4d30-9e53-c91e4e4efb2d-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv\" (UID: \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:52:40.420655 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.420625    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mx64\" (UniqueName: \"kubernetes.io/projected/28a23720-5eda-4d30-9e53-c91e4e4efb2d-kube-api-access-6mx64\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv\" (UID: \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:52:40.521790 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.521769    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:52:40.660703 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:40.660681    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"]
Jul 09 09:52:40.662591 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:52:40.662562    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a23720_5eda_4d30_9e53_c91e4e4efb2d.slice/crio-f9a11213fbfd14e003eb08e214068588a88e5ac489685d49e44d76b9bd7c48f0 WatchSource:0}: Error finding container f9a11213fbfd14e003eb08e214068588a88e5ac489685d49e44d76b9bd7c48f0: Status 404 returned error can't find the container with id f9a11213fbfd14e003eb08e214068588a88e5ac489685d49e44d76b9bd7c48f0
Jul 09 09:52:41.390928 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:41.390895    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv" event={"ID":"28a23720-5eda-4d30-9e53-c91e4e4efb2d","Type":"ContainerStarted","Data":"103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd"}
Jul 09 09:52:41.391344 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:41.390942    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv" event={"ID":"28a23720-5eda-4d30-9e53-c91e4e4efb2d","Type":"ContainerStarted","Data":"f9a11213fbfd14e003eb08e214068588a88e5ac489685d49e44d76b9bd7c48f0"}
Jul 09 09:52:41.391344 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:41.391038    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:52:41.411168 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:41.411120    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv" podStartSLOduration=1.411107408 podStartE2EDuration="1.411107408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:52:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:52:41.408767545 +0000 UTC m=+1164.414936617" watchObservedRunningTime="2026-07-09 09:52:41.411107408 +0000 UTC m=+1164.417276478"
Jul 09 09:52:48.066692 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:52:48.066652    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:53:12.398917 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:53:12.398885    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:53:17.417815 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:53:17.417777    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:53:17.418292 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:53:17.418255    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:53:22.707736 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:53:22.707698    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:53:22.775419 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:53:22.775378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:53:39.558212 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:53:39.558178    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:53:45.723670 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:53:45.723637    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:53:46.144532 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:53:46.144450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:54:04.522083 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:04.522044    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:54:04.811126 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:04.811049    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"]
Jul 09 09:54:04.814829 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:04.814784    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:04.817940 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:04.817919    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"auth-disabled-test-epp-sa-dockercfg-d9bgc\""
Jul 09 09:54:04.818024 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:04.817919    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"auth-disabled-test-kserve-self-signed-certs\""
Jul 09 09:54:04.818024 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:04.817921    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 09:54:04.825380 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:04.825355    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"]
Jul 09 09:54:04.926062 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:04.926033    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09358ed2-25de-47bf-b9e5-3f330bdddf2d-kserve-provision-location\") pod \"auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:04.926172 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:04.926080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09358ed2-25de-47bf-b9e5-3f330bdddf2d-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:04.926172 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:04.926096    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/09358ed2-25de-47bf-b9e5-3f330bdddf2d-tokenizer-uds\") pod \"auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:04.926276 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:04.926179    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptf2c\" (UniqueName: \"kubernetes.io/projected/09358ed2-25de-47bf-b9e5-3f330bdddf2d-kube-api-access-ptf2c\") pod \"auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:04.989533 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:04.989502    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:54:05.026964 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:05.026941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09358ed2-25de-47bf-b9e5-3f330bdddf2d-kserve-provision-location\") pod \"auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:05.027092 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:05.026990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09358ed2-25de-47bf-b9e5-3f330bdddf2d-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:05.027158 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:05.027121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/09358ed2-25de-47bf-b9e5-3f330bdddf2d-tokenizer-uds\") pod \"auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:05.027227 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:05.027208    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptf2c\" (UniqueName: \"kubernetes.io/projected/09358ed2-25de-47bf-b9e5-3f330bdddf2d-kube-api-access-ptf2c\") pod \"auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:05.027366 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:05.027344    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09358ed2-25de-47bf-b9e5-3f330bdddf2d-kserve-provision-location\") pod \"auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:05.027456 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:05.027415    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/09358ed2-25de-47bf-b9e5-3f330bdddf2d-tokenizer-uds\") pod \"auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:05.029572 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:05.029552    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09358ed2-25de-47bf-b9e5-3f330bdddf2d-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:05.035721 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:05.035698    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptf2c\" (UniqueName: \"kubernetes.io/projected/09358ed2-25de-47bf-b9e5-3f330bdddf2d-kube-api-access-ptf2c\") pod \"auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:05.124850 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:05.124775    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:05.265946 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:05.265924    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"]
Jul 09 09:54:05.267831 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:54:05.267790    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09358ed2_25de_47bf_b9e5_3f330bdddf2d.slice/crio-38c0d1549c67aa67faa1664ffef71ba5cdd028a4503ee875ea71ad07f14ebeb8 WatchSource:0}: Error finding container 38c0d1549c67aa67faa1664ffef71ba5cdd028a4503ee875ea71ad07f14ebeb8: Status 404 returned error can't find the container with id 38c0d1549c67aa67faa1664ffef71ba5cdd028a4503ee875ea71ad07f14ebeb8
Jul 09 09:54:05.843304 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:05.843249    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh" event={"ID":"09358ed2-25de-47bf-b9e5-3f330bdddf2d","Type":"ContainerStarted","Data":"c5da21a62be2b842a922118dea298e4ee5191e23fc78a4efe293f4da0a6f7dc1"}
Jul 09 09:54:05.843738 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:05.843358    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh" event={"ID":"09358ed2-25de-47bf-b9e5-3f330bdddf2d","Type":"ContainerStarted","Data":"38c0d1549c67aa67faa1664ffef71ba5cdd028a4503ee875ea71ad07f14ebeb8"}
Jul 09 09:54:06.850135 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:06.850099    2577 generic.go:356] "Generic (PLEG): container finished" podID="09358ed2-25de-47bf-b9e5-3f330bdddf2d" containerID="c5da21a62be2b842a922118dea298e4ee5191e23fc78a4efe293f4da0a6f7dc1" exitCode=0
Jul 09 09:54:06.850493 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:06.850187    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh" event={"ID":"09358ed2-25de-47bf-b9e5-3f330bdddf2d","Type":"ContainerDied","Data":"c5da21a62be2b842a922118dea298e4ee5191e23fc78a4efe293f4da0a6f7dc1"}
Jul 09 09:54:07.856470 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:07.856438    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh" event={"ID":"09358ed2-25de-47bf-b9e5-3f330bdddf2d","Type":"ContainerStarted","Data":"532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4"}
Jul 09 09:54:07.856860 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:07.856566    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:07.881393 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:07.881349    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh" podStartSLOduration=3.237493981 podStartE2EDuration="3.881333135s" totalImagesPullingTime="0s" totalInitContainerRuntime="643.839154ms" isStatefulPod=false podCreationTimestamp="2026-07-09 09:54:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:07.878405157 +0000 UTC m=+1250.884574255" watchObservedRunningTime="2026-07-09 09:54:07.881333135 +0000 UTC m=+1250.887502205"
Jul 09 09:54:15.514392 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:15.514359    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:54:38.864967 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:38.864938    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:54:44.830442 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:44.830393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:54:48.601003 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:48.600971    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:54:48.698295 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:54:48.698266    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:55:08.708782 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:08.708733    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"]
Jul 09 09:55:08.718400 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:55:08.718367    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 09:55:08.718400 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:55:08.718400 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:55:08.718400 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd" cmd=["/bin/sleep","15"]
Jul 09 09:55:08.718710 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:08.718413    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv" message="" err=<
Jul 09 09:55:08.718710 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:55:08.718710 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:55:08.718710 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 09:55:08.718710 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:55:08.718424    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 09:55:08.718710 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:55:08.718710 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:55:08.718710 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv" podUID="28a23720-5eda-4d30-9e53-c91e4e4efb2d" containerName="main" containerID="cri-o://103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd"
Jul 09 09:55:08.718710 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:08.718465    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv" podUID="28a23720-5eda-4d30-9e53-c91e4e4efb2d" containerName="main" containerID="cri-o://103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd" gracePeriod=60
Jul 09 09:55:08.963285 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:08.963237    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:55:08.978738 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:08.978720    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:55:09.023108 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.023081    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:55:09.116668 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.116643    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a23720-5eda-4d30-9e53-c91e4e4efb2d-tls-certs\") pod \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\" (UID: \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\") "
Jul 09 09:55:09.116846 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.116708    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/28a23720-5eda-4d30-9e53-c91e4e4efb2d-tokenizer-uds\") pod \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\" (UID: \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\") "
Jul 09 09:55:09.116846 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.116742    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mx64\" (UniqueName: \"kubernetes.io/projected/28a23720-5eda-4d30-9e53-c91e4e4efb2d-kube-api-access-6mx64\") pod \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\" (UID: \"28a23720-5eda-4d30-9e53-c91e4e4efb2d\") "
Jul 09 09:55:09.117015 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.116989    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a23720-5eda-4d30-9e53-c91e4e4efb2d-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "28a23720-5eda-4d30-9e53-c91e4e4efb2d" (UID: "28a23720-5eda-4d30-9e53-c91e4e4efb2d"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:55:09.118746 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.118717    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28a23720-5eda-4d30-9e53-c91e4e4efb2d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "28a23720-5eda-4d30-9e53-c91e4e4efb2d" (UID: "28a23720-5eda-4d30-9e53-c91e4e4efb2d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:55:09.118882 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.118779    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28a23720-5eda-4d30-9e53-c91e4e4efb2d-kube-api-access-6mx64" (OuterVolumeSpecName: "kube-api-access-6mx64") pod "28a23720-5eda-4d30-9e53-c91e4e4efb2d" (UID: "28a23720-5eda-4d30-9e53-c91e4e4efb2d"). InnerVolumeSpecName "kube-api-access-6mx64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:55:09.186341 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.186310    2577 generic.go:356] "Generic (PLEG): container finished" podID="28a23720-5eda-4d30-9e53-c91e4e4efb2d" containerID="103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd" exitCode=0
Jul 09 09:55:09.186462 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.186358    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"
Jul 09 09:55:09.186462 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.186394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv" event={"ID":"28a23720-5eda-4d30-9e53-c91e4e4efb2d","Type":"ContainerDied","Data":"103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd"}
Jul 09 09:55:09.186462 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.186444    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv" event={"ID":"28a23720-5eda-4d30-9e53-c91e4e4efb2d","Type":"ContainerDied","Data":"f9a11213fbfd14e003eb08e214068588a88e5ac489685d49e44d76b9bd7c48f0"}
Jul 09 09:55:09.186571 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.186467    2577 scope.go:117] "RemoveContainer" containerID="103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd"
Jul 09 09:55:09.195703 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.195685    2577 scope.go:117] "RemoveContainer" containerID="103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd"
Jul 09 09:55:09.195975 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:55:09.195959    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd\": container with ID starting with 103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd not found: ID does not exist" containerID="103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd"
Jul 09 09:55:09.196033 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.195984    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd"} err="failed to get container status \"103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd\": rpc error: code = NotFound desc = could not find container \"103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd\": container with ID starting with 103e1b31e52eae3533b0969d3a748b43b14e27ea0bba0ae680819dd10e23bccd not found: ID does not exist"
Jul 09 09:55:09.208137 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.208111    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"]
Jul 09 09:55:09.212344 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.212321    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesztcv"]
Jul 09 09:55:09.218250 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.218209    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a23720-5eda-4d30-9e53-c91e4e4efb2d-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:55:09.218250 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.218228    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/28a23720-5eda-4d30-9e53-c91e4e4efb2d-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:55:09.218250 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.218237    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mx64\" (UniqueName: \"kubernetes.io/projected/28a23720-5eda-4d30-9e53-c91e4e4efb2d-kube-api-access-6mx64\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:55:09.552898 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:09.552823    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28a23720-5eda-4d30-9e53-c91e4e4efb2d" path="/var/lib/kubelet/pods/28a23720-5eda-4d30-9e53-c91e4e4efb2d/volumes"
Jul 09 09:55:43.215109 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.215040    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"]
Jul 09 09:55:43.215551 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.215414    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a23720-5eda-4d30-9e53-c91e4e4efb2d" containerName="main"
Jul 09 09:55:43.215551 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.215425    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a23720-5eda-4d30-9e53-c91e4e4efb2d" containerName="main"
Jul 09 09:55:43.215551 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.215494    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a23720-5eda-4d30-9e53-c91e4e4efb2d" containerName="main"
Jul 09 09:55:43.219895 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.219872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.222446 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.222427    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-with-7ca60146-epp-sa-dockercfg-k96cx\""
Jul 09 09:55:43.222583 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.222446    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs\""
Jul 09 09:55:43.229331 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.229311    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"]
Jul 09 09:55:43.368997 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.368965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36575c97-85f5-44a7-a4d2-90b9682ae874-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.369148 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.369064    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/36575c97-85f5-44a7-a4d2-90b9682ae874-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.369148 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.369103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/36575c97-85f5-44a7-a4d2-90b9682ae874-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.369148 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.369157    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsp5w\" (UniqueName: \"kubernetes.io/projected/36575c97-85f5-44a7-a4d2-90b9682ae874-kube-api-access-vsp5w\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.469807 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.469730    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/36575c97-85f5-44a7-a4d2-90b9682ae874-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.469807 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.469768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsp5w\" (UniqueName: \"kubernetes.io/projected/36575c97-85f5-44a7-a4d2-90b9682ae874-kube-api-access-vsp5w\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.470000 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.469907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36575c97-85f5-44a7-a4d2-90b9682ae874-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.470059 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.470018    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/36575c97-85f5-44a7-a4d2-90b9682ae874-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.470130 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.470108    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/36575c97-85f5-44a7-a4d2-90b9682ae874-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.470313 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.470294    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/36575c97-85f5-44a7-a4d2-90b9682ae874-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.472398 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.472378    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36575c97-85f5-44a7-a4d2-90b9682ae874-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.479556 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.479532    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsp5w\" (UniqueName: \"kubernetes.io/projected/36575c97-85f5-44a7-a4d2-90b9682ae874-kube-api-access-vsp5w\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.530334 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.530312    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:43.548123 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.548072    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:55:43.602945 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.602896    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:55:43.667899 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:43.667871    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"]
Jul 09 09:55:43.669371 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:55:43.669347    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36575c97_85f5_44a7_a4d2_90b9682ae874.slice/crio-b17184d37723f3bcf0fdf455f097e6990f1898035a2c29c8491840ad4faf5030 WatchSource:0}: Error finding container b17184d37723f3bcf0fdf455f097e6990f1898035a2c29c8491840ad4faf5030: Status 404 returned error can't find the container with id b17184d37723f3bcf0fdf455f097e6990f1898035a2c29c8491840ad4faf5030
Jul 09 09:55:44.371388 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:44.371342    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5" event={"ID":"36575c97-85f5-44a7-a4d2-90b9682ae874","Type":"ContainerStarted","Data":"6eb6e99531506052917979b259c4f219870649721a63d57e960404bfdf3f86ff"}
Jul 09 09:55:44.371737 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:44.371403    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5" event={"ID":"36575c97-85f5-44a7-a4d2-90b9682ae874","Type":"ContainerStarted","Data":"b17184d37723f3bcf0fdf455f097e6990f1898035a2c29c8491840ad4faf5030"}
Jul 09 09:55:45.377812 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:45.377762    2577 generic.go:356] "Generic (PLEG): container finished" podID="36575c97-85f5-44a7-a4d2-90b9682ae874" containerID="6eb6e99531506052917979b259c4f219870649721a63d57e960404bfdf3f86ff" exitCode=0
Jul 09 09:55:45.378248 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:45.377847    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5" event={"ID":"36575c97-85f5-44a7-a4d2-90b9682ae874","Type":"ContainerDied","Data":"6eb6e99531506052917979b259c4f219870649721a63d57e960404bfdf3f86ff"}
Jul 09 09:55:46.384764 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:46.384723    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5" event={"ID":"36575c97-85f5-44a7-a4d2-90b9682ae874","Type":"ContainerStarted","Data":"5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b"}
Jul 09 09:55:46.385173 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:46.384841    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:55:46.406210 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:46.406161    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5" podStartSLOduration=2.756146566 podStartE2EDuration="3.406149006s" totalImagesPullingTime="0s" totalInitContainerRuntime="650.00244ms" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:46.40466705 +0000 UTC m=+1349.410836130" watchObservedRunningTime="2026-07-09 09:55:46.406149006 +0000 UTC m=+1349.412318076"
Jul 09 09:55:53.912435 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:55:53.912397    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:56:09.998267 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:09.998233    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"]
Jul 09 09:56:10.007084 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.007054    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:56:10.008966 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:56:10.008937    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 09:56:10.008966 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:56:10.008966 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:56:10.008966 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4" cmd=["/bin/sleep","15"]
Jul 09 09:56:10.009254 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.008980    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh" message="" err=<
Jul 09 09:56:10.009254 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:56:10.009254 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:56:10.009254 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 09:56:10.009254 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:56:10.008993    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 09:56:10.009254 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:56:10.009254 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:56:10.009254 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh" podUID="09358ed2-25de-47bf-b9e5-3f330bdddf2d" containerName="main" containerID="cri-o://532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4"
Jul 09 09:56:10.009254 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.009035    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh" podUID="09358ed2-25de-47bf-b9e5-3f330bdddf2d" containerName="main" containerID="cri-o://532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4" gracePeriod=60
Jul 09 09:56:10.272869 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.272844    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:56:10.379860 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.379827    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09358ed2-25de-47bf-b9e5-3f330bdddf2d-kserve-provision-location\") pod \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") "
Jul 09 09:56:10.380035 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.379892    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/09358ed2-25de-47bf-b9e5-3f330bdddf2d-tokenizer-uds\") pod \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") "
Jul 09 09:56:10.380035 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.379918    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptf2c\" (UniqueName: \"kubernetes.io/projected/09358ed2-25de-47bf-b9e5-3f330bdddf2d-kube-api-access-ptf2c\") pod \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") "
Jul 09 09:56:10.380035 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.379944    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09358ed2-25de-47bf-b9e5-3f330bdddf2d-tls-certs\") pod \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\" (UID: \"09358ed2-25de-47bf-b9e5-3f330bdddf2d\") "
Jul 09 09:56:10.380222 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.380140    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09358ed2-25de-47bf-b9e5-3f330bdddf2d-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "09358ed2-25de-47bf-b9e5-3f330bdddf2d" (UID: "09358ed2-25de-47bf-b9e5-3f330bdddf2d"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:56:10.380541 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.380509    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09358ed2-25de-47bf-b9e5-3f330bdddf2d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "09358ed2-25de-47bf-b9e5-3f330bdddf2d" (UID: "09358ed2-25de-47bf-b9e5-3f330bdddf2d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:56:10.381985 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.381958    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09358ed2-25de-47bf-b9e5-3f330bdddf2d-kube-api-access-ptf2c" (OuterVolumeSpecName: "kube-api-access-ptf2c") pod "09358ed2-25de-47bf-b9e5-3f330bdddf2d" (UID: "09358ed2-25de-47bf-b9e5-3f330bdddf2d"). InnerVolumeSpecName "kube-api-access-ptf2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:56:10.382079 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.382001    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09358ed2-25de-47bf-b9e5-3f330bdddf2d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09358ed2-25de-47bf-b9e5-3f330bdddf2d" (UID: "09358ed2-25de-47bf-b9e5-3f330bdddf2d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:56:10.399591 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.399568    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:56:10.480585 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.480563    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09358ed2-25de-47bf-b9e5-3f330bdddf2d-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:56:10.480585 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.480584    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/09358ed2-25de-47bf-b9e5-3f330bdddf2d-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:56:10.480709 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.480596    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptf2c\" (UniqueName: \"kubernetes.io/projected/09358ed2-25de-47bf-b9e5-3f330bdddf2d-kube-api-access-ptf2c\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:56:10.480709 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.480615    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09358ed2-25de-47bf-b9e5-3f330bdddf2d-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:56:10.517834 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.517761    2577 generic.go:356] "Generic (PLEG): container finished" podID="09358ed2-25de-47bf-b9e5-3f330bdddf2d" containerID="532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4" exitCode=0
Jul 09 09:56:10.517834 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.517821    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"
Jul 09 09:56:10.517964 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.517838    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh" event={"ID":"09358ed2-25de-47bf-b9e5-3f330bdddf2d","Type":"ContainerDied","Data":"532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4"}
Jul 09 09:56:10.517964 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.517891    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh" event={"ID":"09358ed2-25de-47bf-b9e5-3f330bdddf2d","Type":"ContainerDied","Data":"38c0d1549c67aa67faa1664ffef71ba5cdd028a4503ee875ea71ad07f14ebeb8"}
Jul 09 09:56:10.517964 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.517915    2577 scope.go:117] "RemoveContainer" containerID="532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4"
Jul 09 09:56:10.527763 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.527748    2577 scope.go:117] "RemoveContainer" containerID="c5da21a62be2b842a922118dea298e4ee5191e23fc78a4efe293f4da0a6f7dc1"
Jul 09 09:56:10.536065 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.536044    2577 scope.go:117] "RemoveContainer" containerID="532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4"
Jul 09 09:56:10.536332 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:56:10.536315    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4\": container with ID starting with 532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4 not found: ID does not exist" containerID="532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4"
Jul 09 09:56:10.536402 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.536338    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4"} err="failed to get container status \"532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4\": rpc error: code = NotFound desc = could not find container \"532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4\": container with ID starting with 532a6a76ce5ce8b7f898d4f39648048fcb094faa9b21424cc6a852b3cf10fdd4 not found: ID does not exist"
Jul 09 09:56:10.536402 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.536355    2577 scope.go:117] "RemoveContainer" containerID="c5da21a62be2b842a922118dea298e4ee5191e23fc78a4efe293f4da0a6f7dc1"
Jul 09 09:56:10.536562 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:56:10.536548    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5da21a62be2b842a922118dea298e4ee5191e23fc78a4efe293f4da0a6f7dc1\": container with ID starting with c5da21a62be2b842a922118dea298e4ee5191e23fc78a4efe293f4da0a6f7dc1 not found: ID does not exist" containerID="c5da21a62be2b842a922118dea298e4ee5191e23fc78a4efe293f4da0a6f7dc1"
Jul 09 09:56:10.536603 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.536566    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5da21a62be2b842a922118dea298e4ee5191e23fc78a4efe293f4da0a6f7dc1"} err="failed to get container status \"c5da21a62be2b842a922118dea298e4ee5191e23fc78a4efe293f4da0a6f7dc1\": rpc error: code = NotFound desc = could not find container \"c5da21a62be2b842a922118dea298e4ee5191e23fc78a4efe293f4da0a6f7dc1\": container with ID starting with c5da21a62be2b842a922118dea298e4ee5191e23fc78a4efe293f4da0a6f7dc1 not found: ID does not exist"
Jul 09 09:56:10.545596 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.545573    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"]
Jul 09 09:56:10.547862 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:10.547835    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-866757c5f5-xgvsh"]
Jul 09 09:56:11.552654 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:11.552617    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09358ed2-25de-47bf-b9e5-3f330bdddf2d" path="/var/lib/kubelet/pods/09358ed2-25de-47bf-b9e5-3f330bdddf2d/volumes"
Jul 09 09:56:17.391965 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:17.391931    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:56:44.022467 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:56:44.022428    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:57:20.084147 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:20.084107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:57:27.822976 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:27.822938    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:57:27.823591 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:27.823245    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"]
Jul 09 09:57:27.833149 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:57:27.833119    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 09:57:27.833149 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:57:27.833149 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:57:27.833149 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b" cmd=["/bin/sleep","15"]
Jul 09 09:57:27.833493 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:27.833165    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5" message="" err=<
Jul 09 09:57:27.833493 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:57:27.833493 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:57:27.833493 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 09:57:27.833493 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:57:27.833179    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 09:57:27.833493 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 09:57:27.833493 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 09:57:27.833493 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5" podUID="36575c97-85f5-44a7-a4d2-90b9682ae874" containerName="main" containerID="cri-o://5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b"
Jul 09 09:57:27.833493 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:27.833227    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5" podUID="36575c97-85f5-44a7-a4d2-90b9682ae874" containerName="main" containerID="cri-o://5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b" gracePeriod=60
Jul 09 09:57:27.884410 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:27.884384    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:57:28.089780 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.089757    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:57:28.190312 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.190273    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36575c97-85f5-44a7-a4d2-90b9682ae874-tls-certs\") pod \"36575c97-85f5-44a7-a4d2-90b9682ae874\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") "
Jul 09 09:57:28.190312 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.190304    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsp5w\" (UniqueName: \"kubernetes.io/projected/36575c97-85f5-44a7-a4d2-90b9682ae874-kube-api-access-vsp5w\") pod \"36575c97-85f5-44a7-a4d2-90b9682ae874\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") "
Jul 09 09:57:28.190516 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.190351    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/36575c97-85f5-44a7-a4d2-90b9682ae874-kserve-provision-location\") pod \"36575c97-85f5-44a7-a4d2-90b9682ae874\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") "
Jul 09 09:57:28.190516 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.190386    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/36575c97-85f5-44a7-a4d2-90b9682ae874-tokenizer-uds\") pod \"36575c97-85f5-44a7-a4d2-90b9682ae874\" (UID: \"36575c97-85f5-44a7-a4d2-90b9682ae874\") "
Jul 09 09:57:28.190684 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.190647    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36575c97-85f5-44a7-a4d2-90b9682ae874-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "36575c97-85f5-44a7-a4d2-90b9682ae874" (UID: "36575c97-85f5-44a7-a4d2-90b9682ae874"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:57:28.191091 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.191069    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36575c97-85f5-44a7-a4d2-90b9682ae874-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "36575c97-85f5-44a7-a4d2-90b9682ae874" (UID: "36575c97-85f5-44a7-a4d2-90b9682ae874"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:57:28.192248 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.192224    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36575c97-85f5-44a7-a4d2-90b9682ae874-kube-api-access-vsp5w" (OuterVolumeSpecName: "kube-api-access-vsp5w") pod "36575c97-85f5-44a7-a4d2-90b9682ae874" (UID: "36575c97-85f5-44a7-a4d2-90b9682ae874"). InnerVolumeSpecName "kube-api-access-vsp5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:57:28.192400 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.192379    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36575c97-85f5-44a7-a4d2-90b9682ae874-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36575c97-85f5-44a7-a4d2-90b9682ae874" (UID: "36575c97-85f5-44a7-a4d2-90b9682ae874"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:57:28.291278 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.291245    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36575c97-85f5-44a7-a4d2-90b9682ae874-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:57:28.291278 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.291275    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsp5w\" (UniqueName: \"kubernetes.io/projected/36575c97-85f5-44a7-a4d2-90b9682ae874-kube-api-access-vsp5w\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:57:28.291416 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.291285    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/36575c97-85f5-44a7-a4d2-90b9682ae874-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:57:28.291416 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.291295    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/36575c97-85f5-44a7-a4d2-90b9682ae874-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 09:57:28.949732 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.949690    2577 generic.go:356] "Generic (PLEG): container finished" podID="36575c97-85f5-44a7-a4d2-90b9682ae874" containerID="5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b" exitCode=0
Jul 09 09:57:28.950213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.949747    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"
Jul 09 09:57:28.950213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.949762    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5" event={"ID":"36575c97-85f5-44a7-a4d2-90b9682ae874","Type":"ContainerDied","Data":"5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b"}
Jul 09 09:57:28.950213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.949838    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5" event={"ID":"36575c97-85f5-44a7-a4d2-90b9682ae874","Type":"ContainerDied","Data":"b17184d37723f3bcf0fdf455f097e6990f1898035a2c29c8491840ad4faf5030"}
Jul 09 09:57:28.950213 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.949869    2577 scope.go:117] "RemoveContainer" containerID="5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b"
Jul 09 09:57:28.959357 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.959342    2577 scope.go:117] "RemoveContainer" containerID="6eb6e99531506052917979b259c4f219870649721a63d57e960404bfdf3f86ff"
Jul 09 09:57:28.967597 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.967582    2577 scope.go:117] "RemoveContainer" containerID="5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b"
Jul 09 09:57:28.967858 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:57:28.967839    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b\": container with ID starting with 5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b not found: ID does not exist" containerID="5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b"
Jul 09 09:57:28.967914 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.967867    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b"} err="failed to get container status \"5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b\": rpc error: code = NotFound desc = could not find container \"5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b\": container with ID starting with 5d90e1ccae7fca904bcd2b3b38ff647ac1fcdfac937f473cb9127ca35eca666b not found: ID does not exist"
Jul 09 09:57:28.967914 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.967884    2577 scope.go:117] "RemoveContainer" containerID="6eb6e99531506052917979b259c4f219870649721a63d57e960404bfdf3f86ff"
Jul 09 09:57:28.968093 ip-10-0-132-105 kubenswrapper[2577]: E0709 09:57:28.968077    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eb6e99531506052917979b259c4f219870649721a63d57e960404bfdf3f86ff\": container with ID starting with 6eb6e99531506052917979b259c4f219870649721a63d57e960404bfdf3f86ff not found: ID does not exist" containerID="6eb6e99531506052917979b259c4f219870649721a63d57e960404bfdf3f86ff"
Jul 09 09:57:28.968133 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.968099    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb6e99531506052917979b259c4f219870649721a63d57e960404bfdf3f86ff"} err="failed to get container status \"6eb6e99531506052917979b259c4f219870649721a63d57e960404bfdf3f86ff\": rpc error: code = NotFound desc = could not find container \"6eb6e99531506052917979b259c4f219870649721a63d57e960404bfdf3f86ff\": container with ID starting with 6eb6e99531506052917979b259c4f219870649721a63d57e960404bfdf3f86ff not found: ID does not exist"
Jul 09 09:57:28.976527 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.976507    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"]
Jul 09 09:57:28.983137 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:28.983115    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schemzdr5"]
Jul 09 09:57:29.551973 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:29.551941    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36575c97-85f5-44a7-a4d2-90b9682ae874" path="/var/lib/kubelet/pods/36575c97-85f5-44a7-a4d2-90b9682ae874/volumes"
Jul 09 09:57:53.519774 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.519742    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"]
Jul 09 09:57:53.520328 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.520306    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09358ed2-25de-47bf-b9e5-3f330bdddf2d" containerName="main"
Jul 09 09:57:53.520328 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.520325    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="09358ed2-25de-47bf-b9e5-3f330bdddf2d" containerName="main"
Jul 09 09:57:53.520451 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.520340    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36575c97-85f5-44a7-a4d2-90b9682ae874" containerName="main"
Jul 09 09:57:53.520451 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.520348    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="36575c97-85f5-44a7-a4d2-90b9682ae874" containerName="main"
Jul 09 09:57:53.520451 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.520378    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09358ed2-25de-47bf-b9e5-3f330bdddf2d" containerName="storage-initializer"
Jul 09 09:57:53.520451 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.520388    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="09358ed2-25de-47bf-b9e5-3f330bdddf2d" containerName="storage-initializer"
Jul 09 09:57:53.520451 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.520399    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36575c97-85f5-44a7-a4d2-90b9682ae874" containerName="storage-initializer"
Jul 09 09:57:53.520451 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.520407    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="36575c97-85f5-44a7-a4d2-90b9682ae874" containerName="storage-initializer"
Jul 09 09:57:53.520739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.520491    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="36575c97-85f5-44a7-a4d2-90b9682ae874" containerName="main"
Jul 09 09:57:53.520739 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.520508    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="09358ed2-25de-47bf-b9e5-3f330bdddf2d" containerName="main"
Jul 09 09:57:53.523994 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.523971    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.530902 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.529080    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 09:57:53.530902 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.530152    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs\""
Jul 09 09:57:53.532831 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.531296    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-with-ba4d693a-epp-sa-dockercfg-s9hs7\""
Jul 09 09:57:53.532831 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.531535    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 09:57:53.532831 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.531831    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 09:57:53.532831 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.532395    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 09:57:53.539783 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.539747    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"]
Jul 09 09:57:53.573006 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.572981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/af315a7e-ac57-4b8b-be22-b399963fbcb8-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.573135 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.573027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx6wd\" (UniqueName: \"kubernetes.io/projected/af315a7e-ac57-4b8b-be22-b399963fbcb8-kube-api-access-hx6wd\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.573135 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.573096    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/af315a7e-ac57-4b8b-be22-b399963fbcb8-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.573240 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.573137    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af315a7e-ac57-4b8b-be22-b399963fbcb8-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.673885 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.673859    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/af315a7e-ac57-4b8b-be22-b399963fbcb8-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.674012 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.673891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af315a7e-ac57-4b8b-be22-b399963fbcb8-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.674101 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.674075    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/af315a7e-ac57-4b8b-be22-b399963fbcb8-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.674204 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.674184    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx6wd\" (UniqueName: \"kubernetes.io/projected/af315a7e-ac57-4b8b-be22-b399963fbcb8-kube-api-access-hx6wd\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.674300 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.674282    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/af315a7e-ac57-4b8b-be22-b399963fbcb8-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.674384 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.674368    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/af315a7e-ac57-4b8b-be22-b399963fbcb8-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.676335 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.676316    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af315a7e-ac57-4b8b-be22-b399963fbcb8-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.683734 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.683710    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx6wd\" (UniqueName: \"kubernetes.io/projected/af315a7e-ac57-4b8b-be22-b399963fbcb8-kube-api-access-hx6wd\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.839781 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.839722    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:53.974164 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:53.974138    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"]
Jul 09 09:57:53.975988 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:57:53.975955    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf315a7e_ac57_4b8b_be22_b399963fbcb8.slice/crio-a804d3caceda66786dc40d95a86e883c13e5056b34e2827a908444777a220fba WatchSource:0}: Error finding container a804d3caceda66786dc40d95a86e883c13e5056b34e2827a908444777a220fba: Status 404 returned error can't find the container with id a804d3caceda66786dc40d95a86e883c13e5056b34e2827a908444777a220fba
Jul 09 09:57:54.080519 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:54.080489    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw" event={"ID":"af315a7e-ac57-4b8b-be22-b399963fbcb8","Type":"ContainerStarted","Data":"306637aa0d4bacdb19e677bfbe985b49285052498a1a31987b80c440384dc688"}
Jul 09 09:57:54.080639 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:54.080540    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw" event={"ID":"af315a7e-ac57-4b8b-be22-b399963fbcb8","Type":"ContainerStarted","Data":"a804d3caceda66786dc40d95a86e883c13e5056b34e2827a908444777a220fba"}
Jul 09 09:57:55.088367 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:55.088332    2577 generic.go:356] "Generic (PLEG): container finished" podID="af315a7e-ac57-4b8b-be22-b399963fbcb8" containerID="306637aa0d4bacdb19e677bfbe985b49285052498a1a31987b80c440384dc688" exitCode=0
Jul 09 09:57:55.088827 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:55.088423    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw" event={"ID":"af315a7e-ac57-4b8b-be22-b399963fbcb8","Type":"ContainerDied","Data":"306637aa0d4bacdb19e677bfbe985b49285052498a1a31987b80c440384dc688"}
Jul 09 09:57:55.245440 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:55.245412    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:57:55.310648 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:55.310610    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:57:56.095694 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:56.095659    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw" event={"ID":"af315a7e-ac57-4b8b-be22-b399963fbcb8","Type":"ContainerStarted","Data":"2e16164b79f9f7c69647c0d70969da6560886813c63e286dfd974987291aa990"}
Jul 09 09:57:56.096147 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:56.095768    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:57:56.118630 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:57:56.118585    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw" podStartSLOduration=2.3230967 podStartE2EDuration="3.118568795s" totalImagesPullingTime="0s" totalInitContainerRuntime="795.472095ms" isStatefulPod=false podCreationTimestamp="2026-07-09 09:57:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:57:56.115924696 +0000 UTC m=+1479.122093767" watchObservedRunningTime="2026-07-09 09:57:56.118568795 +0000 UTC m=+1479.124737870"
Jul 09 09:58:12.434357 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:12.434324    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:58:13.729136 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:13.729087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:58:17.459021 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:17.458993    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:58:17.461379 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:17.461355    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 09:58:27.102845 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:27.102792    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 09:58:40.935278 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:40.935189    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:58:41.320318 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.320278    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"]
Jul 09 09:58:41.324788 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.324763    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.328003 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.327982    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs\""
Jul 09 09:58:41.328003 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.327995    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa-dockercfg-f5pjq\""
Jul 09 09:58:41.337432 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.337400    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"]
Jul 09 09:58:41.348332 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.348307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/287f9bb9-8796-4191-a23b-f475d7178497-kserve-provision-location\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.348437 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.348338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/287f9bb9-8796-4191-a23b-f475d7178497-tokenizer-uds\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.348437 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.348362    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/287f9bb9-8796-4191-a23b-f475d7178497-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.348515 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.348436    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsfl7\" (UniqueName: \"kubernetes.io/projected/287f9bb9-8796-4191-a23b-f475d7178497-kube-api-access-rsfl7\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.362280 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.362244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:58:41.449048 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.449024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/287f9bb9-8796-4191-a23b-f475d7178497-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.449168 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.449066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsfl7\" (UniqueName: \"kubernetes.io/projected/287f9bb9-8796-4191-a23b-f475d7178497-kube-api-access-rsfl7\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.449168 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.449126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/287f9bb9-8796-4191-a23b-f475d7178497-kserve-provision-location\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.449168 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.449144    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/287f9bb9-8796-4191-a23b-f475d7178497-tokenizer-uds\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.449583 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.449557    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/287f9bb9-8796-4191-a23b-f475d7178497-tokenizer-uds\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.449701 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.449601    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/287f9bb9-8796-4191-a23b-f475d7178497-kserve-provision-location\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.452096 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.452079    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/287f9bb9-8796-4191-a23b-f475d7178497-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.460500 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.460477    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsfl7\" (UniqueName: \"kubernetes.io/projected/287f9bb9-8796-4191-a23b-f475d7178497-kube-api-access-rsfl7\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.638521 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.638451    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:41.782485 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:41.782457    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"]
Jul 09 09:58:41.784702 ip-10-0-132-105 kubenswrapper[2577]: W0709 09:58:41.784669    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod287f9bb9_8796_4191_a23b_f475d7178497.slice/crio-f8aa97bcb693562d57037371e6e285fd1c7fa23b4d953538da0a3f5ad54a1bca WatchSource:0}: Error finding container f8aa97bcb693562d57037371e6e285fd1c7fa23b4d953538da0a3f5ad54a1bca: Status 404 returned error can't find the container with id f8aa97bcb693562d57037371e6e285fd1c7fa23b4d953538da0a3f5ad54a1bca
Jul 09 09:58:42.352173 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:42.352133    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s" event={"ID":"287f9bb9-8796-4191-a23b-f475d7178497","Type":"ContainerStarted","Data":"b469cc1e796fd55ef16d81facca367d7301796762dbc394d3fd317582f624605"}
Jul 09 09:58:42.352532 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:42.352180    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s" event={"ID":"287f9bb9-8796-4191-a23b-f475d7178497","Type":"ContainerStarted","Data":"f8aa97bcb693562d57037371e6e285fd1c7fa23b4d953538da0a3f5ad54a1bca"}
Jul 09 09:58:43.359068 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:43.359034    2577 generic.go:356] "Generic (PLEG): container finished" podID="287f9bb9-8796-4191-a23b-f475d7178497" containerID="b469cc1e796fd55ef16d81facca367d7301796762dbc394d3fd317582f624605" exitCode=0
Jul 09 09:58:43.359501 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:43.359120    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s" event={"ID":"287f9bb9-8796-4191-a23b-f475d7178497","Type":"ContainerDied","Data":"b469cc1e796fd55ef16d81facca367d7301796762dbc394d3fd317582f624605"}
Jul 09 09:58:44.367857 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:44.367826    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s" event={"ID":"287f9bb9-8796-4191-a23b-f475d7178497","Type":"ContainerStarted","Data":"46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e"}
Jul 09 09:58:44.368337 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:44.367919    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 09:58:44.389164 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:44.389107    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s" podStartSLOduration=2.673259577 podStartE2EDuration="3.389093346s" totalImagesPullingTime="0s" totalInitContainerRuntime="715.833769ms" isStatefulPod=false podCreationTimestamp="2026-07-09 09:58:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:58:44.385939498 +0000 UTC m=+1527.392108568" watchObservedRunningTime="2026-07-09 09:58:44.389093346 +0000 UTC m=+1527.395262419"
Jul 09 09:58:52.409830 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:58:52.409781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 09:59:15.375199 ip-10-0-132-105 kubenswrapper[2577]: I0709 09:59:15.375162    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 10:00:14.675899 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.675866    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"]
Jul 09 10:00:14.678988 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.678933    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:00:14.685725 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:00:14.685692    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:00:14.685725 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:00:14.685725 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:00:14.685725 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="2e16164b79f9f7c69647c0d70969da6560886813c63e286dfd974987291aa990" cmd=["/bin/sleep","15"]
Jul 09 10:00:14.686005 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.685743    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw" message="" err=<
Jul 09 10:00:14.686005 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:00:14.686005 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:00:14.686005 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:00:14.686005 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:00:14.685759    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:00:14.686005 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:00:14.686005 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:00:14.686005 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw" podUID="af315a7e-ac57-4b8b-be22-b399963fbcb8" containerName="main" containerID="cri-o://2e16164b79f9f7c69647c0d70969da6560886813c63e286dfd974987291aa990"
Jul 09 10:00:14.686005 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.685831    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw" podUID="af315a7e-ac57-4b8b-be22-b399963fbcb8" containerName="main" containerID="cri-o://2e16164b79f9f7c69647c0d70969da6560886813c63e286dfd974987291aa990" gracePeriod=60
Jul 09 10:00:14.858502 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.858477    2577 generic.go:356] "Generic (PLEG): container finished" podID="af315a7e-ac57-4b8b-be22-b399963fbcb8" containerID="2e16164b79f9f7c69647c0d70969da6560886813c63e286dfd974987291aa990" exitCode=0
Jul 09 10:00:14.858640 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.858515    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw" event={"ID":"af315a7e-ac57-4b8b-be22-b399963fbcb8","Type":"ContainerDied","Data":"2e16164b79f9f7c69647c0d70969da6560886813c63e286dfd974987291aa990"}
Jul 09 10:00:14.946329 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.946306    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 10:00:14.984019 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.983995    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/af315a7e-ac57-4b8b-be22-b399963fbcb8-kserve-provision-location\") pod \"af315a7e-ac57-4b8b-be22-b399963fbcb8\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") "
Jul 09 10:00:14.984142 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.984055    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/af315a7e-ac57-4b8b-be22-b399963fbcb8-tokenizer-uds\") pod \"af315a7e-ac57-4b8b-be22-b399963fbcb8\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") "
Jul 09 10:00:14.984142 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.984092    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx6wd\" (UniqueName: \"kubernetes.io/projected/af315a7e-ac57-4b8b-be22-b399963fbcb8-kube-api-access-hx6wd\") pod \"af315a7e-ac57-4b8b-be22-b399963fbcb8\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") "
Jul 09 10:00:14.984241 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.984141    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af315a7e-ac57-4b8b-be22-b399963fbcb8-tls-certs\") pod \"af315a7e-ac57-4b8b-be22-b399963fbcb8\" (UID: \"af315a7e-ac57-4b8b-be22-b399963fbcb8\") "
Jul 09 10:00:14.984368 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.984342    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af315a7e-ac57-4b8b-be22-b399963fbcb8-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "af315a7e-ac57-4b8b-be22-b399963fbcb8" (UID: "af315a7e-ac57-4b8b-be22-b399963fbcb8"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:00:14.984821 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.984781    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af315a7e-ac57-4b8b-be22-b399963fbcb8-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "af315a7e-ac57-4b8b-be22-b399963fbcb8" (UID: "af315a7e-ac57-4b8b-be22-b399963fbcb8"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:00:14.986122 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.986098    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af315a7e-ac57-4b8b-be22-b399963fbcb8-kube-api-access-hx6wd" (OuterVolumeSpecName: "kube-api-access-hx6wd") pod "af315a7e-ac57-4b8b-be22-b399963fbcb8" (UID: "af315a7e-ac57-4b8b-be22-b399963fbcb8"). InnerVolumeSpecName "kube-api-access-hx6wd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:00:14.986317 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:14.986284    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af315a7e-ac57-4b8b-be22-b399963fbcb8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "af315a7e-ac57-4b8b-be22-b399963fbcb8" (UID: "af315a7e-ac57-4b8b-be22-b399963fbcb8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:00:15.084703 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:15.084663    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/af315a7e-ac57-4b8b-be22-b399963fbcb8-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:00:15.084703 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:15.084701    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx6wd\" (UniqueName: \"kubernetes.io/projected/af315a7e-ac57-4b8b-be22-b399963fbcb8-kube-api-access-hx6wd\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:00:15.084703 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:15.084712    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af315a7e-ac57-4b8b-be22-b399963fbcb8-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:00:15.084942 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:15.084721    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/af315a7e-ac57-4b8b-be22-b399963fbcb8-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:00:15.865480 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:15.865405    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw" event={"ID":"af315a7e-ac57-4b8b-be22-b399963fbcb8","Type":"ContainerDied","Data":"a804d3caceda66786dc40d95a86e883c13e5056b34e2827a908444777a220fba"}
Jul 09 10:00:15.865480 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:15.865423    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"
Jul 09 10:00:15.865480 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:15.865461    2577 scope.go:117] "RemoveContainer" containerID="2e16164b79f9f7c69647c0d70969da6560886813c63e286dfd974987291aa990"
Jul 09 10:00:15.874535 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:15.874518    2577 scope.go:117] "RemoveContainer" containerID="306637aa0d4bacdb19e677bfbe985b49285052498a1a31987b80c440384dc688"
Jul 09 10:00:15.885137 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:15.885118    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"]
Jul 09 10:00:15.889052 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:15.889033    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schefrzdw"]
Jul 09 10:00:17.552290 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:17.552254    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af315a7e-ac57-4b8b-be22-b399963fbcb8" path="/var/lib/kubelet/pods/af315a7e-ac57-4b8b-be22-b399963fbcb8/volumes"
Jul 09 10:00:17.678397 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:17.678365    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:00:35.162757 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:35.162717    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:00:36.525023 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:36.524984    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:00:36.584752 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:36.584716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:00:37.812331 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:37.812290    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:00:37.873748 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:37.873714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:00:38.247404 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:38.247376    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"]
Jul 09 10:00:38.247738 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:38.247727    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af315a7e-ac57-4b8b-be22-b399963fbcb8" containerName="main"
Jul 09 10:00:38.247787 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:38.247740    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="af315a7e-ac57-4b8b-be22-b399963fbcb8" containerName="main"
Jul 09 10:00:38.247787 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:38.247753    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af315a7e-ac57-4b8b-be22-b399963fbcb8" containerName="storage-initializer"
Jul 09 10:00:38.247787 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:38.247760    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="af315a7e-ac57-4b8b-be22-b399963fbcb8" containerName="storage-initializer"
Jul 09 10:00:38.247899 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:38.247836    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="af315a7e-ac57-4b8b-be22-b399963fbcb8" containerName="main"
Jul 09 10:00:38.252046 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:38.252029    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:38.260708 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:38.260685    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"]
Jul 09 10:00:38.323054 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:38.323027    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:00:38.356864 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:38.356840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68fqx\" (UniqueName: \"kubernetes.io/projected/773ee706-6b79-4a33-a277-dc678f4e6518-kube-api-access-68fqx\") pod \"e2e-pvc-model-download-xvtw2\" (UID: \"773ee706-6b79-4a33-a277-dc678f4e6518\") " pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:38.457654 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:38.457631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68fqx\" (UniqueName: \"kubernetes.io/projected/773ee706-6b79-4a33-a277-dc678f4e6518-kube-api-access-68fqx\") pod \"e2e-pvc-model-download-xvtw2\" (UID: \"773ee706-6b79-4a33-a277-dc678f4e6518\") " pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:38.466760 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:38.466734    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68fqx\" (UniqueName: \"kubernetes.io/projected/773ee706-6b79-4a33-a277-dc678f4e6518-kube-api-access-68fqx\") pod \"e2e-pvc-model-download-xvtw2\" (UID: \"773ee706-6b79-4a33-a277-dc678f4e6518\") " pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:43.796860 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:43.796827    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-63da8511-2b18-4fae-9ef5-e3e1723a6f9b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc464e1a50b69522\") pod \"e2e-pvc-model-download-xvtw2\" (UID: \"773ee706-6b79-4a33-a277-dc678f4e6518\") " pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:43.804074 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:43.804053    2577 operation_generator.go:1469] "Controller attach succeeded for volume \"pvc-63da8511-2b18-4fae-9ef5-e3e1723a6f9b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc464e1a50b69522\") pod \"e2e-pvc-model-download-xvtw2\" (UID: \"773ee706-6b79-4a33-a277-dc678f4e6518\") device path: \"\"" pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:43.897223 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:43.897200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-63da8511-2b18-4fae-9ef5-e3e1723a6f9b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc464e1a50b69522\") pod \"e2e-pvc-model-download-xvtw2\" (UID: \"773ee706-6b79-4a33-a277-dc678f4e6518\") " pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:43.897353 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:43.897324    2577 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-63da8511-2b18-4fae-9ef5-e3e1723a6f9b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc464e1a50b69522\") pod \"e2e-pvc-model-download-xvtw2\" (UID: \"773ee706-6b79-4a33-a277-dc678f4e6518\") DevicePath \"\"" pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:43.901933 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:43.901913    2577 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-63da8511-2b18-4fae-9ef5-e3e1723a6f9b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc464e1a50b69522\") pod \"e2e-pvc-model-download-xvtw2\" (UID: \"773ee706-6b79-4a33-a277-dc678f4e6518\") DevicePath \"csi-bb2a9598ceb3f654bd4591f1b09448a7460d1a6a4563d25b2b651b4d31f111a6\"" pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:44.118913 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:44.118884    2577 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-63da8511-2b18-4fae-9ef5-e3e1723a6f9b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc464e1a50b69522\") pod \"e2e-pvc-model-download-xvtw2\" (UID: \"773ee706-6b79-4a33-a277-dc678f4e6518\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/59c6c4e924c1e9af9a1fbe2af17a5efe6f7065d771aa791587657032d06c40d9/globalmount\"" pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:44.135568 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:44.135544    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-63da8511-2b18-4fae-9ef5-e3e1723a6f9b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc464e1a50b69522\") pod \"e2e-pvc-model-download-xvtw2\" (UID: \"773ee706-6b79-4a33-a277-dc678f4e6518\") " pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:44.275302 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:44.275279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:44.412088 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:44.412065    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"]
Jul 09 10:00:44.414066 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:00:44.414040    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod773ee706_6b79_4a33_a277_dc678f4e6518.slice/crio-95d6a2114ee9054fd5c0ec5ac072a805bfa0648aea8743dfc677be3a1151c7ec WatchSource:0}: Error finding container 95d6a2114ee9054fd5c0ec5ac072a805bfa0648aea8743dfc677be3a1151c7ec: Status 404 returned error can't find the container with id 95d6a2114ee9054fd5c0ec5ac072a805bfa0648aea8743dfc677be3a1151c7ec
Jul 09 10:00:45.026063 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:45.026024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2" event={"ID":"773ee706-6b79-4a33-a277-dc678f4e6518","Type":"ContainerStarted","Data":"a89ddd294170b8541d3ffa6d8485dd36afb67f3d5d257aae951cad1baa792d4a"}
Jul 09 10:00:45.026455 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:45.026075    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2" event={"ID":"773ee706-6b79-4a33-a277-dc678f4e6518","Type":"ContainerStarted","Data":"95d6a2114ee9054fd5c0ec5ac072a805bfa0648aea8743dfc677be3a1151c7ec"}
Jul 09 10:00:45.046369 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:45.046325    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2" podStartSLOduration=10.046311125 podStartE2EDuration="10.046311125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 10:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:00:45.043448922 +0000 UTC m=+1648.049618009" watchObservedRunningTime="2026-07-09 10:00:45.046311125 +0000 UTC m=+1648.052480196"
Jul 09 10:00:45.657018 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:45.656979    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"]
Jul 09 10:00:45.664686 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:45.660351    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:00:45.678013 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:00:45.677965    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:00:45.678013 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:00:45.678013 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:00:45.678013 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e" cmd=["/bin/sleep","15"]
Jul 09 10:00:45.678349 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:45.678023    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s" message="" err=<
Jul 09 10:00:45.678349 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:00:45.678349 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:00:45.678349 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:00:45.678349 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:00:45.678041    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:00:45.678349 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:00:45.678349 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:00:45.678349 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s" podUID="287f9bb9-8796-4191-a23b-f475d7178497" containerName="main" containerID="cri-o://46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e"
Jul 09 10:00:45.678349 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:45.678085    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s" podUID="287f9bb9-8796-4191-a23b-f475d7178497" containerName="main" containerID="cri-o://46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e" gracePeriod=60
Jul 09 10:00:45.754561 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:45.754448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:00:45.988206 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:45.988179    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 10:00:46.036091 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.036051    2577 generic.go:356] "Generic (PLEG): container finished" podID="287f9bb9-8796-4191-a23b-f475d7178497" containerID="46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e" exitCode=0
Jul 09 10:00:46.036526 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.036448    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s" event={"ID":"287f9bb9-8796-4191-a23b-f475d7178497","Type":"ContainerDied","Data":"46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e"}
Jul 09 10:00:46.036526 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.036471    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"
Jul 09 10:00:46.036526 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.036514    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s" event={"ID":"287f9bb9-8796-4191-a23b-f475d7178497","Type":"ContainerDied","Data":"f8aa97bcb693562d57037371e6e285fd1c7fa23b4d953538da0a3f5ad54a1bca"}
Jul 09 10:00:46.036702 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.036539    2577 scope.go:117] "RemoveContainer" containerID="46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e"
Jul 09 10:00:46.048006 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.047983    2577 scope.go:117] "RemoveContainer" containerID="b469cc1e796fd55ef16d81facca367d7301796762dbc394d3fd317582f624605"
Jul 09 10:00:46.058533 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.058506    2577 scope.go:117] "RemoveContainer" containerID="46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e"
Jul 09 10:00:46.058878 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:00:46.058851    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e\": container with ID starting with 46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e not found: ID does not exist" containerID="46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e"
Jul 09 10:00:46.058966 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.058887    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e"} err="failed to get container status \"46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e\": rpc error: code = NotFound desc = could not find container \"46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e\": container with ID starting with 46c5cd433c0b8615a5269cc52bc50546497698c5fb65dd8a0a083936a593496e not found: ID does not exist"
Jul 09 10:00:46.058966 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.058912    2577 scope.go:117] "RemoveContainer" containerID="b469cc1e796fd55ef16d81facca367d7301796762dbc394d3fd317582f624605"
Jul 09 10:00:46.059221 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:00:46.059203    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b469cc1e796fd55ef16d81facca367d7301796762dbc394d3fd317582f624605\": container with ID starting with b469cc1e796fd55ef16d81facca367d7301796762dbc394d3fd317582f624605 not found: ID does not exist" containerID="b469cc1e796fd55ef16d81facca367d7301796762dbc394d3fd317582f624605"
Jul 09 10:00:46.059287 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.059230    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b469cc1e796fd55ef16d81facca367d7301796762dbc394d3fd317582f624605"} err="failed to get container status \"b469cc1e796fd55ef16d81facca367d7301796762dbc394d3fd317582f624605\": rpc error: code = NotFound desc = could not find container \"b469cc1e796fd55ef16d81facca367d7301796762dbc394d3fd317582f624605\": container with ID starting with b469cc1e796fd55ef16d81facca367d7301796762dbc394d3fd317582f624605 not found: ID does not exist"
Jul 09 10:00:46.127734 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.127707    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/287f9bb9-8796-4191-a23b-f475d7178497-kserve-provision-location\") pod \"287f9bb9-8796-4191-a23b-f475d7178497\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") "
Jul 09 10:00:46.127909 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.127757    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/287f9bb9-8796-4191-a23b-f475d7178497-tokenizer-uds\") pod \"287f9bb9-8796-4191-a23b-f475d7178497\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") "
Jul 09 10:00:46.127909 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.127808    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/287f9bb9-8796-4191-a23b-f475d7178497-tls-certs\") pod \"287f9bb9-8796-4191-a23b-f475d7178497\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") "
Jul 09 10:00:46.127909 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.127852    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsfl7\" (UniqueName: \"kubernetes.io/projected/287f9bb9-8796-4191-a23b-f475d7178497-kube-api-access-rsfl7\") pod \"287f9bb9-8796-4191-a23b-f475d7178497\" (UID: \"287f9bb9-8796-4191-a23b-f475d7178497\") "
Jul 09 10:00:46.128106 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.128070    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/287f9bb9-8796-4191-a23b-f475d7178497-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "287f9bb9-8796-4191-a23b-f475d7178497" (UID: "287f9bb9-8796-4191-a23b-f475d7178497"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:00:46.128202 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.128179    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/287f9bb9-8796-4191-a23b-f475d7178497-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:00:46.128499 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.128459    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/287f9bb9-8796-4191-a23b-f475d7178497-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "287f9bb9-8796-4191-a23b-f475d7178497" (UID: "287f9bb9-8796-4191-a23b-f475d7178497"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:00:46.130519 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.130496    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/287f9bb9-8796-4191-a23b-f475d7178497-kube-api-access-rsfl7" (OuterVolumeSpecName: "kube-api-access-rsfl7") pod "287f9bb9-8796-4191-a23b-f475d7178497" (UID: "287f9bb9-8796-4191-a23b-f475d7178497"). InnerVolumeSpecName "kube-api-access-rsfl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:00:46.130628 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.130511    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/287f9bb9-8796-4191-a23b-f475d7178497-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "287f9bb9-8796-4191-a23b-f475d7178497" (UID: "287f9bb9-8796-4191-a23b-f475d7178497"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:00:46.229755 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.229659    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/287f9bb9-8796-4191-a23b-f475d7178497-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:00:46.229755 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.229691    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/287f9bb9-8796-4191-a23b-f475d7178497-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:00:46.229755 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.229724    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsfl7\" (UniqueName: \"kubernetes.io/projected/287f9bb9-8796-4191-a23b-f475d7178497-kube-api-access-rsfl7\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:00:46.360988 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.360958    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"]
Jul 09 10:00:46.366917 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:46.366894    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schecdx9s"]
Jul 09 10:00:47.551849 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:47.551816    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="287f9bb9-8796-4191-a23b-f475d7178497" path="/var/lib/kubelet/pods/287f9bb9-8796-4191-a23b-f475d7178497/volumes"
Jul 09 10:00:51.065021 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:51.064991    2577 generic.go:356] "Generic (PLEG): container finished" podID="773ee706-6b79-4a33-a277-dc678f4e6518" containerID="a89ddd294170b8541d3ffa6d8485dd36afb67f3d5d257aae951cad1baa792d4a" exitCode=0
Jul 09 10:00:51.065371 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:51.065071    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2" event={"ID":"773ee706-6b79-4a33-a277-dc678f4e6518","Type":"ContainerDied","Data":"a89ddd294170b8541d3ffa6d8485dd36afb67f3d5d257aae951cad1baa792d4a"}
Jul 09 10:00:52.241553 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:52.241524    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:52.370367 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:52.370302    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68fqx\" (UniqueName: \"kubernetes.io/projected/773ee706-6b79-4a33-a277-dc678f4e6518-kube-api-access-68fqx\") pod \"773ee706-6b79-4a33-a277-dc678f4e6518\" (UID: \"773ee706-6b79-4a33-a277-dc678f4e6518\") "
Jul 09 10:00:52.370479 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:52.370419    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-storage\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc464e1a50b69522\") pod \"773ee706-6b79-4a33-a277-dc678f4e6518\" (UID: \"773ee706-6b79-4a33-a277-dc678f4e6518\") "
Jul 09 10:00:52.372351 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:52.372324    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/773ee706-6b79-4a33-a277-dc678f4e6518-kube-api-access-68fqx" (OuterVolumeSpecName: "kube-api-access-68fqx") pod "773ee706-6b79-4a33-a277-dc678f4e6518" (UID: "773ee706-6b79-4a33-a277-dc678f4e6518"). InnerVolumeSpecName "kube-api-access-68fqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:00:52.374213 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:52.374189    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0bc464e1a50b69522" (OuterVolumeSpecName: "model-storage") pod "773ee706-6b79-4a33-a277-dc678f4e6518" (UID: "773ee706-6b79-4a33-a277-dc678f4e6518"). InnerVolumeSpecName "pvc-63da8511-2b18-4fae-9ef5-e3e1723a6f9b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Jul 09 10:00:52.471607 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:52.471584    2577 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-63da8511-2b18-4fae-9ef5-e3e1723a6f9b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc464e1a50b69522\") on node \"ip-10-0-132-105.ec2.internal\" "
Jul 09 10:00:52.471607 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:52.471606    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68fqx\" (UniqueName: \"kubernetes.io/projected/773ee706-6b79-4a33-a277-dc678f4e6518-kube-api-access-68fqx\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:00:52.582097 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:52.582078    2577 operation_generator.go:895] UnmountDevice succeeded for volume "pvc-63da8511-2b18-4fae-9ef5-e3e1723a6f9b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0bc464e1a50b69522") on node "ip-10-0-132-105.ec2.internal"
Jul 09 10:00:52.673627 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:52.673570    2577 reconciler_common.go:299] "Volume detached for volume \"pvc-63da8511-2b18-4fae-9ef5-e3e1723a6f9b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc464e1a50b69522\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"csi-bb2a9598ceb3f654bd4591f1b09448a7460d1a6a4563d25b2b651b4d31f111a6\""
Jul 09 10:00:53.079188 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:53.079164    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2"
Jul 09 10:00:53.079313 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:53.079196    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/e2e-pvc-model-download-xvtw2" event={"ID":"773ee706-6b79-4a33-a277-dc678f4e6518","Type":"ContainerDied","Data":"95d6a2114ee9054fd5c0ec5ac072a805bfa0648aea8743dfc677be3a1151c7ec"}
Jul 09 10:00:53.079313 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:00:53.079230    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95d6a2114ee9054fd5c0ec5ac072a805bfa0648aea8743dfc677be3a1151c7ec"
Jul 09 10:01:09.116438 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.116405    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:01:09.329503 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.329459    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"]
Jul 09 10:01:09.329995 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.329976    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="773ee706-6b79-4a33-a277-dc678f4e6518" containerName="storage-initializer"
Jul 09 10:01:09.329995 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.329997    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="773ee706-6b79-4a33-a277-dc678f4e6518" containerName="storage-initializer"
Jul 09 10:01:09.330100 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.330010    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="287f9bb9-8796-4191-a23b-f475d7178497" containerName="main"
Jul 09 10:01:09.330100 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.330015    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="287f9bb9-8796-4191-a23b-f475d7178497" containerName="main"
Jul 09 10:01:09.330100 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.330031    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="287f9bb9-8796-4191-a23b-f475d7178497" containerName="storage-initializer"
Jul 09 10:01:09.330100 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.330038    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="287f9bb9-8796-4191-a23b-f475d7178497" containerName="storage-initializer"
Jul 09 10:01:09.330100 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.330100    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="287f9bb9-8796-4191-a23b-f475d7178497" containerName="main"
Jul 09 10:01:09.330253 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.330110    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="773ee706-6b79-4a33-a277-dc678f4e6518" containerName="storage-initializer"
Jul 09 10:01:09.333503 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.333486    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:01:09.336083 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.336063    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 10:01:09.336989 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.336966    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 10:01:09.337103 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.337018    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-pvc-router-manage-59b9d263-epp-sa-dockercfg-5d9ts\""
Jul 09 10:01:09.337103 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.337030    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 10:01:09.337103 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.337030    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs\""
Jul 09 10:01:09.341574 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.341552    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"]
Jul 09 10:01:09.347355 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.347334    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:01:09.488286 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.488244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqcgz\" (UniqueName: \"kubernetes.io/projected/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-kube-api-access-lqcgz\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s\" (UID: \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:01:09.488516 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.488487    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-tokenizer-uds\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s\" (UID: \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:01:09.488646 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.488555    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-tls-certs\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s\" (UID: \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:01:09.589258 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.589230    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqcgz\" (UniqueName: \"kubernetes.io/projected/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-kube-api-access-lqcgz\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s\" (UID: \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:01:09.589391 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.589271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-tokenizer-uds\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s\" (UID: \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:01:09.589391 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.589293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-tls-certs\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s\" (UID: \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:01:09.589689 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.589667    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-tokenizer-uds\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s\" (UID: \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:01:09.591770 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.591750    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-tls-certs\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s\" (UID: \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:01:09.597247 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.597226    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqcgz\" (UniqueName: \"kubernetes.io/projected/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-kube-api-access-lqcgz\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s\" (UID: \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:01:09.643551 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.643524    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:01:09.784592 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:09.784568    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"]
Jul 09 10:01:09.786926 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:01:09.786896    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod335a3cb5_292a_43a7_92a7_f7cbfcf213ac.slice/crio-7609262bdd3b645cdfeb498394053b401cde782559f98b6e4720220eea00e87c WatchSource:0}: Error finding container 7609262bdd3b645cdfeb498394053b401cde782559f98b6e4720220eea00e87c: Status 404 returned error can't find the container with id 7609262bdd3b645cdfeb498394053b401cde782559f98b6e4720220eea00e87c
Jul 09 10:01:10.170961 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:10.170882    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s" event={"ID":"335a3cb5-292a-43a7-92a7-f7cbfcf213ac","Type":"ContainerStarted","Data":"a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd"}
Jul 09 10:01:10.170961 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:10.170942    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s" event={"ID":"335a3cb5-292a-43a7-92a7-f7cbfcf213ac","Type":"ContainerStarted","Data":"7609262bdd3b645cdfeb498394053b401cde782559f98b6e4720220eea00e87c"}
Jul 09 10:01:10.171350 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:10.170965    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:01:10.189782 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:10.189730    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s" podStartSLOduration=1.189715636 podStartE2EDuration="1.189715636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 10:01:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:01:10.188073434 +0000 UTC m=+1673.194242539" watchObservedRunningTime="2026-07-09 10:01:10.189715636 +0000 UTC m=+1673.195884706"
Jul 09 10:01:32.023613 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.023570    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:01:32.335015 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.334861    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"]
Jul 09 10:01:32.339687 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.339665    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.342181 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.342160    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 10:01:32.342296 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.342216    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Jul 09 10:01:32.342860 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.342844    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-epp-sa-dockercfg-hwwvg\""
Jul 09 10:01:32.352132 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.352113    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"]
Jul 09 10:01:32.365295 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.365271    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:01:32.460020 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.459985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0b885c95-f584-47f8-a671-0509767d7cc8-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.460134 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.460098    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbngs\" (UniqueName: \"kubernetes.io/projected/0b885c95-f584-47f8-a671-0509767d7cc8-kube-api-access-bbngs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.460184 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.460146    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b885c95-f584-47f8-a671-0509767d7cc8-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.460184 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.460171    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0b885c95-f584-47f8-a671-0509767d7cc8-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.560769 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.560739    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0b885c95-f584-47f8-a671-0509767d7cc8-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.560890 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.560819    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbngs\" (UniqueName: \"kubernetes.io/projected/0b885c95-f584-47f8-a671-0509767d7cc8-kube-api-access-bbngs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.560890 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.560844    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b885c95-f584-47f8-a671-0509767d7cc8-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.560890 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.560862    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0b885c95-f584-47f8-a671-0509767d7cc8-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.561181 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.561165    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0b885c95-f584-47f8-a671-0509767d7cc8-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.561222 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.561171    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0b885c95-f584-47f8-a671-0509767d7cc8-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.563298 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.563279    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b885c95-f584-47f8-a671-0509767d7cc8-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.570572 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.570544    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbngs\" (UniqueName: \"kubernetes.io/projected/0b885c95-f584-47f8-a671-0509767d7cc8-kube-api-access-bbngs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.648789 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.648733    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:32.785558 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:32.785534    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"]
Jul 09 10:01:32.787697 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:01:32.787675    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b885c95_f584_47f8_a671_0509767d7cc8.slice/crio-96a4888b3b21db838dc3027a9df4d520ea1f6f046ebd02abc3f559ae3d450b3a WatchSource:0}: Error finding container 96a4888b3b21db838dc3027a9df4d520ea1f6f046ebd02abc3f559ae3d450b3a: Status 404 returned error can't find the container with id 96a4888b3b21db838dc3027a9df4d520ea1f6f046ebd02abc3f559ae3d450b3a
Jul 09 10:01:33.301227 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:33.301191    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw" event={"ID":"0b885c95-f584-47f8-a671-0509767d7cc8","Type":"ContainerStarted","Data":"96284b321f2dcf191eb6447feecc402c0789027be737c1de387e8fcf856ee990"}
Jul 09 10:01:33.301667 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:33.301242    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw" event={"ID":"0b885c95-f584-47f8-a671-0509767d7cc8","Type":"ContainerStarted","Data":"96a4888b3b21db838dc3027a9df4d520ea1f6f046ebd02abc3f559ae3d450b3a"}
Jul 09 10:01:34.308148 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:34.308109    2577 generic.go:356] "Generic (PLEG): container finished" podID="0b885c95-f584-47f8-a671-0509767d7cc8" containerID="96284b321f2dcf191eb6447feecc402c0789027be737c1de387e8fcf856ee990" exitCode=0
Jul 09 10:01:34.308533 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:34.308193    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw" event={"ID":"0b885c95-f584-47f8-a671-0509767d7cc8","Type":"ContainerDied","Data":"96284b321f2dcf191eb6447feecc402c0789027be737c1de387e8fcf856ee990"}
Jul 09 10:01:35.315653 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:35.315618    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw" event={"ID":"0b885c95-f584-47f8-a671-0509767d7cc8","Type":"ContainerStarted","Data":"18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70"}
Jul 09 10:01:35.316070 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:35.315677    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:01:35.341306 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:35.341241    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw" podStartSLOduration=2.697709555 podStartE2EDuration="3.341223805s" totalImagesPullingTime="0s" totalInitContainerRuntime="643.51425ms" isStatefulPod=false podCreationTimestamp="2026-07-09 10:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:01:35.340834826 +0000 UTC m=+1698.347003909" watchObservedRunningTime="2026-07-09 10:01:35.341223805 +0000 UTC m=+1698.347392879"
Jul 09 10:01:41.178703 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:41.178629    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:01:41.660857 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:41.660816    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:01:47.115248 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:01:47.115198    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:02:06.322538 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:02:06.322510    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:02:16.953693 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:02:16.953653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:02:31.917420 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:02:31.917381    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:03:17.496157 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:17.496128    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:03:17.500103 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:17.500083    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:03:46.586486 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:46.586421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:03:46.600589 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:46.600561    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"]
Jul 09 10:03:46.609393 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:03:46.609363    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:03:46.609393 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:03:46.609393 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:03:46.609393 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd" cmd=["/bin/sleep","15"]
Jul 09 10:03:46.609627 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:46.609409    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s" message="" err=<
Jul 09 10:03:46.609627 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:03:46.609627 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:03:46.609627 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:03:46.609627 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:03:46.609421    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:03:46.609627 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:03:46.609627 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:03:46.609627 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s" podUID="335a3cb5-292a-43a7-92a7-f7cbfcf213ac" containerName="main" containerID="cri-o://a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd"
Jul 09 10:03:46.609627 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:46.609452    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s" podUID="335a3cb5-292a-43a7-92a7-f7cbfcf213ac" containerName="main" containerID="cri-o://a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd" gracePeriod=60
Jul 09 10:03:46.730086 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:46.730055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:03:46.866046 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:46.866022    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:03:46.945136 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:46.945108    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqcgz\" (UniqueName: \"kubernetes.io/projected/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-kube-api-access-lqcgz\") pod \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\" (UID: \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\") "
Jul 09 10:03:46.945136 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:46.945139    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-tls-certs\") pod \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\" (UID: \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\") "
Jul 09 10:03:46.945304 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:46.945187    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-tokenizer-uds\") pod \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\" (UID: \"335a3cb5-292a-43a7-92a7-f7cbfcf213ac\") "
Jul 09 10:03:46.945512 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:46.945490    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "335a3cb5-292a-43a7-92a7-f7cbfcf213ac" (UID: "335a3cb5-292a-43a7-92a7-f7cbfcf213ac"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:03:46.947185 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:46.947154    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "335a3cb5-292a-43a7-92a7-f7cbfcf213ac" (UID: "335a3cb5-292a-43a7-92a7-f7cbfcf213ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:03:46.947330 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:46.947294    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-kube-api-access-lqcgz" (OuterVolumeSpecName: "kube-api-access-lqcgz") pod "335a3cb5-292a-43a7-92a7-f7cbfcf213ac" (UID: "335a3cb5-292a-43a7-92a7-f7cbfcf213ac"). InnerVolumeSpecName "kube-api-access-lqcgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:03:47.016876 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:47.016847    2577 generic.go:356] "Generic (PLEG): container finished" podID="335a3cb5-292a-43a7-92a7-f7cbfcf213ac" containerID="a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd" exitCode=0
Jul 09 10:03:47.016983 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:47.016878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s" event={"ID":"335a3cb5-292a-43a7-92a7-f7cbfcf213ac","Type":"ContainerDied","Data":"a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd"}
Jul 09 10:03:47.016983 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:47.016904    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"
Jul 09 10:03:47.016983 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:47.016950    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s" event={"ID":"335a3cb5-292a-43a7-92a7-f7cbfcf213ac","Type":"ContainerDied","Data":"7609262bdd3b645cdfeb498394053b401cde782559f98b6e4720220eea00e87c"}
Jul 09 10:03:47.016983 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:47.016981    2577 scope.go:117] "RemoveContainer" containerID="a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd"
Jul 09 10:03:47.026899 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:47.026879    2577 scope.go:117] "RemoveContainer" containerID="a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd"
Jul 09 10:03:47.027155 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:03:47.027137    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd\": container with ID starting with a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd not found: ID does not exist" containerID="a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd"
Jul 09 10:03:47.027207 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:47.027166    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd"} err="failed to get container status \"a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd\": rpc error: code = NotFound desc = could not find container \"a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd\": container with ID starting with a7bb0e1dbe79daa7685c37340de341794befc7b01104e23bf90a5757826f9fbd not found: ID does not exist"
Jul 09 10:03:47.040337 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:47.040317    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"]
Jul 09 10:03:47.045013 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:47.044992    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheqg54s"]
Jul 09 10:03:47.045952 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:47.045928    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:03:47.045952 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:47.045949    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqcgz\" (UniqueName: \"kubernetes.io/projected/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-kube-api-access-lqcgz\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:03:47.046072 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:47.045960    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/335a3cb5-292a-43a7-92a7-f7cbfcf213ac-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:03:47.552889 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:47.552849    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="335a3cb5-292a-43a7-92a7-f7cbfcf213ac" path="/var/lib/kubelet/pods/335a3cb5-292a-43a7-92a7-f7cbfcf213ac/volumes"
Jul 09 10:03:54.911272 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:54.911237    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:03:54.969495 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:03:54.969454    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:04:11.137063 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.137023    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:04:11.374810 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.374759    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"]
Jul 09 10:04:11.375342 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.375319    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="335a3cb5-292a-43a7-92a7-f7cbfcf213ac" containerName="main"
Jul 09 10:04:11.375342 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.375340    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="335a3cb5-292a-43a7-92a7-f7cbfcf213ac" containerName="main"
Jul 09 10:04:11.375519 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.375445    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="335a3cb5-292a-43a7-92a7-f7cbfcf213ac" containerName="main"
Jul 09 10:04:11.378366 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.378344    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:04:11.381863 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.381847    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs\""
Jul 09 10:04:11.381966 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.381849    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-pvc-router-manage-e8706282-epp-sa-dockercfg-xb5hj\""
Jul 09 10:04:11.392106 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.392049    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"]
Jul 09 10:04:11.411327 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.411305    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:04:11.423271 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.423247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxvk4\" (UniqueName: \"kubernetes.io/projected/c2770304-d021-416b-8cd0-0def15527f24-kube-api-access-lxvk4\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn\" (UID: \"c2770304-d021-416b-8cd0-0def15527f24\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:04:11.423369 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.423325    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c2770304-d021-416b-8cd0-0def15527f24-tokenizer-uds\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn\" (UID: \"c2770304-d021-416b-8cd0-0def15527f24\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:04:11.423369 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.423363    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2770304-d021-416b-8cd0-0def15527f24-tls-certs\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn\" (UID: \"c2770304-d021-416b-8cd0-0def15527f24\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:04:11.523931 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.523910    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxvk4\" (UniqueName: \"kubernetes.io/projected/c2770304-d021-416b-8cd0-0def15527f24-kube-api-access-lxvk4\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn\" (UID: \"c2770304-d021-416b-8cd0-0def15527f24\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:04:11.524031 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.523968    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c2770304-d021-416b-8cd0-0def15527f24-tokenizer-uds\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn\" (UID: \"c2770304-d021-416b-8cd0-0def15527f24\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:04:11.524031 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.524001    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2770304-d021-416b-8cd0-0def15527f24-tls-certs\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn\" (UID: \"c2770304-d021-416b-8cd0-0def15527f24\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:04:11.524337 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.524321    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c2770304-d021-416b-8cd0-0def15527f24-tokenizer-uds\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn\" (UID: \"c2770304-d021-416b-8cd0-0def15527f24\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:04:11.526544 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.526520    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2770304-d021-416b-8cd0-0def15527f24-tls-certs\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn\" (UID: \"c2770304-d021-416b-8cd0-0def15527f24\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:04:11.532930 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.532908    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxvk4\" (UniqueName: \"kubernetes.io/projected/c2770304-d021-416b-8cd0-0def15527f24-kube-api-access-lxvk4\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn\" (UID: \"c2770304-d021-416b-8cd0-0def15527f24\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:04:11.687725 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.687660    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:04:11.833913 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:11.833880    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"]
Jul 09 10:04:12.158602 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:12.158555    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn" event={"ID":"c2770304-d021-416b-8cd0-0def15527f24","Type":"ContainerStarted","Data":"f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b"}
Jul 09 10:04:12.159042 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:12.158630    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:04:12.159042 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:12.158653    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn" event={"ID":"c2770304-d021-416b-8cd0-0def15527f24","Type":"ContainerStarted","Data":"308336888b61a3c1de0cb85dd285fa752e8c512f5d4127dea2eb3f894bcf0608"}
Jul 09 10:04:12.178992 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:12.178939    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn" podStartSLOduration=1.178925379 podStartE2EDuration="1.178925379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 10:04:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:04:12.177633182 +0000 UTC m=+1855.183802278" watchObservedRunningTime="2026-07-09 10:04:12.178925379 +0000 UTC m=+1855.185094450"
Jul 09 10:04:26.070284 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:26.070248    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:04:43.165850 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:04:43.165750    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:06:32.594873 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:32.594841    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:06:58.375716 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.375675    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"]
Jul 09 10:06:58.377086 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.377055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:06:58.385989 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:06:58.385954    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:06:58.385989 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:06:58.385989 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:06:58.385989 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b" cmd=["/bin/sleep","15"]
Jul 09 10:06:58.386299 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.386013    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn" message="" err=<
Jul 09 10:06:58.386299 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:06:58.386299 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:06:58.386299 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:06:58.386299 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:06:58.386032    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:06:58.386299 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:06:58.386299 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:06:58.386299 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn" podUID="c2770304-d021-416b-8cd0-0def15527f24" containerName="main" containerID="cri-o://f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b"
Jul 09 10:06:58.386299 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.386076    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn" podUID="c2770304-d021-416b-8cd0-0def15527f24" containerName="main" containerID="cri-o://f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b" gracePeriod=60
Jul 09 10:06:58.475696 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.475617    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:06:58.647868 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.647846    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:06:58.831611 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.831584    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2770304-d021-416b-8cd0-0def15527f24-tls-certs\") pod \"c2770304-d021-416b-8cd0-0def15527f24\" (UID: \"c2770304-d021-416b-8cd0-0def15527f24\") "
Jul 09 10:06:58.831747 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.831654    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c2770304-d021-416b-8cd0-0def15527f24-tokenizer-uds\") pod \"c2770304-d021-416b-8cd0-0def15527f24\" (UID: \"c2770304-d021-416b-8cd0-0def15527f24\") "
Jul 09 10:06:58.831747 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.831683    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxvk4\" (UniqueName: \"kubernetes.io/projected/c2770304-d021-416b-8cd0-0def15527f24-kube-api-access-lxvk4\") pod \"c2770304-d021-416b-8cd0-0def15527f24\" (UID: \"c2770304-d021-416b-8cd0-0def15527f24\") "
Jul 09 10:06:58.831992 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.831966    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2770304-d021-416b-8cd0-0def15527f24-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "c2770304-d021-416b-8cd0-0def15527f24" (UID: "c2770304-d021-416b-8cd0-0def15527f24"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:06:58.833659 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.833627    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2770304-d021-416b-8cd0-0def15527f24-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c2770304-d021-416b-8cd0-0def15527f24" (UID: "c2770304-d021-416b-8cd0-0def15527f24"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:06:58.833768 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.833677    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2770304-d021-416b-8cd0-0def15527f24-kube-api-access-lxvk4" (OuterVolumeSpecName: "kube-api-access-lxvk4") pod "c2770304-d021-416b-8cd0-0def15527f24" (UID: "c2770304-d021-416b-8cd0-0def15527f24"). InnerVolumeSpecName "kube-api-access-lxvk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:06:58.932430 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.932373    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c2770304-d021-416b-8cd0-0def15527f24-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:06:58.932430 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.932398    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxvk4\" (UniqueName: \"kubernetes.io/projected/c2770304-d021-416b-8cd0-0def15527f24-kube-api-access-lxvk4\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:06:58.932430 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:58.932410    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2770304-d021-416b-8cd0-0def15527f24-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:06:59.034533 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:59.034499    2577 generic.go:356] "Generic (PLEG): container finished" podID="c2770304-d021-416b-8cd0-0def15527f24" containerID="f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b" exitCode=0
Jul 09 10:06:59.034656 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:59.034550    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"
Jul 09 10:06:59.034656 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:59.034575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn" event={"ID":"c2770304-d021-416b-8cd0-0def15527f24","Type":"ContainerDied","Data":"f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b"}
Jul 09 10:06:59.034656 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:59.034621    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn" event={"ID":"c2770304-d021-416b-8cd0-0def15527f24","Type":"ContainerDied","Data":"308336888b61a3c1de0cb85dd285fa752e8c512f5d4127dea2eb3f894bcf0608"}
Jul 09 10:06:59.034656 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:59.034640    2577 scope.go:117] "RemoveContainer" containerID="f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b"
Jul 09 10:06:59.043839 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:59.043824    2577 scope.go:117] "RemoveContainer" containerID="f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b"
Jul 09 10:06:59.044094 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:06:59.044077    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b\": container with ID starting with f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b not found: ID does not exist" containerID="f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b"
Jul 09 10:06:59.044141 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:59.044102    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b"} err="failed to get container status \"f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b\": rpc error: code = NotFound desc = could not find container \"f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b\": container with ID starting with f83dd1ac31bea6f90b1379baad4f9d8e10f8d28e6039d0b5bc2975edf7dce33b not found: ID does not exist"
Jul 09 10:06:59.075219 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:59.075184    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"]
Jul 09 10:06:59.080249 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:59.080224    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-schewcdcn"]
Jul 09 10:06:59.553125 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:06:59.553096    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2770304-d021-416b-8cd0-0def15527f24" path="/var/lib/kubelet/pods/c2770304-d021-416b-8cd0-0def15527f24/volumes"
Jul 09 10:07:36.621338 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:07:36.621306    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:07:36.723327 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:07:36.723300    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:07:38.706577 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:07:38.706547    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:07:38.764229 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:07:38.764200    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:08:01.498912 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:01.498832    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:08:17.533533 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:17.533506    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:08:17.538326 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:17.538305    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:08:35.220619 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.220582    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"]
Jul 09 10:08:35.231526 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:08:35.231485    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:08:35.231526 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:08:35.231526 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:08:35.231526 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70" cmd=["/bin/sleep","15"]
Jul 09 10:08:35.231886 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.231541    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw" message="" err=<
Jul 09 10:08:35.231886 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:08:35.231886 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:08:35.231886 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:08:35.231886 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:08:35.231556    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:08:35.231886 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:08:35.231886 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:08:35.231886 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw" podUID="0b885c95-f584-47f8-a671-0509767d7cc8" containerName="main" containerID="cri-o://18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70"
Jul 09 10:08:35.231886 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.231611    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw" podUID="0b885c95-f584-47f8-a671-0509767d7cc8" containerName="main" containerID="cri-o://18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70" gracePeriod=60
Jul 09 10:08:35.250703 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.250679    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:08:35.315247 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.315217    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:08:35.484295 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.484239    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:08:35.560156 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.560123    2577 generic.go:356] "Generic (PLEG): container finished" podID="0b885c95-f584-47f8-a671-0509767d7cc8" containerID="18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70" exitCode=0
Jul 09 10:08:35.560317 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.560172    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"
Jul 09 10:08:35.560317 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.560199    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw" event={"ID":"0b885c95-f584-47f8-a671-0509767d7cc8","Type":"ContainerDied","Data":"18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70"}
Jul 09 10:08:35.560317 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.560258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw" event={"ID":"0b885c95-f584-47f8-a671-0509767d7cc8","Type":"ContainerDied","Data":"96a4888b3b21db838dc3027a9df4d520ea1f6f046ebd02abc3f559ae3d450b3a"}
Jul 09 10:08:35.560317 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.560286    2577 scope.go:117] "RemoveContainer" containerID="18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70"
Jul 09 10:08:35.569677 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.569661    2577 scope.go:117] "RemoveContainer" containerID="96284b321f2dcf191eb6447feecc402c0789027be737c1de387e8fcf856ee990"
Jul 09 10:08:35.579467 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.579441    2577 scope.go:117] "RemoveContainer" containerID="18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70"
Jul 09 10:08:35.579721 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:08:35.579702    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70\": container with ID starting with 18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70 not found: ID does not exist" containerID="18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70"
Jul 09 10:08:35.579770 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.579731    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70"} err="failed to get container status \"18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70\": rpc error: code = NotFound desc = could not find container \"18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70\": container with ID starting with 18132795ebe761396c2778a0349cb34e8fbaa774cc26814d0d59349868d7ff70 not found: ID does not exist"
Jul 09 10:08:35.579770 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.579748    2577 scope.go:117] "RemoveContainer" containerID="96284b321f2dcf191eb6447feecc402c0789027be737c1de387e8fcf856ee990"
Jul 09 10:08:35.580011 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:08:35.579994    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96284b321f2dcf191eb6447feecc402c0789027be737c1de387e8fcf856ee990\": container with ID starting with 96284b321f2dcf191eb6447feecc402c0789027be737c1de387e8fcf856ee990 not found: ID does not exist" containerID="96284b321f2dcf191eb6447feecc402c0789027be737c1de387e8fcf856ee990"
Jul 09 10:08:35.580068 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.580014    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96284b321f2dcf191eb6447feecc402c0789027be737c1de387e8fcf856ee990"} err="failed to get container status \"96284b321f2dcf191eb6447feecc402c0789027be737c1de387e8fcf856ee990\": rpc error: code = NotFound desc = could not find container \"96284b321f2dcf191eb6447feecc402c0789027be737c1de387e8fcf856ee990\": container with ID starting with 96284b321f2dcf191eb6447feecc402c0789027be737c1de387e8fcf856ee990 not found: ID does not exist"
Jul 09 10:08:35.639586 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.639561    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbngs\" (UniqueName: \"kubernetes.io/projected/0b885c95-f584-47f8-a671-0509767d7cc8-kube-api-access-bbngs\") pod \"0b885c95-f584-47f8-a671-0509767d7cc8\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") "
Jul 09 10:08:35.639689 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.639614    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0b885c95-f584-47f8-a671-0509767d7cc8-tokenizer-uds\") pod \"0b885c95-f584-47f8-a671-0509767d7cc8\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") "
Jul 09 10:08:35.639689 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.639642    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b885c95-f584-47f8-a671-0509767d7cc8-tls-certs\") pod \"0b885c95-f584-47f8-a671-0509767d7cc8\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") "
Jul 09 10:08:35.639855 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.639832    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0b885c95-f584-47f8-a671-0509767d7cc8-kserve-provision-location\") pod \"0b885c95-f584-47f8-a671-0509767d7cc8\" (UID: \"0b885c95-f584-47f8-a671-0509767d7cc8\") "
Jul 09 10:08:35.639932 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.639866    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b885c95-f584-47f8-a671-0509767d7cc8-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "0b885c95-f584-47f8-a671-0509767d7cc8" (UID: "0b885c95-f584-47f8-a671-0509767d7cc8"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:08:35.640067 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.640045    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0b885c95-f584-47f8-a671-0509767d7cc8-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:08:35.640647 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.640610    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b885c95-f584-47f8-a671-0509767d7cc8-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "0b885c95-f584-47f8-a671-0509767d7cc8" (UID: "0b885c95-f584-47f8-a671-0509767d7cc8"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:08:35.641823 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.641782    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b885c95-f584-47f8-a671-0509767d7cc8-kube-api-access-bbngs" (OuterVolumeSpecName: "kube-api-access-bbngs") pod "0b885c95-f584-47f8-a671-0509767d7cc8" (UID: "0b885c95-f584-47f8-a671-0509767d7cc8"). InnerVolumeSpecName "kube-api-access-bbngs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:08:35.641913 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.641838    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b885c95-f584-47f8-a671-0509767d7cc8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0b885c95-f584-47f8-a671-0509767d7cc8" (UID: "0b885c95-f584-47f8-a671-0509767d7cc8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:08:35.741067 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.741010    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0b885c95-f584-47f8-a671-0509767d7cc8-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:08:35.741067 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.741034    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbngs\" (UniqueName: \"kubernetes.io/projected/0b885c95-f584-47f8-a671-0509767d7cc8-kube-api-access-bbngs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:08:35.741067 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.741046    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b885c95-f584-47f8-a671-0509767d7cc8-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:08:35.883344 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.883321    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"]
Jul 09 10:08:35.886948 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:35.886922    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-54c5987dcxrmw"]
Jul 09 10:08:37.551939 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:37.551910    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b885c95-f584-47f8-a671-0509767d7cc8" path="/var/lib/kubelet/pods/0b885c95-f584-47f8-a671-0509767d7cc8/volumes"
Jul 09 10:08:55.715325 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.715293    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"]
Jul 09 10:08:55.715773 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.715625    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b885c95-f584-47f8-a671-0509767d7cc8" containerName="main"
Jul 09 10:08:55.715773 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.715635    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b885c95-f584-47f8-a671-0509767d7cc8" containerName="main"
Jul 09 10:08:55.715773 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.715654    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b885c95-f584-47f8-a671-0509767d7cc8" containerName="storage-initializer"
Jul 09 10:08:55.715773 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.715660    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b885c95-f584-47f8-a671-0509767d7cc8" containerName="storage-initializer"
Jul 09 10:08:55.715773 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.715669    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2770304-d021-416b-8cd0-0def15527f24" containerName="main"
Jul 09 10:08:55.715773 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.715675    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2770304-d021-416b-8cd0-0def15527f24" containerName="main"
Jul 09 10:08:55.715773 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.715732    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2770304-d021-416b-8cd0-0def15527f24" containerName="main"
Jul 09 10:08:55.715773 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.715739    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b885c95-f584-47f8-a671-0509767d7cc8" containerName="main"
Jul 09 10:08:55.718731 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.718715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.721093 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.721074    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-1-openshift-default-dockercfg-f9j98\""
Jul 09 10:08:55.721192 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.721127    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 10:08:55.721192 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.721146    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 10:08:55.721279 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.721251    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-gw-ca-root-cert\""
Jul 09 10:08:55.730599 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.730580    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"]
Jul 09 10:08:55.780431 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.780403    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-envoy\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.780565 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.780450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sxdz\" (UniqueName: \"kubernetes.io/projected/b9f84a30-564b-459c-ac66-1eef56a6c593-kube-api-access-8sxdz\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.780565 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.780485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-token\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.780565 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.780533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-credential-socket\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.780565 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.780563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-data\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.780754 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.780600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9f84a30-564b-459c-ac66-1eef56a6c593-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.780754 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.780659    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-workload-socket\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.780754 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.780689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-workload-certs\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.780754 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.780708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-podinfo\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.881262 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.881231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-credential-socket\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.881404 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.881271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-data\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.881404 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.881301    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9f84a30-564b-459c-ac66-1eef56a6c593-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.881518 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.881415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-workload-socket\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.881518 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.881443    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-workload-certs\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.881518 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.881461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-podinfo\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.881518 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.881511    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-envoy\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.881723 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.881546    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sxdz\" (UniqueName: \"kubernetes.io/projected/b9f84a30-564b-459c-ac66-1eef56a6c593-kube-api-access-8sxdz\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.881723 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.881576    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-token\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.881723 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.881670    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-data\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.881919 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.881778    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-credential-socket\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.881982 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.881952    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-workload-certs\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.882091 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.882074    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-workload-socket\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.882128 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.882089    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9f84a30-564b-459c-ac66-1eef56a6c593-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.883776 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.883756    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-envoy\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.884044 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.884027    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-podinfo\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.890445 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.890420    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-token\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:55.890527 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:55.890512    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sxdz\" (UniqueName: \"kubernetes.io/projected/b9f84a30-564b-459c-ac66-1eef56a6c593-kube-api-access-8sxdz\") pod \"router-gateway-1-openshift-default-857cf9f944-zw9hc\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:56.029218 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:56.029156    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:56.164752 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:56.164722    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"]
Jul 09 10:08:56.166652 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:08:56.166625    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f84a30_564b_459c_ac66_1eef56a6c593.slice/crio-f65a1686162e38e1e0eea2461bcd16b3b14b594cef14004934b3675e7b4025af WatchSource:0}: Error finding container f65a1686162e38e1e0eea2461bcd16b3b14b594cef14004934b3675e7b4025af: Status 404 returned error can't find the container with id f65a1686162e38e1e0eea2461bcd16b3b14b594cef14004934b3675e7b4025af
Jul 09 10:08:56.168864 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:56.168834    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 10:08:56.168966 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:56.168917    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 10:08:56.169027 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:56.168960    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 10:08:56.671687 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:56.671654    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc" event={"ID":"b9f84a30-564b-459c-ac66-1eef56a6c593","Type":"ContainerStarted","Data":"7c06d141cad9f030ef559821a1829f7a45e7126055163de9eb0f01eabc7166d3"}
Jul 09 10:08:56.671862 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:56.671705    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc" event={"ID":"b9f84a30-564b-459c-ac66-1eef56a6c593","Type":"ContainerStarted","Data":"f65a1686162e38e1e0eea2461bcd16b3b14b594cef14004934b3675e7b4025af"}
Jul 09 10:08:56.697728 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:56.697685    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc" podStartSLOduration=1.6976715420000001 podStartE2EDuration="1.697671542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:08:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:08:56.695845635 +0000 UTC m=+2139.702014732" watchObservedRunningTime="2026-07-09 10:08:56.697671542 +0000 UTC m=+2139.703840612"
Jul 09 10:08:57.029949 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:57.029920    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:57.031191 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:57.031163    2577 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc" podUID="b9f84a30-564b-459c-ac66-1eef56a6c593" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 09 10:08:57.776855 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:57.776817    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"]
Jul 09 10:08:58.034183 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:58.034100    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:58.683051 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:58.683019    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:58.683886 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:58.683862    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:08:59.693229 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:08:59.693174    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc" podUID="b9f84a30-564b-459c-ac66-1eef56a6c593" containerName="istio-proxy" containerID="cri-o://7c06d141cad9f030ef559821a1829f7a45e7126055163de9eb0f01eabc7166d3" gracePeriod=30
Jul 09 10:09:07.545194 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.545145    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:09:07.580730 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.580701    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-workload-socket\") pod \"b9f84a30-564b-459c-ac66-1eef56a6c593\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") "
Jul 09 10:09:07.580881 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.580738    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-token\") pod \"b9f84a30-564b-459c-ac66-1eef56a6c593\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") "
Jul 09 10:09:07.580881 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.580760    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-data\") pod \"b9f84a30-564b-459c-ac66-1eef56a6c593\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") "
Jul 09 10:09:07.580881 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.580777    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9f84a30-564b-459c-ac66-1eef56a6c593-istiod-ca-cert\") pod \"b9f84a30-564b-459c-ac66-1eef56a6c593\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") "
Jul 09 10:09:07.580881 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.580806    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-workload-certs\") pod \"b9f84a30-564b-459c-ac66-1eef56a6c593\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") "
Jul 09 10:09:07.580881 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.580846    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-podinfo\") pod \"b9f84a30-564b-459c-ac66-1eef56a6c593\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") "
Jul 09 10:09:07.581160 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.580902    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-envoy\") pod \"b9f84a30-564b-459c-ac66-1eef56a6c593\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") "
Jul 09 10:09:07.581160 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.580945    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sxdz\" (UniqueName: \"kubernetes.io/projected/b9f84a30-564b-459c-ac66-1eef56a6c593-kube-api-access-8sxdz\") pod \"b9f84a30-564b-459c-ac66-1eef56a6c593\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") "
Jul 09 10:09:07.581160 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.580985    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-credential-socket\") pod \"b9f84a30-564b-459c-ac66-1eef56a6c593\" (UID: \"b9f84a30-564b-459c-ac66-1eef56a6c593\") "
Jul 09 10:09:07.581160 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.581025    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b9f84a30-564b-459c-ac66-1eef56a6c593" (UID: "b9f84a30-564b-459c-ac66-1eef56a6c593"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:09:07.581354 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.581204    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9f84a30-564b-459c-ac66-1eef56a6c593-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b9f84a30-564b-459c-ac66-1eef56a6c593" (UID: "b9f84a30-564b-459c-ac66-1eef56a6c593"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 10:09:07.581354 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.581209    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b9f84a30-564b-459c-ac66-1eef56a6c593" (UID: "b9f84a30-564b-459c-ac66-1eef56a6c593"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:09:07.581354 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.581236    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-workload-socket\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:09:07.581354 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.581260    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-data" (OuterVolumeSpecName: "istio-data") pod "b9f84a30-564b-459c-ac66-1eef56a6c593" (UID: "b9f84a30-564b-459c-ac66-1eef56a6c593"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:09:07.581560 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.581438    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b9f84a30-564b-459c-ac66-1eef56a6c593" (UID: "b9f84a30-564b-459c-ac66-1eef56a6c593"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:09:07.582953 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.582913    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-token" (OuterVolumeSpecName: "istio-token") pod "b9f84a30-564b-459c-ac66-1eef56a6c593" (UID: "b9f84a30-564b-459c-ac66-1eef56a6c593"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:09:07.583319 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.583297    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b9f84a30-564b-459c-ac66-1eef56a6c593" (UID: "b9f84a30-564b-459c-ac66-1eef56a6c593"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 10:09:07.583400 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.583314    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9f84a30-564b-459c-ac66-1eef56a6c593-kube-api-access-8sxdz" (OuterVolumeSpecName: "kube-api-access-8sxdz") pod "b9f84a30-564b-459c-ac66-1eef56a6c593" (UID: "b9f84a30-564b-459c-ac66-1eef56a6c593"). InnerVolumeSpecName "kube-api-access-8sxdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:09:07.583494 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.583478    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b9f84a30-564b-459c-ac66-1eef56a6c593" (UID: "b9f84a30-564b-459c-ac66-1eef56a6c593"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:09:07.682359 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.682271    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-token\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:09:07.682359 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.682303    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-data\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:09:07.682359 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.682315    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9f84a30-564b-459c-ac66-1eef56a6c593-istiod-ca-cert\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:09:07.682359 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.682327    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-workload-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:09:07.682359 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.682340    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-podinfo\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:09:07.682359 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.682352    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-istio-envoy\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:09:07.682759 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.682366    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sxdz\" (UniqueName: \"kubernetes.io/projected/b9f84a30-564b-459c-ac66-1eef56a6c593-kube-api-access-8sxdz\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:09:07.682759 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.682378    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f84a30-564b-459c-ac66-1eef56a6c593-credential-socket\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:09:07.739379 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.739346    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:09:07.744329 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.744302    2577 generic.go:356] "Generic (PLEG): container finished" podID="b9f84a30-564b-459c-ac66-1eef56a6c593" containerID="7c06d141cad9f030ef559821a1829f7a45e7126055163de9eb0f01eabc7166d3" exitCode=0
Jul 09 10:09:07.744460 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.744350    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"
Jul 09 10:09:07.744460 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.744388    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc" event={"ID":"b9f84a30-564b-459c-ac66-1eef56a6c593","Type":"ContainerDied","Data":"7c06d141cad9f030ef559821a1829f7a45e7126055163de9eb0f01eabc7166d3"}
Jul 09 10:09:07.744460 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.744446    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc" event={"ID":"b9f84a30-564b-459c-ac66-1eef56a6c593","Type":"ContainerDied","Data":"f65a1686162e38e1e0eea2461bcd16b3b14b594cef14004934b3675e7b4025af"}
Jul 09 10:09:07.744586 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.744475    2577 scope.go:117] "RemoveContainer" containerID="7c06d141cad9f030ef559821a1829f7a45e7126055163de9eb0f01eabc7166d3"
Jul 09 10:09:07.755025 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.755006    2577 scope.go:117] "RemoveContainer" containerID="7c06d141cad9f030ef559821a1829f7a45e7126055163de9eb0f01eabc7166d3"
Jul 09 10:09:07.755295 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:09:07.755264    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c06d141cad9f030ef559821a1829f7a45e7126055163de9eb0f01eabc7166d3\": container with ID starting with 7c06d141cad9f030ef559821a1829f7a45e7126055163de9eb0f01eabc7166d3 not found: ID does not exist" containerID="7c06d141cad9f030ef559821a1829f7a45e7126055163de9eb0f01eabc7166d3"
Jul 09 10:09:07.755354 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.755306    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c06d141cad9f030ef559821a1829f7a45e7126055163de9eb0f01eabc7166d3"} err="failed to get container status \"7c06d141cad9f030ef559821a1829f7a45e7126055163de9eb0f01eabc7166d3\": rpc error: code = NotFound desc = could not find container \"7c06d141cad9f030ef559821a1829f7a45e7126055163de9eb0f01eabc7166d3\": container with ID starting with 7c06d141cad9f030ef559821a1829f7a45e7126055163de9eb0f01eabc7166d3 not found: ID does not exist"
Jul 09 10:09:07.769093 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.769064    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"]
Jul 09 10:09:07.775282 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:07.775254    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-857cf9f944-zw9hc"]
Jul 09 10:09:08.271437 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.271403    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"]
Jul 09 10:09:08.271999 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.271978    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9f84a30-564b-459c-ac66-1eef56a6c593" containerName="istio-proxy"
Jul 09 10:09:08.272058 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.272003    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9f84a30-564b-459c-ac66-1eef56a6c593" containerName="istio-proxy"
Jul 09 10:09:08.272110 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.272097    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9f84a30-564b-459c-ac66-1eef56a6c593" containerName="istio-proxy"
Jul 09 10:09:08.276870 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.276849    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.279126 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.279106    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 10:09:08.279958 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.279938    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 10:09:08.279958 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.279954    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-test-kserve-self-signed-certs\""
Jul 09 10:09:08.280114 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.279985    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 10:09:08.280114 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.280020    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-test-epp-sa-dockercfg-946gq\""
Jul 09 10:09:08.280114 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.279987    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 10:09:08.284837 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.284816    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"]
Jul 09 10:09:08.387885 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.387859    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjd9s\" (UniqueName: \"kubernetes.io/projected/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-kube-api-access-zjd9s\") pod \"router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.388017 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.387895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.388017 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.387940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.388017 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.387970    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.489104 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.489079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjd9s\" (UniqueName: \"kubernetes.io/projected/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-kube-api-access-zjd9s\") pod \"router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.489248 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.489115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.489248 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.489152    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.489248 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.489185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.489754 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.489725    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.489975 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.489950    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.492778 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.492747    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.499315 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.499289    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjd9s\" (UniqueName: \"kubernetes.io/projected/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-kube-api-access-zjd9s\") pod \"router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.587738 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.587678    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:08.725522 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.725489    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"]
Jul 09 10:09:08.726792 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:09:08.726760    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod009bb69d_2e0d_4e82_9cd4_cf082e73a6e3.slice/crio-68265266bdca6eb4d5974bd3a192a822130f93c637692999aa0f7b1db8fac9de WatchSource:0}: Error finding container 68265266bdca6eb4d5974bd3a192a822130f93c637692999aa0f7b1db8fac9de: Status 404 returned error can't find the container with id 68265266bdca6eb4d5974bd3a192a822130f93c637692999aa0f7b1db8fac9de
Jul 09 10:09:08.752519 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:08.752492    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67" event={"ID":"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3","Type":"ContainerStarted","Data":"68265266bdca6eb4d5974bd3a192a822130f93c637692999aa0f7b1db8fac9de"}
Jul 09 10:09:09.495678 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:09.495653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:09:09.552053 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:09.552023    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9f84a30-564b-459c-ac66-1eef56a6c593" path="/var/lib/kubelet/pods/b9f84a30-564b-459c-ac66-1eef56a6c593/volumes"
Jul 09 10:09:09.554159 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:09.554139    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:09:09.760818 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:09.760764    2577 generic.go:356] "Generic (PLEG): container finished" podID="009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" containerID="301a9faadb6f5642296819f5381d8fb0bcff09069b5cb7e384b83abcbd00f0eb" exitCode=0
Jul 09 10:09:09.761242 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:09.760826    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67" event={"ID":"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3","Type":"ContainerDied","Data":"301a9faadb6f5642296819f5381d8fb0bcff09069b5cb7e384b83abcbd00f0eb"}
Jul 09 10:09:10.771264 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:10.768700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67" event={"ID":"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3","Type":"ContainerStarted","Data":"c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6"}
Jul 09 10:09:10.771264 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:10.769614    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:09:10.790959 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:10.790915    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67" podStartSLOduration=2.145493152 podStartE2EDuration="2.790900993s" totalImagesPullingTime="0s" totalInitContainerRuntime="645.407841ms" isStatefulPod=false podCreationTimestamp="2026-07-09 10:09:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:09:10.788990868 +0000 UTC m=+2153.795159967" watchObservedRunningTime="2026-07-09 10:09:10.790900993 +0000 UTC m=+2153.797070064"
Jul 09 10:09:31.266682 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:31.266635    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:09:42.780733 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:09:42.780707    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:11:15.147745 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:11:15.147694    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:11:15.218148 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:11:15.218125    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:12:23.747868 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:12:23.747835    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:13:16.950481 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:16.950449    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"]
Jul 09 10:13:16.952367 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:16.952340    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:13:17.015033 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.015004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:13:17.066170 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:13:17.066134    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:13:17.066170 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:13:17.066170 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:13:17.066170 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6" cmd=["/bin/sleep","15"]
Jul 09 10:13:17.066407 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.066183    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67" message="" err=<
Jul 09 10:13:17.066407 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:13:17.066407 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:13:17.066407 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:13:17.066407 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:13:17.066194    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:13:17.066407 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:13:17.066407 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:13:17.066407 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67" podUID="009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" containerName="main" containerID="cri-o://c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6"
Jul 09 10:13:17.066407 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.066239    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67" podUID="009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" containerName="main" containerID="cri-o://c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6" gracePeriod=60
Jul 09 10:13:17.326530 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.326507    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:13:17.470433 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.470398    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjd9s\" (UniqueName: \"kubernetes.io/projected/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-kube-api-access-zjd9s\") pod \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") "
Jul 09 10:13:17.470598 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.470443    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-tokenizer-uds\") pod \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") "
Jul 09 10:13:17.470598 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.470469    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-tls-certs\") pod \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") "
Jul 09 10:13:17.470598 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.470522    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-kserve-provision-location\") pod \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\" (UID: \"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3\") "
Jul 09 10:13:17.470772 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.470729    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" (UID: "009bb69d-2e0d-4e82-9cd4-cf082e73a6e3"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:13:17.471241 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.471216    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" (UID: "009bb69d-2e0d-4e82-9cd4-cf082e73a6e3"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:13:17.472402 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.472383    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-kube-api-access-zjd9s" (OuterVolumeSpecName: "kube-api-access-zjd9s") pod "009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" (UID: "009bb69d-2e0d-4e82-9cd4-cf082e73a6e3"). InnerVolumeSpecName "kube-api-access-zjd9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:13:17.472535 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.472512    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" (UID: "009bb69d-2e0d-4e82-9cd4-cf082e73a6e3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:13:17.571133 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.571102    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:13:17.571271 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.571135    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjd9s\" (UniqueName: \"kubernetes.io/projected/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-kube-api-access-zjd9s\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:13:17.571271 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.571151    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:13:17.571271 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.571164    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:13:17.579334 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.579313    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:13:17.583683 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:17.583660    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:13:18.063014 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:18.062976    2577 generic.go:356] "Generic (PLEG): container finished" podID="009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" containerID="c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6" exitCode=0
Jul 09 10:13:18.063514 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:18.063036    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"
Jul 09 10:13:18.063514 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:18.063055    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67" event={"ID":"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3","Type":"ContainerDied","Data":"c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6"}
Jul 09 10:13:18.063514 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:18.063153    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67" event={"ID":"009bb69d-2e0d-4e82-9cd4-cf082e73a6e3","Type":"ContainerDied","Data":"68265266bdca6eb4d5974bd3a192a822130f93c637692999aa0f7b1db8fac9de"}
Jul 09 10:13:18.063514 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:18.063183    2577 scope.go:117] "RemoveContainer" containerID="c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6"
Jul 09 10:13:18.072366 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:18.072164    2577 scope.go:117] "RemoveContainer" containerID="301a9faadb6f5642296819f5381d8fb0bcff09069b5cb7e384b83abcbd00f0eb"
Jul 09 10:13:18.081411 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:18.081393    2577 scope.go:117] "RemoveContainer" containerID="c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6"
Jul 09 10:13:18.081689 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:13:18.081668    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6\": container with ID starting with c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6 not found: ID does not exist" containerID="c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6"
Jul 09 10:13:18.081820 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:18.081697    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6"} err="failed to get container status \"c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6\": rpc error: code = NotFound desc = could not find container \"c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6\": container with ID starting with c09159e09e049f328c7160f4310b33d8e9e7cd7547ff0dea790cdfd1270177e6 not found: ID does not exist"
Jul 09 10:13:18.081820 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:18.081718    2577 scope.go:117] "RemoveContainer" containerID="301a9faadb6f5642296819f5381d8fb0bcff09069b5cb7e384b83abcbd00f0eb"
Jul 09 10:13:18.081820 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:18.081766    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"]
Jul 09 10:13:18.082008 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:13:18.081983    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"301a9faadb6f5642296819f5381d8fb0bcff09069b5cb7e384b83abcbd00f0eb\": container with ID starting with 301a9faadb6f5642296819f5381d8fb0bcff09069b5cb7e384b83abcbd00f0eb not found: ID does not exist" containerID="301a9faadb6f5642296819f5381d8fb0bcff09069b5cb7e384b83abcbd00f0eb"
Jul 09 10:13:18.082048 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:18.082015    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"301a9faadb6f5642296819f5381d8fb0bcff09069b5cb7e384b83abcbd00f0eb"} err="failed to get container status \"301a9faadb6f5642296819f5381d8fb0bcff09069b5cb7e384b83abcbd00f0eb\": rpc error: code = NotFound desc = could not find container \"301a9faadb6f5642296819f5381d8fb0bcff09069b5cb7e384b83abcbd00f0eb\": container with ID starting with 301a9faadb6f5642296819f5381d8fb0bcff09069b5cb7e384b83abcbd00f0eb not found: ID does not exist"
Jul 09 10:13:18.087452 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:18.087431    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-64f4948457-zxj67"]
Jul 09 10:13:19.551669 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:19.551637    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" path="/var/lib/kubelet/pods/009bb69d-2e0d-4e82-9cd4-cf082e73a6e3/volumes"
Jul 09 10:13:43.228122 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.228042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:13:43.416939 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.416903    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"]
Jul 09 10:13:43.417279 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.417264    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" containerName="main"
Jul 09 10:13:43.417326 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.417281    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" containerName="main"
Jul 09 10:13:43.417326 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.417293    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" containerName="storage-initializer"
Jul 09 10:13:43.417326 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.417299    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" containerName="storage-initializer"
Jul 09 10:13:43.417686 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.417377    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="009bb69d-2e0d-4e82-9cd4-cf082e73a6e3" containerName="main"
Jul 09 10:13:43.420633 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.420611    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.423633 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.423613    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs\""
Jul 09 10:13:43.423904 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.423885    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa-dockercfg-l9x7g\""
Jul 09 10:13:43.424600 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.424453    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 10:13:43.424600 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.424495    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 10:13:43.424768 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.424647    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 10:13:43.424768 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.424762    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 10:13:43.435527 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.435505    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"]
Jul 09 10:13:43.454943 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.454918    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:13:43.564938 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.564873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/522c3161-fbd7-4ff4-9008-05bf017c5c60-kserve-provision-location\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.564938 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.564909    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfrg4\" (UniqueName: \"kubernetes.io/projected/522c3161-fbd7-4ff4-9008-05bf017c5c60-kube-api-access-lfrg4\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.564938 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.564936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/522c3161-fbd7-4ff4-9008-05bf017c5c60-tokenizer-uds\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.565128 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.564959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/522c3161-fbd7-4ff4-9008-05bf017c5c60-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.666263 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.666231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfrg4\" (UniqueName: \"kubernetes.io/projected/522c3161-fbd7-4ff4-9008-05bf017c5c60-kube-api-access-lfrg4\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.666407 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.666285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/522c3161-fbd7-4ff4-9008-05bf017c5c60-tokenizer-uds\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.666407 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.666306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/522c3161-fbd7-4ff4-9008-05bf017c5c60-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.666407 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.666371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/522c3161-fbd7-4ff4-9008-05bf017c5c60-kserve-provision-location\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.666662 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.666635    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/522c3161-fbd7-4ff4-9008-05bf017c5c60-tokenizer-uds\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.666770 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.666682    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/522c3161-fbd7-4ff4-9008-05bf017c5c60-kserve-provision-location\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.668810 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.668775    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/522c3161-fbd7-4ff4-9008-05bf017c5c60-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.675188 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.675163    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfrg4\" (UniqueName: \"kubernetes.io/projected/522c3161-fbd7-4ff4-9008-05bf017c5c60-kube-api-access-lfrg4\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.732964 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.732939    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:43.875872 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:43.875845    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"]
Jul 09 10:13:43.878397 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:13:43.878371    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod522c3161_fbd7_4ff4_9008_05bf017c5c60.slice/crio-6c3c2ac7665e1b13e48ebb7fe230202c9c3fa150c524387d660a79f1342b347f WatchSource:0}: Error finding container 6c3c2ac7665e1b13e48ebb7fe230202c9c3fa150c524387d660a79f1342b347f: Status 404 returned error can't find the container with id 6c3c2ac7665e1b13e48ebb7fe230202c9c3fa150c524387d660a79f1342b347f
Jul 09 10:13:44.204308 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:44.204220    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt" event={"ID":"522c3161-fbd7-4ff4-9008-05bf017c5c60","Type":"ContainerStarted","Data":"7cb664e3a722488c6cdf4c849b08274441cc4e12cc80015f60d043db9402531c"}
Jul 09 10:13:44.204308 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:44.204288    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt" event={"ID":"522c3161-fbd7-4ff4-9008-05bf017c5c60","Type":"ContainerStarted","Data":"6c3c2ac7665e1b13e48ebb7fe230202c9c3fa150c524387d660a79f1342b347f"}
Jul 09 10:13:45.210410 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:45.210373    2577 generic.go:356] "Generic (PLEG): container finished" podID="522c3161-fbd7-4ff4-9008-05bf017c5c60" containerID="7cb664e3a722488c6cdf4c849b08274441cc4e12cc80015f60d043db9402531c" exitCode=0
Jul 09 10:13:45.210787 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:45.210458    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt" event={"ID":"522c3161-fbd7-4ff4-9008-05bf017c5c60","Type":"ContainerDied","Data":"7cb664e3a722488c6cdf4c849b08274441cc4e12cc80015f60d043db9402531c"}
Jul 09 10:13:46.218155 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:46.218114    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt" event={"ID":"522c3161-fbd7-4ff4-9008-05bf017c5c60","Type":"ContainerStarted","Data":"5f35d5d728795e174225e23537a1bec0e240ef1f5179ea8ec1a51a1072b486d8"}
Jul 09 10:13:46.218535 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:46.218314    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:13:46.240120 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:46.240071    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt" podStartSLOduration=2.479832221 podStartE2EDuration="3.240057117s" totalImagesPullingTime="0s" totalInitContainerRuntime="760.224896ms" isStatefulPod=false podCreationTimestamp="2026-07-09 10:13:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:13:46.237401632 +0000 UTC m=+2429.243570703" watchObservedRunningTime="2026-07-09 10:13:46.240057117 +0000 UTC m=+2429.246226219"
Jul 09 10:13:57.719456 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:57.719423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:13:59.632817 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:59.632769    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:13:59.720678 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:13:59.720649    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:14:17.225597 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:17.225563    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:14:45.597510 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:45.597480    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:14:45.776286 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:45.776251    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"]
Jul 09 10:14:45.780723 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:45.780699    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:45.784383 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:45.784174    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-kserve-self-signed-certs\""
Jul 09 10:14:45.786361 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:45.786335    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-epp-sa-dockercfg-zgpdz\""
Jul 09 10:14:45.794465 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:45.794441    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"]
Jul 09 10:14:45.831319 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:45.831286    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:14:45.932987 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:45.932907    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9fb6d08f-a4a7-4384-bd42-835614398049-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:45.933106 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:45.933026    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fb6d08f-a4a7-4384-bd42-835614398049-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:45.933106 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:45.933058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9fb6d08f-a4a7-4384-bd42-835614398049-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:45.933106 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:45.933100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s77f2\" (UniqueName: \"kubernetes.io/projected/9fb6d08f-a4a7-4384-bd42-835614398049-kube-api-access-s77f2\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:46.033517 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:46.033487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77f2\" (UniqueName: \"kubernetes.io/projected/9fb6d08f-a4a7-4384-bd42-835614398049-kube-api-access-s77f2\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:46.033670 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:46.033541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9fb6d08f-a4a7-4384-bd42-835614398049-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:46.033670 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:46.033613    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fb6d08f-a4a7-4384-bd42-835614398049-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:46.033670 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:46.033639    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9fb6d08f-a4a7-4384-bd42-835614398049-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:46.034038 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:46.034012    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9fb6d08f-a4a7-4384-bd42-835614398049-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:46.034120 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:46.034065    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9fb6d08f-a4a7-4384-bd42-835614398049-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:46.036180 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:46.036162    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fb6d08f-a4a7-4384-bd42-835614398049-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:46.042669 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:46.042646    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s77f2\" (UniqueName: \"kubernetes.io/projected/9fb6d08f-a4a7-4384-bd42-835614398049-kube-api-access-s77f2\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:46.092545 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:46.092523    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:46.237175 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:46.237152    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"]
Jul 09 10:14:46.239021 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:14:46.238988    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb6d08f_a4a7_4384_bd42_835614398049.slice/crio-fce2d2f6daa7ab226f59a9a975113c02ce313142177a64b137b6bb89f771c921 WatchSource:0}: Error finding container fce2d2f6daa7ab226f59a9a975113c02ce313142177a64b137b6bb89f771c921: Status 404 returned error can't find the container with id fce2d2f6daa7ab226f59a9a975113c02ce313142177a64b137b6bb89f771c921
Jul 09 10:14:46.541552 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:46.541478    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4" event={"ID":"9fb6d08f-a4a7-4384-bd42-835614398049","Type":"ContainerStarted","Data":"61d24cdab58ef1950ddf3609620b4cf393c9b67cff1aea383addccf8b349c4ef"}
Jul 09 10:14:46.541552 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:46.541524    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4" event={"ID":"9fb6d08f-a4a7-4384-bd42-835614398049","Type":"ContainerStarted","Data":"fce2d2f6daa7ab226f59a9a975113c02ce313142177a64b137b6bb89f771c921"}
Jul 09 10:14:47.548061 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:47.548025    2577 generic.go:356] "Generic (PLEG): container finished" podID="9fb6d08f-a4a7-4384-bd42-835614398049" containerID="61d24cdab58ef1950ddf3609620b4cf393c9b67cff1aea383addccf8b349c4ef" exitCode=0
Jul 09 10:14:47.552357 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:47.552334    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4" event={"ID":"9fb6d08f-a4a7-4384-bd42-835614398049","Type":"ContainerDied","Data":"61d24cdab58ef1950ddf3609620b4cf393c9b67cff1aea383addccf8b349c4ef"}
Jul 09 10:14:48.555942 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:48.555907    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4" event={"ID":"9fb6d08f-a4a7-4384-bd42-835614398049","Type":"ContainerStarted","Data":"94cc663bc2aeddc03b21a56034ceee35559e8397e7d5b16d01c86dc749f998b0"}
Jul 09 10:14:48.556340 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:48.555978    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:14:48.577532 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:48.577483    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4" podStartSLOduration=2.89111774 podStartE2EDuration="3.577469567s" totalImagesPullingTime="0s" totalInitContainerRuntime="686.351827ms" isStatefulPod=false podCreationTimestamp="2026-07-09 10:14:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:14:48.574889265 +0000 UTC m=+2491.581058361" watchObservedRunningTime="2026-07-09 10:14:48.577469567 +0000 UTC m=+2491.583638638"
Jul 09 10:14:52.179711 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:14:52.179675    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:15:19.562045 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:15:19.562018    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:15:29.442078 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:15:29.442042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:15:32.585933 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:15:32.585888    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:16:48.934470 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:48.934434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:16:48.957705 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:48.957663    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"]
Jul 09 10:16:48.966624 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:16:48.966593    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:16:48.966624 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:16:48.966624 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:16:48.966624 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="5f35d5d728795e174225e23537a1bec0e240ef1f5179ea8ec1a51a1072b486d8" cmd=["/bin/sleep","15"]
Jul 09 10:16:48.966957 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:48.966640    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt" message="" err=<
Jul 09 10:16:48.966957 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:16:48.966957 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:16:48.966957 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:16:48.966957 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:16:48.966653    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:16:48.966957 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:16:48.966957 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:16:48.966957 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt" podUID="522c3161-fbd7-4ff4-9008-05bf017c5c60" containerName="main" containerID="cri-o://5f35d5d728795e174225e23537a1bec0e240ef1f5179ea8ec1a51a1072b486d8"
Jul 09 10:16:48.966957 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:48.966686    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt" podUID="522c3161-fbd7-4ff4-9008-05bf017c5c60" containerName="main" containerID="cri-o://5f35d5d728795e174225e23537a1bec0e240ef1f5179ea8ec1a51a1072b486d8" gracePeriod=60
Jul 09 10:16:49.008636 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.008607    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:16:49.207914 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.207889    2577 generic.go:356] "Generic (PLEG): container finished" podID="522c3161-fbd7-4ff4-9008-05bf017c5c60" containerID="5f35d5d728795e174225e23537a1bec0e240ef1f5179ea8ec1a51a1072b486d8" exitCode=0
Jul 09 10:16:49.208080 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.207928    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt" event={"ID":"522c3161-fbd7-4ff4-9008-05bf017c5c60","Type":"ContainerDied","Data":"5f35d5d728795e174225e23537a1bec0e240ef1f5179ea8ec1a51a1072b486d8"}
Jul 09 10:16:49.208080 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.207972    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt" event={"ID":"522c3161-fbd7-4ff4-9008-05bf017c5c60","Type":"ContainerDied","Data":"6c3c2ac7665e1b13e48ebb7fe230202c9c3fa150c524387d660a79f1342b347f"}
Jul 09 10:16:49.208080 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.207993    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c3c2ac7665e1b13e48ebb7fe230202c9c3fa150c524387d660a79f1342b347f"
Jul 09 10:16:49.217248 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.217230    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:16:49.338984 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.338953    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/522c3161-fbd7-4ff4-9008-05bf017c5c60-tls-certs\") pod \"522c3161-fbd7-4ff4-9008-05bf017c5c60\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") "
Jul 09 10:16:49.339142 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.338990    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/522c3161-fbd7-4ff4-9008-05bf017c5c60-tokenizer-uds\") pod \"522c3161-fbd7-4ff4-9008-05bf017c5c60\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") "
Jul 09 10:16:49.339142 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.339025    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/522c3161-fbd7-4ff4-9008-05bf017c5c60-kserve-provision-location\") pod \"522c3161-fbd7-4ff4-9008-05bf017c5c60\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") "
Jul 09 10:16:49.339142 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.339048    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfrg4\" (UniqueName: \"kubernetes.io/projected/522c3161-fbd7-4ff4-9008-05bf017c5c60-kube-api-access-lfrg4\") pod \"522c3161-fbd7-4ff4-9008-05bf017c5c60\" (UID: \"522c3161-fbd7-4ff4-9008-05bf017c5c60\") "
Jul 09 10:16:49.339327 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.339248    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/522c3161-fbd7-4ff4-9008-05bf017c5c60-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "522c3161-fbd7-4ff4-9008-05bf017c5c60" (UID: "522c3161-fbd7-4ff4-9008-05bf017c5c60"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:16:49.339782 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.339758    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/522c3161-fbd7-4ff4-9008-05bf017c5c60-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "522c3161-fbd7-4ff4-9008-05bf017c5c60" (UID: "522c3161-fbd7-4ff4-9008-05bf017c5c60"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:16:49.341139 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.341115    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/522c3161-fbd7-4ff4-9008-05bf017c5c60-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "522c3161-fbd7-4ff4-9008-05bf017c5c60" (UID: "522c3161-fbd7-4ff4-9008-05bf017c5c60"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:16:49.341228 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.341137    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/522c3161-fbd7-4ff4-9008-05bf017c5c60-kube-api-access-lfrg4" (OuterVolumeSpecName: "kube-api-access-lfrg4") pod "522c3161-fbd7-4ff4-9008-05bf017c5c60" (UID: "522c3161-fbd7-4ff4-9008-05bf017c5c60"). InnerVolumeSpecName "kube-api-access-lfrg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:16:49.440125 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.440098    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/522c3161-fbd7-4ff4-9008-05bf017c5c60-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:16:49.440125 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.440122    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/522c3161-fbd7-4ff4-9008-05bf017c5c60-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:16:49.440262 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.440133    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/522c3161-fbd7-4ff4-9008-05bf017c5c60-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:16:49.440262 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:49.440143    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfrg4\" (UniqueName: \"kubernetes.io/projected/522c3161-fbd7-4ff4-9008-05bf017c5c60-kube-api-access-lfrg4\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:16:50.213329 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:50.213301    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"
Jul 09 10:16:50.232510 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:50.232477    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"]
Jul 09 10:16:50.238026 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:50.238002    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schev5dqt"]
Jul 09 10:16:51.551498 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:16:51.551465    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="522c3161-fbd7-4ff4-9008-05bf017c5c60" path="/var/lib/kubelet/pods/522c3161-fbd7-4ff4-9008-05bf017c5c60/volumes"
Jul 09 10:17:14.805639 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:14.805605    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:17:14.997222 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:14.997193    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"]
Jul 09 10:17:14.997570 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:14.997552    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="522c3161-fbd7-4ff4-9008-05bf017c5c60" containerName="main"
Jul 09 10:17:14.997570 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:14.997566    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="522c3161-fbd7-4ff4-9008-05bf017c5c60" containerName="main"
Jul 09 10:17:14.997721 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:14.997577    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="522c3161-fbd7-4ff4-9008-05bf017c5c60" containerName="storage-initializer"
Jul 09 10:17:14.997721 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:14.997583    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="522c3161-fbd7-4ff4-9008-05bf017c5c60" containerName="storage-initializer"
Jul 09 10:17:14.997721 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:14.997639    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="522c3161-fbd7-4ff4-9008-05bf017c5c60" containerName="main"
Jul 09 10:17:15.000693 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.000672    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.003019 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.003000    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-epp-sa-dockercfg-gg2vf\""
Jul 09 10:17:15.003122 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.003055    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Jul 09 10:17:15.011673 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.011651    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"]
Jul 09 10:17:15.020259 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.020234    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9082c420-b5dc-4b7c-9e05-4da88c718156-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.020359 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.020264    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzpqz\" (UniqueName: \"kubernetes.io/projected/9082c420-b5dc-4b7c-9e05-4da88c718156-kube-api-access-lzpqz\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.020359 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.020293    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9082c420-b5dc-4b7c-9e05-4da88c718156-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.020447 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.020423    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9082c420-b5dc-4b7c-9e05-4da88c718156-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.030396 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.030371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:17:15.121187 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.121118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9082c420-b5dc-4b7c-9e05-4da88c718156-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.121187 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.121149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzpqz\" (UniqueName: \"kubernetes.io/projected/9082c420-b5dc-4b7c-9e05-4da88c718156-kube-api-access-lzpqz\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.121187 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.121183    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9082c420-b5dc-4b7c-9e05-4da88c718156-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.121412 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.121221    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9082c420-b5dc-4b7c-9e05-4da88c718156-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.121490 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.121467    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9082c420-b5dc-4b7c-9e05-4da88c718156-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.121541 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.121480    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9082c420-b5dc-4b7c-9e05-4da88c718156-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.123559 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.123538    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9082c420-b5dc-4b7c-9e05-4da88c718156-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.129158 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.129134    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzpqz\" (UniqueName: \"kubernetes.io/projected/9082c420-b5dc-4b7c-9e05-4da88c718156-kube-api-access-lzpqz\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.310506 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.310481    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:15.457009 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:15.456980    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"]
Jul 09 10:17:15.458973 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:17:15.458945    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9082c420_b5dc_4b7c_9e05_4da88c718156.slice/crio-a76bbddc718bf9f7f5742004a1d8d615cd2951f49f103655157d99378617e9d4 WatchSource:0}: Error finding container a76bbddc718bf9f7f5742004a1d8d615cd2951f49f103655157d99378617e9d4: Status 404 returned error can't find the container with id a76bbddc718bf9f7f5742004a1d8d615cd2951f49f103655157d99378617e9d4
Jul 09 10:17:16.360425 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:16.360386    2577 generic.go:356] "Generic (PLEG): container finished" podID="9082c420-b5dc-4b7c-9e05-4da88c718156" containerID="98292afac08e404a8281ef52c1c64721c5387c66c5ff72cd555f090ce4ea6f9d" exitCode=0
Jul 09 10:17:16.360879 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:16.360475    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh" event={"ID":"9082c420-b5dc-4b7c-9e05-4da88c718156","Type":"ContainerDied","Data":"98292afac08e404a8281ef52c1c64721c5387c66c5ff72cd555f090ce4ea6f9d"}
Jul 09 10:17:16.360879 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:16.360531    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh" event={"ID":"9082c420-b5dc-4b7c-9e05-4da88c718156","Type":"ContainerStarted","Data":"a76bbddc718bf9f7f5742004a1d8d615cd2951f49f103655157d99378617e9d4"}
Jul 09 10:17:17.368305 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:17.368272    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh" event={"ID":"9082c420-b5dc-4b7c-9e05-4da88c718156","Type":"ContainerStarted","Data":"456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b"}
Jul 09 10:17:17.368774 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:17.368395    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:17.394780 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:17.394733    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh" podStartSLOduration=2.749236801 podStartE2EDuration="3.394720268s" totalImagesPullingTime="0s" totalInitContainerRuntime="645.483467ms" isStatefulPod=false podCreationTimestamp="2026-07-09 10:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:17:17.392692174 +0000 UTC m=+2640.398861244" watchObservedRunningTime="2026-07-09 10:17:17.394720268 +0000 UTC m=+2640.400889339"
Jul 09 10:17:26.157834 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.157789    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"]
Jul 09 10:17:26.166889 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:17:26.166858    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:17:26.166889 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:17:26.166889 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:17:26.166889 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="94cc663bc2aeddc03b21a56034ceee35559e8397e7d5b16d01c86dc749f998b0" cmd=["/bin/sleep","15"]
Jul 09 10:17:26.167108 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.166903    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4" message="" err=<
Jul 09 10:17:26.167108 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:17:26.167108 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:17:26.167108 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:17:26.167108 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:17:26.166917    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:17:26.167108 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:17:26.167108 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:17:26.167108 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4" podUID="9fb6d08f-a4a7-4384-bd42-835614398049" containerName="main" containerID="cri-o://94cc663bc2aeddc03b21a56034ceee35559e8397e7d5b16d01c86dc749f998b0"
Jul 09 10:17:26.167108 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.166950    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4" podUID="9fb6d08f-a4a7-4384-bd42-835614398049" containerName="main" containerID="cri-o://94cc663bc2aeddc03b21a56034ceee35559e8397e7d5b16d01c86dc749f998b0" gracePeriod=60
Jul 09 10:17:26.419355 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.419327    2577 generic.go:356] "Generic (PLEG): container finished" podID="9fb6d08f-a4a7-4384-bd42-835614398049" containerID="94cc663bc2aeddc03b21a56034ceee35559e8397e7d5b16d01c86dc749f998b0" exitCode=0
Jul 09 10:17:26.419457 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.419402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4" event={"ID":"9fb6d08f-a4a7-4384-bd42-835614398049","Type":"ContainerDied","Data":"94cc663bc2aeddc03b21a56034ceee35559e8397e7d5b16d01c86dc749f998b0"}
Jul 09 10:17:26.419457 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.419448    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4" event={"ID":"9fb6d08f-a4a7-4384-bd42-835614398049","Type":"ContainerDied","Data":"fce2d2f6daa7ab226f59a9a975113c02ce313142177a64b137b6bb89f771c921"}
Jul 09 10:17:26.419538 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.419462    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fce2d2f6daa7ab226f59a9a975113c02ce313142177a64b137b6bb89f771c921"
Jul 09 10:17:26.426206 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.426187    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:17:26.512940 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.512914    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9fb6d08f-a4a7-4384-bd42-835614398049-tokenizer-uds\") pod \"9fb6d08f-a4a7-4384-bd42-835614398049\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") "
Jul 09 10:17:26.513063 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.512945    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fb6d08f-a4a7-4384-bd42-835614398049-tls-certs\") pod \"9fb6d08f-a4a7-4384-bd42-835614398049\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") "
Jul 09 10:17:26.513063 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.512968    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s77f2\" (UniqueName: \"kubernetes.io/projected/9fb6d08f-a4a7-4384-bd42-835614398049-kube-api-access-s77f2\") pod \"9fb6d08f-a4a7-4384-bd42-835614398049\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") "
Jul 09 10:17:26.513063 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.513005    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9fb6d08f-a4a7-4384-bd42-835614398049-kserve-provision-location\") pod \"9fb6d08f-a4a7-4384-bd42-835614398049\" (UID: \"9fb6d08f-a4a7-4384-bd42-835614398049\") "
Jul 09 10:17:26.513262 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.513211    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb6d08f-a4a7-4384-bd42-835614398049-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "9fb6d08f-a4a7-4384-bd42-835614398049" (UID: "9fb6d08f-a4a7-4384-bd42-835614398049"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:17:26.513633 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.513612    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb6d08f-a4a7-4384-bd42-835614398049-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9fb6d08f-a4a7-4384-bd42-835614398049" (UID: "9fb6d08f-a4a7-4384-bd42-835614398049"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:17:26.514975 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.514953    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fb6d08f-a4a7-4384-bd42-835614398049-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9fb6d08f-a4a7-4384-bd42-835614398049" (UID: "9fb6d08f-a4a7-4384-bd42-835614398049"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:17:26.515078 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.515057    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb6d08f-a4a7-4384-bd42-835614398049-kube-api-access-s77f2" (OuterVolumeSpecName: "kube-api-access-s77f2") pod "9fb6d08f-a4a7-4384-bd42-835614398049" (UID: "9fb6d08f-a4a7-4384-bd42-835614398049"). InnerVolumeSpecName "kube-api-access-s77f2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:17:26.613708 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.613681    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9fb6d08f-a4a7-4384-bd42-835614398049-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:17:26.613708 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.613702    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fb6d08f-a4a7-4384-bd42-835614398049-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:17:26.613708 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.613711    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s77f2\" (UniqueName: \"kubernetes.io/projected/9fb6d08f-a4a7-4384-bd42-835614398049-kube-api-access-s77f2\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:17:26.613893 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.613719    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9fb6d08f-a4a7-4384-bd42-835614398049-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:17:26.784994 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:26.784963    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:17:27.424465 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:27.424426    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"
Jul 09 10:17:27.449648 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:27.449618    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"]
Jul 09 10:17:27.455274 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:27.455246    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-znfc4"]
Jul 09 10:17:27.552210 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:27.552183    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fb6d08f-a4a7-4384-bd42-835614398049" path="/var/lib/kubelet/pods/9fb6d08f-a4a7-4384-bd42-835614398049/volumes"
Jul 09 10:17:28.935275 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:28.935238    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:17:35.833125 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:35.833093    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:17:36.132334 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.132258    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"]
Jul 09 10:17:36.132846 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.132777    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fb6d08f-a4a7-4384-bd42-835614398049" containerName="storage-initializer"
Jul 09 10:17:36.132846 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.132815    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fb6d08f-a4a7-4384-bd42-835614398049" containerName="storage-initializer"
Jul 09 10:17:36.132846 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.132837    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fb6d08f-a4a7-4384-bd42-835614398049" containerName="main"
Jul 09 10:17:36.132846 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.132846    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fb6d08f-a4a7-4384-bd42-835614398049" containerName="main"
Jul 09 10:17:36.133023 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.132979    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fb6d08f-a4a7-4384-bd42-835614398049" containerName="main"
Jul 09 10:17:36.137258 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.137239    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.139748 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.139732    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-kserve-self-signed-certs\""
Jul 09 10:17:36.140422 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.140403    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-epp-sa-dockercfg-t2tpm\""
Jul 09 10:17:36.147052 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.147028    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"]
Jul 09 10:17:36.163376 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.163349    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:17:36.186421 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.186398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.186534 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.186434    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.186534 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.186472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xskc8\" (UniqueName: \"kubernetes.io/projected/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-kube-api-access-xskc8\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.186660 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.186568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.287903 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.287874    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.288033 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.287928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.288033 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.287956    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.288033 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.287980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xskc8\" (UniqueName: \"kubernetes.io/projected/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-kube-api-access-xskc8\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.288328 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.288311    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.288406 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.288359    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.290274 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.290253    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.295784 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.295755    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xskc8\" (UniqueName: \"kubernetes.io/projected/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-kube-api-access-xskc8\") pod \"stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.448438 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.448372    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:36.586503 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:36.586477    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"]
Jul 09 10:17:36.588356 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:17:36.588328    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c0e1d52_97bf_4a39_b0ab_e6850e3a892d.slice/crio-29c64a25b23ddafa8dedb6ca79b7f1f668232567cf12344f425dc60c32269287 WatchSource:0}: Error finding container 29c64a25b23ddafa8dedb6ca79b7f1f668232567cf12344f425dc60c32269287: Status 404 returned error can't find the container with id 29c64a25b23ddafa8dedb6ca79b7f1f668232567cf12344f425dc60c32269287
Jul 09 10:17:37.480529 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:37.480500    2577 generic.go:356] "Generic (PLEG): container finished" podID="8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" containerID="48756f253607f72b12fafa22a1a5d9336352fb8f9f5b54dd5d2183338d3750bd" exitCode=0
Jul 09 10:17:37.480890 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:37.480543    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq" event={"ID":"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d","Type":"ContainerDied","Data":"48756f253607f72b12fafa22a1a5d9336352fb8f9f5b54dd5d2183338d3750bd"}
Jul 09 10:17:37.480890 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:37.480579    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq" event={"ID":"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d","Type":"ContainerStarted","Data":"29c64a25b23ddafa8dedb6ca79b7f1f668232567cf12344f425dc60c32269287"}
Jul 09 10:17:38.487353 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:38.487317    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq" event={"ID":"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d","Type":"ContainerStarted","Data":"216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15"}
Jul 09 10:17:38.487767 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:38.487369    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:17:38.509318 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:38.509265    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq" podStartSLOduration=1.848998232 podStartE2EDuration="2.509250082s" totalImagesPullingTime="0s" totalInitContainerRuntime="660.25185ms" isStatefulPod=false podCreationTimestamp="2026-07-09 10:17:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:17:38.506454009 +0000 UTC m=+2661.512623080" watchObservedRunningTime="2026-07-09 10:17:38.509250082 +0000 UTC m=+2661.515419182"
Jul 09 10:17:46.490273 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:46.490244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:17:48.375720 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:48.375693    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:17:49.186881 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:17:49.186844    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:18:09.495445 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:18:09.495417    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:18:17.616929 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:18:17.616902    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:18:17.623407 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:18:17.623382    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:19:17.187342 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:19:17.187305    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:20:23.964612 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:23.964568    2577 scope.go:117] "RemoveContainer" containerID="7cb664e3a722488c6cdf4c849b08274441cc4e12cc80015f60d043db9402531c"
Jul 09 10:20:23.974110 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:23.974086    2577 scope.go:117] "RemoveContainer" containerID="5f35d5d728795e174225e23537a1bec0e240ef1f5179ea8ec1a51a1072b486d8"
Jul 09 10:20:37.388121 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.388073    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"]
Jul 09 10:20:37.400702 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:20:37.400668    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:20:37.400702 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:20:37.400702 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:20:37.400702 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15" cmd=["/bin/sleep","15"]
Jul 09 10:20:37.401049 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.400716    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq" message="" err=<
Jul 09 10:20:37.401049 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:20:37.401049 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:20:37.401049 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:20:37.401049 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:20:37.400728    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:20:37.401049 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:20:37.401049 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:20:37.401049 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq" podUID="8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" containerName="main" containerID="cri-o://216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15"
Jul 09 10:20:37.401049 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.400773    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq" podUID="8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" containerName="main" containerID="cri-o://216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15" gracePeriod=60
Jul 09 10:20:37.413612 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.413576    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:20:37.517226 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.517196    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:20:37.665387 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.665366    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:20:37.756851 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.756817    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-tokenizer-uds\") pod \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") "
Jul 09 10:20:37.757005 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.756883    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-kserve-provision-location\") pod \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") "
Jul 09 10:20:37.757005 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.756959    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-tls-certs\") pod \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") "
Jul 09 10:20:37.757128 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.757011    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xskc8\" (UniqueName: \"kubernetes.io/projected/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-kube-api-access-xskc8\") pod \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\" (UID: \"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d\") "
Jul 09 10:20:37.757128 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.757059    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" (UID: "8c0e1d52-97bf-4a39-b0ab-e6850e3a892d"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:20:37.757239 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.757228    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:20:37.757444 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.757423    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" (UID: "8c0e1d52-97bf-4a39-b0ab-e6850e3a892d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:20:37.759025 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.758995    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" (UID: "8c0e1d52-97bf-4a39-b0ab-e6850e3a892d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:20:37.759118 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.759082    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-kube-api-access-xskc8" (OuterVolumeSpecName: "kube-api-access-xskc8") pod "8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" (UID: "8c0e1d52-97bf-4a39-b0ab-e6850e3a892d"). InnerVolumeSpecName "kube-api-access-xskc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:20:37.858164 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.858132    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:20:37.858164 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.858160    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xskc8\" (UniqueName: \"kubernetes.io/projected/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-kube-api-access-xskc8\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:20:37.858316 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:37.858174    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:20:38.485964 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:38.485933    2577 generic.go:356] "Generic (PLEG): container finished" podID="8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" containerID="216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15" exitCode=0
Jul 09 10:20:38.486376 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:38.485987    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"
Jul 09 10:20:38.486376 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:38.486012    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq" event={"ID":"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d","Type":"ContainerDied","Data":"216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15"}
Jul 09 10:20:38.486376 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:38.486060    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq" event={"ID":"8c0e1d52-97bf-4a39-b0ab-e6850e3a892d","Type":"ContainerDied","Data":"29c64a25b23ddafa8dedb6ca79b7f1f668232567cf12344f425dc60c32269287"}
Jul 09 10:20:38.486376 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:38.486081    2577 scope.go:117] "RemoveContainer" containerID="216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15"
Jul 09 10:20:38.496258 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:38.496235    2577 scope.go:117] "RemoveContainer" containerID="48756f253607f72b12fafa22a1a5d9336352fb8f9f5b54dd5d2183338d3750bd"
Jul 09 10:20:38.504029 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:38.504013    2577 scope.go:117] "RemoveContainer" containerID="216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15"
Jul 09 10:20:38.504280 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:20:38.504264    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15\": container with ID starting with 216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15 not found: ID does not exist" containerID="216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15"
Jul 09 10:20:38.504357 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:38.504287    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15"} err="failed to get container status \"216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15\": rpc error: code = NotFound desc = could not find container \"216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15\": container with ID starting with 216a8fd9e24ba2253de4671de21e1f6cc4cb5cbf9dc3980962a7e0d99aa02a15 not found: ID does not exist"
Jul 09 10:20:38.504357 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:38.504304    2577 scope.go:117] "RemoveContainer" containerID="48756f253607f72b12fafa22a1a5d9336352fb8f9f5b54dd5d2183338d3750bd"
Jul 09 10:20:38.504550 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:20:38.504533    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48756f253607f72b12fafa22a1a5d9336352fb8f9f5b54dd5d2183338d3750bd\": container with ID starting with 48756f253607f72b12fafa22a1a5d9336352fb8f9f5b54dd5d2183338d3750bd not found: ID does not exist" containerID="48756f253607f72b12fafa22a1a5d9336352fb8f9f5b54dd5d2183338d3750bd"
Jul 09 10:20:38.504588 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:38.504557    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48756f253607f72b12fafa22a1a5d9336352fb8f9f5b54dd5d2183338d3750bd"} err="failed to get container status \"48756f253607f72b12fafa22a1a5d9336352fb8f9f5b54dd5d2183338d3750bd\": rpc error: code = NotFound desc = could not find container \"48756f253607f72b12fafa22a1a5d9336352fb8f9f5b54dd5d2183338d3750bd\": container with ID starting with 48756f253607f72b12fafa22a1a5d9336352fb8f9f5b54dd5d2183338d3750bd not found: ID does not exist"
Jul 09 10:20:38.508862 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:38.508841    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"]
Jul 09 10:20:38.514943 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:38.514921    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-6d64d9cf6c-nwkqq"]
Jul 09 10:20:39.551091 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:20:39.551057    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" path="/var/lib/kubelet/pods/8c0e1d52-97bf-4a39-b0ab-e6850e3a892d/volumes"
Jul 09 10:21:16.138843 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:21:16.138786    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:21:23.995213 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:21:23.995180    2577 scope.go:117] "RemoveContainer" containerID="61d24cdab58ef1950ddf3609620b4cf393c9b67cff1aea383addccf8b349c4ef"
Jul 09 10:21:24.003687 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:21:24.003668    2577 scope.go:117] "RemoveContainer" containerID="94cc663bc2aeddc03b21a56034ceee35559e8397e7d5b16d01c86dc749f998b0"
Jul 09 10:23:17.652137 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:17.652106    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:23:17.660564 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:17.660536    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:23:46.757166 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:46.757136    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"]
Jul 09 10:23:46.766136 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:23:46.766102    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:23:46.766136 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:23:46.766136 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:23:46.766136 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b" cmd=["/bin/sleep","15"]
Jul 09 10:23:46.766371 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:46.766154    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh" message="" err=<
Jul 09 10:23:46.766371 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:23:46.766371 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:23:46.766371 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:23:46.766371 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:23:46.766167    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:23:46.766371 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:23:46.766371 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:23:46.766371 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh" podUID="9082c420-b5dc-4b7c-9e05-4da88c718156" containerName="main" containerID="cri-o://456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b"
Jul 09 10:23:46.766371 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:46.766215    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh" podUID="9082c420-b5dc-4b7c-9e05-4da88c718156" containerName="main" containerID="cri-o://456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b" gracePeriod=60
Jul 09 10:23:46.816840 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:46.816790    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:23:47.022154 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.022130    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:23:47.127315 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.127283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzpqz\" (UniqueName: \"kubernetes.io/projected/9082c420-b5dc-4b7c-9e05-4da88c718156-kube-api-access-lzpqz\") pod \"9082c420-b5dc-4b7c-9e05-4da88c718156\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") "
Jul 09 10:23:47.127493 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.127395    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9082c420-b5dc-4b7c-9e05-4da88c718156-kserve-provision-location\") pod \"9082c420-b5dc-4b7c-9e05-4da88c718156\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") "
Jul 09 10:23:47.127493 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.127427    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9082c420-b5dc-4b7c-9e05-4da88c718156-tls-certs\") pod \"9082c420-b5dc-4b7c-9e05-4da88c718156\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") "
Jul 09 10:23:47.127493 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.127481    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9082c420-b5dc-4b7c-9e05-4da88c718156-tokenizer-uds\") pod \"9082c420-b5dc-4b7c-9e05-4da88c718156\" (UID: \"9082c420-b5dc-4b7c-9e05-4da88c718156\") "
Jul 09 10:23:47.127973 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.127904    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9082c420-b5dc-4b7c-9e05-4da88c718156-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "9082c420-b5dc-4b7c-9e05-4da88c718156" (UID: "9082c420-b5dc-4b7c-9e05-4da88c718156"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:23:47.128639 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.128605    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9082c420-b5dc-4b7c-9e05-4da88c718156-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9082c420-b5dc-4b7c-9e05-4da88c718156" (UID: "9082c420-b5dc-4b7c-9e05-4da88c718156"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:23:47.129899 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.129872    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9082c420-b5dc-4b7c-9e05-4da88c718156-kube-api-access-lzpqz" (OuterVolumeSpecName: "kube-api-access-lzpqz") pod "9082c420-b5dc-4b7c-9e05-4da88c718156" (UID: "9082c420-b5dc-4b7c-9e05-4da88c718156"). InnerVolumeSpecName "kube-api-access-lzpqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:23:47.130383 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.130363    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9082c420-b5dc-4b7c-9e05-4da88c718156-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9082c420-b5dc-4b7c-9e05-4da88c718156" (UID: "9082c420-b5dc-4b7c-9e05-4da88c718156"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:23:47.228604 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.228574    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9082c420-b5dc-4b7c-9e05-4da88c718156-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:23:47.228604 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.228597    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9082c420-b5dc-4b7c-9e05-4da88c718156-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:23:47.228604 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.228607    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9082c420-b5dc-4b7c-9e05-4da88c718156-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:23:47.228880 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.228617    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzpqz\" (UniqueName: \"kubernetes.io/projected/9082c420-b5dc-4b7c-9e05-4da88c718156-kube-api-access-lzpqz\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:23:47.516205 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.516175    2577 generic.go:356] "Generic (PLEG): container finished" podID="9082c420-b5dc-4b7c-9e05-4da88c718156" containerID="456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b" exitCode=0
Jul 09 10:23:47.516335 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.516225    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"
Jul 09 10:23:47.516335 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.516257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh" event={"ID":"9082c420-b5dc-4b7c-9e05-4da88c718156","Type":"ContainerDied","Data":"456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b"}
Jul 09 10:23:47.516335 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.516311    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh" event={"ID":"9082c420-b5dc-4b7c-9e05-4da88c718156","Type":"ContainerDied","Data":"a76bbddc718bf9f7f5742004a1d8d615cd2951f49f103655157d99378617e9d4"}
Jul 09 10:23:47.516335 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.516332    2577 scope.go:117] "RemoveContainer" containerID="456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b"
Jul 09 10:23:47.526387 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.526303    2577 scope.go:117] "RemoveContainer" containerID="98292afac08e404a8281ef52c1c64721c5387c66c5ff72cd555f090ce4ea6f9d"
Jul 09 10:23:47.537299 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.537170    2577 scope.go:117] "RemoveContainer" containerID="456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b"
Jul 09 10:23:47.537957 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:23:47.537930    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b\": container with ID starting with 456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b not found: ID does not exist" containerID="456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b"
Jul 09 10:23:47.538058 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.537965    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b"} err="failed to get container status \"456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b\": rpc error: code = NotFound desc = could not find container \"456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b\": container with ID starting with 456e59a2107cf677d5efb29d393fb6aeac0056a2f588f92696d84491e5fd707b not found: ID does not exist"
Jul 09 10:23:47.538058 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.537986    2577 scope.go:117] "RemoveContainer" containerID="98292afac08e404a8281ef52c1c64721c5387c66c5ff72cd555f090ce4ea6f9d"
Jul 09 10:23:47.538272 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:23:47.538236    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98292afac08e404a8281ef52c1c64721c5387c66c5ff72cd555f090ce4ea6f9d\": container with ID starting with 98292afac08e404a8281ef52c1c64721c5387c66c5ff72cd555f090ce4ea6f9d not found: ID does not exist" containerID="98292afac08e404a8281ef52c1c64721c5387c66c5ff72cd555f090ce4ea6f9d"
Jul 09 10:23:47.538383 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.538279    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98292afac08e404a8281ef52c1c64721c5387c66c5ff72cd555f090ce4ea6f9d"} err="failed to get container status \"98292afac08e404a8281ef52c1c64721c5387c66c5ff72cd555f090ce4ea6f9d\": rpc error: code = NotFound desc = could not find container \"98292afac08e404a8281ef52c1c64721c5387c66c5ff72cd555f090ce4ea6f9d\": container with ID starting with 98292afac08e404a8281ef52c1c64721c5387c66c5ff72cd555f090ce4ea6f9d not found: ID does not exist"
Jul 09 10:23:47.540439 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.540416    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"]
Jul 09 10:23:47.544334 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.544313    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-76797blfdh"]
Jul 09 10:23:47.552339 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:47.552314    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9082c420-b5dc-4b7c-9e05-4da88c718156" path="/var/lib/kubelet/pods/9082c420-b5dc-4b7c-9e05-4da88c718156/volumes"
Jul 09 10:23:50.744289 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:50.744247    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:23:51.544090 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:23:51.544050    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:24:07.457439 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.457410    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"]
Jul 09 10:24:07.457831 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.457815    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" containerName="main"
Jul 09 10:24:07.457887 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.457834    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" containerName="main"
Jul 09 10:24:07.457887 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.457848    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" containerName="storage-initializer"
Jul 09 10:24:07.457887 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.457856    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" containerName="storage-initializer"
Jul 09 10:24:07.457887 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.457868    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9082c420-b5dc-4b7c-9e05-4da88c718156" containerName="main"
Jul 09 10:24:07.457887 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.457876    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9082c420-b5dc-4b7c-9e05-4da88c718156" containerName="main"
Jul 09 10:24:07.458072 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.457890    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9082c420-b5dc-4b7c-9e05-4da88c718156" containerName="storage-initializer"
Jul 09 10:24:07.458072 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.457897    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9082c420-b5dc-4b7c-9e05-4da88c718156" containerName="storage-initializer"
Jul 09 10:24:07.458072 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.457979    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9082c420-b5dc-4b7c-9e05-4da88c718156" containerName="main"
Jul 09 10:24:07.458072 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.457990    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c0e1d52-97bf-4a39-b0ab-e6850e3a892d" containerName="main"
Jul 09 10:24:07.461048 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.461029    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.463506 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.463486    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-2-openshift-default-dockercfg-fd25z\""
Jul 09 10:24:07.463506 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.463505    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-gw-ca-root-cert\""
Jul 09 10:24:07.463683 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.463566    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 10:24:07.463683 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.463508    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 10:24:07.472561 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.472537    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"]
Jul 09 10:24:07.482260 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.482219    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:24:07.549250 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.549224    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:24:07.580924 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.580903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-workload-socket\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.581065 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.580937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-podinfo\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.581065 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.580971    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-data\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.581155 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.581073    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bzmv\" (UniqueName: \"kubernetes.io/projected/805f3d59-4b9f-4bb9-a188-240e6a7aa306-kube-api-access-5bzmv\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.581155 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.581097    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-credential-socket\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.581155 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.581118    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-workload-certs\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.581155 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.581144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-envoy\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.581291 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.581171    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-token\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.581291 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.581193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.681491 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.681470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-podinfo\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.681638 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.681535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-data\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.681638 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.681568    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bzmv\" (UniqueName: \"kubernetes.io/projected/805f3d59-4b9f-4bb9-a188-240e6a7aa306-kube-api-access-5bzmv\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.681638 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.681596    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-credential-socket\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.681638 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.681623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-workload-certs\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.681902 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.681656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-envoy\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.681902 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.681699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-token\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.681902 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.681733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.681902 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.681784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-workload-socket\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.682177 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.682118    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-credential-socket\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.682260 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.682181    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-workload-certs\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.682260 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.682213    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-workload-socket\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.682385 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.682368    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-data\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.682615 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.682594    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.683954 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.683925    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-envoy\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.684046 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.684028    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-podinfo\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.690880 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.690858    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-token\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.691242 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.691217    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bzmv\" (UniqueName: \"kubernetes.io/projected/805f3d59-4b9f-4bb9-a188-240e6a7aa306-kube-api-access-5bzmv\") pod \"router-gateway-2-openshift-default-6f899b565b-q5dtd\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.773303 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.773279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:07.917836 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.912961    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"]
Jul 09 10:24:07.919568 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:24:07.919540    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805f3d59_4b9f_4bb9_a188_240e6a7aa306.slice/crio-76583c0394c3231c9c1c2200ded3905939bd480a2db37aa46ca5cae680b7b848 WatchSource:0}: Error finding container 76583c0394c3231c9c1c2200ded3905939bd480a2db37aa46ca5cae680b7b848: Status 404 returned error can't find the container with id 76583c0394c3231c9c1c2200ded3905939bd480a2db37aa46ca5cae680b7b848
Jul 09 10:24:07.921840 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.921783    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 10:24:07.921950 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.921920    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 10:24:07.922019 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:07.922001    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 10:24:08.627857 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:08.627792    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd" event={"ID":"805f3d59-4b9f-4bb9-a188-240e6a7aa306","Type":"ContainerStarted","Data":"756864378fa814160bffb3c9f2eee3c861bdb180f141d91d17ab77159a1a67e6"}
Jul 09 10:24:08.628328 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:08.627883    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd" event={"ID":"805f3d59-4b9f-4bb9-a188-240e6a7aa306","Type":"ContainerStarted","Data":"76583c0394c3231c9c1c2200ded3905939bd480a2db37aa46ca5cae680b7b848"}
Jul 09 10:24:08.651865 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:08.651812    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd" podStartSLOduration=1.6517815580000001 podStartE2EDuration="1.651781558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:24:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:24:08.648872389 +0000 UTC m=+3051.655041485" watchObservedRunningTime="2026-07-09 10:24:08.651781558 +0000 UTC m=+3051.657950629"
Jul 09 10:24:08.774335 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:08.774307    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:08.776109 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:08.776070    2577 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd" podUID="805f3d59-4b9f-4bb9-a188-240e6a7aa306" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 09 10:24:09.778046 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:09.777970    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:10.637938 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:10.637909    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:10.638695 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:10.638675    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:11.045646 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:11.045615    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"]
Jul 09 10:24:11.969043 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:11.969006    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:24:12.648379 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:12.648336    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd" podUID="805f3d59-4b9f-4bb9-a188-240e6a7aa306" containerName="istio-proxy" containerID="cri-o://756864378fa814160bffb3c9f2eee3c861bdb180f141d91d17ab77159a1a67e6" gracePeriod=30
Jul 09 10:24:17.895896 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.895867    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:17.970538 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.970509    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bzmv\" (UniqueName: \"kubernetes.io/projected/805f3d59-4b9f-4bb9-a188-240e6a7aa306-kube-api-access-5bzmv\") pod \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") "
Jul 09 10:24:17.970700 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.970549    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-envoy\") pod \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") "
Jul 09 10:24:17.970700 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.970575    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-credential-socket\") pod \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") "
Jul 09 10:24:17.970700 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.970591    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-token\") pod \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") "
Jul 09 10:24:17.970700 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.970686    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-workload-certs\") pod \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") "
Jul 09 10:24:17.970955 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.970731    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istiod-ca-cert\") pod \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") "
Jul 09 10:24:17.970955 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.970792    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-podinfo\") pod \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") "
Jul 09 10:24:17.970955 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.970859    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-workload-socket\") pod \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") "
Jul 09 10:24:17.970955 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.970878    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "805f3d59-4b9f-4bb9-a188-240e6a7aa306" (UID: "805f3d59-4b9f-4bb9-a188-240e6a7aa306"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:24:17.970955 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.970889    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-data\") pod \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\" (UID: \"805f3d59-4b9f-4bb9-a188-240e6a7aa306\") "
Jul 09 10:24:17.971215 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.971004    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "805f3d59-4b9f-4bb9-a188-240e6a7aa306" (UID: "805f3d59-4b9f-4bb9-a188-240e6a7aa306"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:24:17.971215 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.971042    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-data" (OuterVolumeSpecName: "istio-data") pod "805f3d59-4b9f-4bb9-a188-240e6a7aa306" (UID: "805f3d59-4b9f-4bb9-a188-240e6a7aa306"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:24:17.971349 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.971241    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "805f3d59-4b9f-4bb9-a188-240e6a7aa306" (UID: "805f3d59-4b9f-4bb9-a188-240e6a7aa306"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:24:17.971349 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.971260    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "805f3d59-4b9f-4bb9-a188-240e6a7aa306" (UID: "805f3d59-4b9f-4bb9-a188-240e6a7aa306"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 10:24:17.971349 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.971255    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-data\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:24:17.971349 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.971308    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-credential-socket\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:24:17.971349 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.971320    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-workload-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:24:17.972888 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.972857    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "805f3d59-4b9f-4bb9-a188-240e6a7aa306" (UID: "805f3d59-4b9f-4bb9-a188-240e6a7aa306"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 10:24:17.973002 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.972969    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805f3d59-4b9f-4bb9-a188-240e6a7aa306-kube-api-access-5bzmv" (OuterVolumeSpecName: "kube-api-access-5bzmv") pod "805f3d59-4b9f-4bb9-a188-240e6a7aa306" (UID: "805f3d59-4b9f-4bb9-a188-240e6a7aa306"). InnerVolumeSpecName "kube-api-access-5bzmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:24:17.973157 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.973138    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "805f3d59-4b9f-4bb9-a188-240e6a7aa306" (UID: "805f3d59-4b9f-4bb9-a188-240e6a7aa306"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:24:17.973220 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:17.973170    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-token" (OuterVolumeSpecName: "istio-token") pod "805f3d59-4b9f-4bb9-a188-240e6a7aa306" (UID: "805f3d59-4b9f-4bb9-a188-240e6a7aa306"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:24:18.071767 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.071712    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bzmv\" (UniqueName: \"kubernetes.io/projected/805f3d59-4b9f-4bb9-a188-240e6a7aa306-kube-api-access-5bzmv\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:24:18.071767 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.071733    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-envoy\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:24:18.071767 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.071743    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-token\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:24:18.071767 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.071751    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istiod-ca-cert\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:24:18.071767 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.071758    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/805f3d59-4b9f-4bb9-a188-240e6a7aa306-istio-podinfo\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:24:18.071767 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.071766    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/805f3d59-4b9f-4bb9-a188-240e6a7aa306-workload-socket\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:24:18.685014 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.684979    2577 generic.go:356] "Generic (PLEG): container finished" podID="805f3d59-4b9f-4bb9-a188-240e6a7aa306" containerID="756864378fa814160bffb3c9f2eee3c861bdb180f141d91d17ab77159a1a67e6" exitCode=0
Jul 09 10:24:18.685207 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.685017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd" event={"ID":"805f3d59-4b9f-4bb9-a188-240e6a7aa306","Type":"ContainerDied","Data":"756864378fa814160bffb3c9f2eee3c861bdb180f141d91d17ab77159a1a67e6"}
Jul 09 10:24:18.685207 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.685033    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"
Jul 09 10:24:18.685207 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.685073    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd" event={"ID":"805f3d59-4b9f-4bb9-a188-240e6a7aa306","Type":"ContainerDied","Data":"76583c0394c3231c9c1c2200ded3905939bd480a2db37aa46ca5cae680b7b848"}
Jul 09 10:24:18.685207 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.685096    2577 scope.go:117] "RemoveContainer" containerID="756864378fa814160bffb3c9f2eee3c861bdb180f141d91d17ab77159a1a67e6"
Jul 09 10:24:18.695266 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.695246    2577 scope.go:117] "RemoveContainer" containerID="756864378fa814160bffb3c9f2eee3c861bdb180f141d91d17ab77159a1a67e6"
Jul 09 10:24:18.695535 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:24:18.695518    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"756864378fa814160bffb3c9f2eee3c861bdb180f141d91d17ab77159a1a67e6\": container with ID starting with 756864378fa814160bffb3c9f2eee3c861bdb180f141d91d17ab77159a1a67e6 not found: ID does not exist" containerID="756864378fa814160bffb3c9f2eee3c861bdb180f141d91d17ab77159a1a67e6"
Jul 09 10:24:18.695604 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.695543    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"756864378fa814160bffb3c9f2eee3c861bdb180f141d91d17ab77159a1a67e6"} err="failed to get container status \"756864378fa814160bffb3c9f2eee3c861bdb180f141d91d17ab77159a1a67e6\": rpc error: code = NotFound desc = could not find container \"756864378fa814160bffb3c9f2eee3c861bdb180f141d91d17ab77159a1a67e6\": container with ID starting with 756864378fa814160bffb3c9f2eee3c861bdb180f141d91d17ab77159a1a67e6 not found: ID does not exist"
Jul 09 10:24:18.709018 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.708998    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"]
Jul 09 10:24:18.714425 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:18.714405    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-6f899b565b-q5dtd"]
Jul 09 10:24:19.552206 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:19.552166    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="805f3d59-4b9f-4bb9-a188-240e6a7aa306" path="/var/lib/kubelet/pods/805f3d59-4b9f-4bb9-a188-240e6a7aa306/volumes"
Jul 09 10:24:24.656465 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:24.656433    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:24:25.175416 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.175382    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"]
Jul 09 10:24:25.175983 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.175962    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805f3d59-4b9f-4bb9-a188-240e6a7aa306" containerName="istio-proxy"
Jul 09 10:24:25.175983 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.175984    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="805f3d59-4b9f-4bb9-a188-240e6a7aa306" containerName="istio-proxy"
Jul 09 10:24:25.176144 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.176044    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="805f3d59-4b9f-4bb9-a188-240e6a7aa306" containerName="istio-proxy"
Jul 09 10:24:25.180599 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.180576    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.183265 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.183245    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-epp-sa-dockercfg-kwkgs\""
Jul 09 10:24:25.183493 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.183469    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 10:24:25.183890 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.183869    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 10:24:25.183974 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.183909    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 10:24:25.184023 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.183967    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 10:24:25.184023 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.183876    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Jul 09 10:24:25.193924 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.193902    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"]
Jul 09 10:24:25.212494 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.212461    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:24:25.325193 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.325167    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzq4n\" (UniqueName: \"kubernetes.io/projected/049e936a-4f2f-4145-b98e-123fe7bbf45a-kube-api-access-hzq4n\") pod \"router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.325344 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.325209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/049e936a-4f2f-4145-b98e-123fe7bbf45a-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.325344 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.325238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/049e936a-4f2f-4145-b98e-123fe7bbf45a-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.325474 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.325319    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/049e936a-4f2f-4145-b98e-123fe7bbf45a-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.354143 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.354110    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:24:25.426488 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.426417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/049e936a-4f2f-4145-b98e-123fe7bbf45a-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.426594 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.426536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzq4n\" (UniqueName: \"kubernetes.io/projected/049e936a-4f2f-4145-b98e-123fe7bbf45a-kube-api-access-hzq4n\") pod \"router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.426594 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.426569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/049e936a-4f2f-4145-b98e-123fe7bbf45a-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.426722 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.426704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/049e936a-4f2f-4145-b98e-123fe7bbf45a-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.427115 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.427090    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/049e936a-4f2f-4145-b98e-123fe7bbf45a-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.427208 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.427109    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/049e936a-4f2f-4145-b98e-123fe7bbf45a-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.429090 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.429063    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/049e936a-4f2f-4145-b98e-123fe7bbf45a-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.441840 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.441791    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzq4n\" (UniqueName: \"kubernetes.io/projected/049e936a-4f2f-4145-b98e-123fe7bbf45a-kube-api-access-hzq4n\") pod \"router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.490510 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.490487    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:25.630888 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.630858    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"]
Jul 09 10:24:25.631493 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:24:25.631470    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod049e936a_4f2f_4145_b98e_123fe7bbf45a.slice/crio-5fe4f596297cd32d5bec2e58c6db6dac334070e815b83bef0692862ddb7dfc0f WatchSource:0}: Error finding container 5fe4f596297cd32d5bec2e58c6db6dac334070e815b83bef0692862ddb7dfc0f: Status 404 returned error can't find the container with id 5fe4f596297cd32d5bec2e58c6db6dac334070e815b83bef0692862ddb7dfc0f
Jul 09 10:24:25.722379 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.722350    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq" event={"ID":"049e936a-4f2f-4145-b98e-123fe7bbf45a","Type":"ContainerStarted","Data":"10bd655ba4ce71c6c59f020b3e02ebec4ebea450baf19ec32902704bf72b15af"}
Jul 09 10:24:25.722705 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:25.722403    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq" event={"ID":"049e936a-4f2f-4145-b98e-123fe7bbf45a","Type":"ContainerStarted","Data":"5fe4f596297cd32d5bec2e58c6db6dac334070e815b83bef0692862ddb7dfc0f"}
Jul 09 10:24:26.730484 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:26.730442    2577 generic.go:356] "Generic (PLEG): container finished" podID="049e936a-4f2f-4145-b98e-123fe7bbf45a" containerID="10bd655ba4ce71c6c59f020b3e02ebec4ebea450baf19ec32902704bf72b15af" exitCode=0
Jul 09 10:24:26.730966 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:26.730501    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq" event={"ID":"049e936a-4f2f-4145-b98e-123fe7bbf45a","Type":"ContainerDied","Data":"10bd655ba4ce71c6c59f020b3e02ebec4ebea450baf19ec32902704bf72b15af"}
Jul 09 10:24:27.738078 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:27.738041    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq" event={"ID":"049e936a-4f2f-4145-b98e-123fe7bbf45a","Type":"ContainerStarted","Data":"344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc"}
Jul 09 10:24:27.738459 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:27.738159    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:58.744963 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:58.744931    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:24:58.766050 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:24:58.765979    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq" podStartSLOduration=33.116786014 podStartE2EDuration="33.765960461s" totalImagesPullingTime="0s" totalInitContainerRuntime="649.174447ms" isStatefulPod=false podCreationTimestamp="2026-07-09 10:24:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:24:27.771761323 +0000 UTC m=+3070.777930394" watchObservedRunningTime="2026-07-09 10:24:58.765960461 +0000 UTC m=+3101.772129533"
Jul 09 10:25:16.318809 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:25:16.318755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:25:40.543835 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:25:40.543741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:26:15.998327 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:26:15.998287    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:26:58.910068 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:26:58.910034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:27:13.149383 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:13.149339    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:27:30.975142 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:30.975107    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"]
Jul 09 10:27:30.979540 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:30.979498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:27:30.985066 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:27:30.985035    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:27:30.985066 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:27:30.985066 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:27:30.985066 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc" cmd=["/bin/sleep","15"]
Jul 09 10:27:30.985314 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:30.985090    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq" message="" err=<
Jul 09 10:27:30.985314 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:27:30.985314 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:27:30.985314 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:27:30.985314 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:27:30.985108    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:27:30.985314 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:27:30.985314 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:27:30.985314 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq" podUID="049e936a-4f2f-4145-b98e-123fe7bbf45a" containerName="main" containerID="cri-o://344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc"
Jul 09 10:27:30.985314 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:30.985165    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq" podUID="049e936a-4f2f-4145-b98e-123fe7bbf45a" containerName="main" containerID="cri-o://344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc" gracePeriod=60
Jul 09 10:27:31.046548 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.046516    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:27:31.238220 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.238157    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:27:31.366680 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.366650    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/049e936a-4f2f-4145-b98e-123fe7bbf45a-kserve-provision-location\") pod \"049e936a-4f2f-4145-b98e-123fe7bbf45a\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") "
Jul 09 10:27:31.366866 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.366688    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzq4n\" (UniqueName: \"kubernetes.io/projected/049e936a-4f2f-4145-b98e-123fe7bbf45a-kube-api-access-hzq4n\") pod \"049e936a-4f2f-4145-b98e-123fe7bbf45a\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") "
Jul 09 10:27:31.366866 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.366763    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/049e936a-4f2f-4145-b98e-123fe7bbf45a-tokenizer-uds\") pod \"049e936a-4f2f-4145-b98e-123fe7bbf45a\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") "
Jul 09 10:27:31.366866 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.366783    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/049e936a-4f2f-4145-b98e-123fe7bbf45a-tls-certs\") pod \"049e936a-4f2f-4145-b98e-123fe7bbf45a\" (UID: \"049e936a-4f2f-4145-b98e-123fe7bbf45a\") "
Jul 09 10:27:31.367056 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.367000    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/049e936a-4f2f-4145-b98e-123fe7bbf45a-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "049e936a-4f2f-4145-b98e-123fe7bbf45a" (UID: "049e936a-4f2f-4145-b98e-123fe7bbf45a"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:27:31.367404 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.367381    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/049e936a-4f2f-4145-b98e-123fe7bbf45a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "049e936a-4f2f-4145-b98e-123fe7bbf45a" (UID: "049e936a-4f2f-4145-b98e-123fe7bbf45a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:27:31.368811 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.368779    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/049e936a-4f2f-4145-b98e-123fe7bbf45a-kube-api-access-hzq4n" (OuterVolumeSpecName: "kube-api-access-hzq4n") pod "049e936a-4f2f-4145-b98e-123fe7bbf45a" (UID: "049e936a-4f2f-4145-b98e-123fe7bbf45a"). InnerVolumeSpecName "kube-api-access-hzq4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:27:31.368899 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.368865    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049e936a-4f2f-4145-b98e-123fe7bbf45a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "049e936a-4f2f-4145-b98e-123fe7bbf45a" (UID: "049e936a-4f2f-4145-b98e-123fe7bbf45a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:27:31.467779 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.467749    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/049e936a-4f2f-4145-b98e-123fe7bbf45a-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:27:31.467779 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.467773    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/049e936a-4f2f-4145-b98e-123fe7bbf45a-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:27:31.467779 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.467782    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/049e936a-4f2f-4145-b98e-123fe7bbf45a-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:27:31.467990 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.467790    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzq4n\" (UniqueName: \"kubernetes.io/projected/049e936a-4f2f-4145-b98e-123fe7bbf45a-kube-api-access-hzq4n\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:27:31.724141 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.724102    2577 generic.go:356] "Generic (PLEG): container finished" podID="049e936a-4f2f-4145-b98e-123fe7bbf45a" containerID="344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc" exitCode=0
Jul 09 10:27:31.724327 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.724160    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"
Jul 09 10:27:31.724327 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.724186    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq" event={"ID":"049e936a-4f2f-4145-b98e-123fe7bbf45a","Type":"ContainerDied","Data":"344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc"}
Jul 09 10:27:31.724327 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.724254    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq" event={"ID":"049e936a-4f2f-4145-b98e-123fe7bbf45a","Type":"ContainerDied","Data":"5fe4f596297cd32d5bec2e58c6db6dac334070e815b83bef0692862ddb7dfc0f"}
Jul 09 10:27:31.724327 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.724284    2577 scope.go:117] "RemoveContainer" containerID="344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc"
Jul 09 10:27:31.734404 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.734389    2577 scope.go:117] "RemoveContainer" containerID="10bd655ba4ce71c6c59f020b3e02ebec4ebea450baf19ec32902704bf72b15af"
Jul 09 10:27:31.742078 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.742055    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"]
Jul 09 10:27:31.742508 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.742496    2577 scope.go:117] "RemoveContainer" containerID="344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc"
Jul 09 10:27:31.742732 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:27:31.742716    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc\": container with ID starting with 344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc not found: ID does not exist" containerID="344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc"
Jul 09 10:27:31.742775 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.742737    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc"} err="failed to get container status \"344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc\": rpc error: code = NotFound desc = could not find container \"344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc\": container with ID starting with 344dabe8286cf7f16cc04d2f78a85e2dd925f887bf478c30d556f196d98e93dc not found: ID does not exist"
Jul 09 10:27:31.742775 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.742752    2577 scope.go:117] "RemoveContainer" containerID="10bd655ba4ce71c6c59f020b3e02ebec4ebea450baf19ec32902704bf72b15af"
Jul 09 10:27:31.743011 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:27:31.742989    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10bd655ba4ce71c6c59f020b3e02ebec4ebea450baf19ec32902704bf72b15af\": container with ID starting with 10bd655ba4ce71c6c59f020b3e02ebec4ebea450baf19ec32902704bf72b15af not found: ID does not exist" containerID="10bd655ba4ce71c6c59f020b3e02ebec4ebea450baf19ec32902704bf72b15af"
Jul 09 10:27:31.743073 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.743021    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10bd655ba4ce71c6c59f020b3e02ebec4ebea450baf19ec32902704bf72b15af"} err="failed to get container status \"10bd655ba4ce71c6c59f020b3e02ebec4ebea450baf19ec32902704bf72b15af\": rpc error: code = NotFound desc = could not find container \"10bd655ba4ce71c6c59f020b3e02ebec4ebea450baf19ec32902704bf72b15af\": container with ID starting with 10bd655ba4ce71c6c59f020b3e02ebec4ebea450baf19ec32902704bf72b15af not found: ID does not exist"
Jul 09 10:27:31.746126 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:31.746103    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-8bb68db4-fjqfq"]
Jul 09 10:27:33.552080 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:27:33.552046    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="049e936a-4f2f-4145-b98e-123fe7bbf45a" path="/var/lib/kubelet/pods/049e936a-4f2f-4145-b98e-123fe7bbf45a/volumes"
Jul 09 10:28:05.525154 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:05.525119    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:28:05.796894 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:05.796778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:28:17.685691 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:17.685662    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:28:17.696880 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:17.696861    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:28:34.032826 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.032776    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:28:34.261185 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.261147    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"]
Jul 09 10:28:34.261528 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.261512    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="049e936a-4f2f-4145-b98e-123fe7bbf45a" containerName="storage-initializer"
Jul 09 10:28:34.261528 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.261529    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="049e936a-4f2f-4145-b98e-123fe7bbf45a" containerName="storage-initializer"
Jul 09 10:28:34.261665 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.261545    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="049e936a-4f2f-4145-b98e-123fe7bbf45a" containerName="main"
Jul 09 10:28:34.261665 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.261550    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="049e936a-4f2f-4145-b98e-123fe7bbf45a" containerName="main"
Jul 09 10:28:34.261665 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.261622    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="049e936a-4f2f-4145-b98e-123fe7bbf45a" containerName="main"
Jul 09 10:28:34.264848 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.264828    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.267355 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.267336    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"tls-verification-test-kserve-self-signed-certs\""
Jul 09 10:28:34.269205 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.269190    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 10:28:34.269318 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.269300    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 10:28:34.269390 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.269340    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 10:28:34.269390 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.269369    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"tls-verification-test-epp-sa-dockercfg-dfs7n\""
Jul 09 10:28:34.269616 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.269599    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 10:28:34.278596 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.278578    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"]
Jul 09 10:28:34.291022 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.290950    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:28:34.349092 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.349069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5qql\" (UniqueName: \"kubernetes.io/projected/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-kube-api-access-s5qql\") pod \"tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.349192 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.349125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-tokenizer-uds\") pod \"tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.349244 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.349222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-kserve-provision-location\") pod \"tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.349282 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.349256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-tls-certs\") pod \"tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.450399 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.450377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-tokenizer-uds\") pod \"tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.450504 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.450418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-kserve-provision-location\") pod \"tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.450504 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.450437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-tls-certs\") pod \"tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.450504 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.450469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5qql\" (UniqueName: \"kubernetes.io/projected/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-kube-api-access-s5qql\") pod \"tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.450766 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.450746    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-tokenizer-uds\") pod \"tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.450879 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.450783    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-kserve-provision-location\") pod \"tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.453080 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.453056    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-tls-certs\") pod \"tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.465249 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.465222    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5qql\" (UniqueName: \"kubernetes.io/projected/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-kube-api-access-s5qql\") pod \"tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.574575 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.574518    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:34.711679 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:34.711653    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"]
Jul 09 10:28:34.713164 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:28:34.713134    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4b03d6_1a9f_41b8_9823_d8d0eda95632.slice/crio-ef94c2917a470a3ae80dd7f56c1d8ea96e6dbe7639eaa7d0d71a662338e282bf WatchSource:0}: Error finding container ef94c2917a470a3ae80dd7f56c1d8ea96e6dbe7639eaa7d0d71a662338e282bf: Status 404 returned error can't find the container with id ef94c2917a470a3ae80dd7f56c1d8ea96e6dbe7639eaa7d0d71a662338e282bf
Jul 09 10:28:35.058629 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:35.058594    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7" event={"ID":"bf4b03d6-1a9f-41b8-9823-d8d0eda95632","Type":"ContainerStarted","Data":"dcb94ae254f5d1b54f98588a2d5bbd56c0dd2bd19006cbd873fafe44932f3a1f"}
Jul 09 10:28:35.059036 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:35.058651    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7" event={"ID":"bf4b03d6-1a9f-41b8-9823-d8d0eda95632","Type":"ContainerStarted","Data":"ef94c2917a470a3ae80dd7f56c1d8ea96e6dbe7639eaa7d0d71a662338e282bf"}
Jul 09 10:28:36.064995 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:36.064963    2577 generic.go:356] "Generic (PLEG): container finished" podID="bf4b03d6-1a9f-41b8-9823-d8d0eda95632" containerID="dcb94ae254f5d1b54f98588a2d5bbd56c0dd2bd19006cbd873fafe44932f3a1f" exitCode=0
Jul 09 10:28:36.065381 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:36.065045    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7" event={"ID":"bf4b03d6-1a9f-41b8-9823-d8d0eda95632","Type":"ContainerDied","Data":"dcb94ae254f5d1b54f98588a2d5bbd56c0dd2bd19006cbd873fafe44932f3a1f"}
Jul 09 10:28:37.072605 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:37.072573    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7" event={"ID":"bf4b03d6-1a9f-41b8-9823-d8d0eda95632","Type":"ContainerStarted","Data":"0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be"}
Jul 09 10:28:37.073116 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:37.072642    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:28:37.094609 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:37.094563    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7" podStartSLOduration=2.41245887 podStartE2EDuration="3.094548884s" totalImagesPullingTime="0s" totalInitContainerRuntime="682.090014ms" isStatefulPod=false podCreationTimestamp="2026-07-09 10:28:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:28:37.092149033 +0000 UTC m=+3320.098318124" watchObservedRunningTime="2026-07-09 10:28:37.094548884 +0000 UTC m=+3320.100717955"
Jul 09 10:28:39.888969 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:28:39.888882    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:29:08.080289 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:29:08.080258    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:29:25.906418 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:29:25.906379    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:30:23.233077 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:23.233041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:30:53.457437 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:53.457401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:30:53.601018 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:53.600980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:30:53.700309 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:53.700272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:30:53.882046 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:53.882018    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"]
Jul 09 10:30:53.885482 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:53.885464    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:53.887968 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:53.887944    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs\""
Jul 09 10:30:53.888630 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:53.888600    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-dc21cb14-epp-sa-dockercfg-gc6cn\""
Jul 09 10:30:53.898177 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:53.898156    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"]
Jul 09 10:30:53.916304 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:53.916280    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:30:53.961263 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:53.961239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b93180fd-f02f-4116-8f62-351948a5115f-tokenizer-uds\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:53.961349 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:53.961269    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b93180fd-f02f-4116-8f62-351948a5115f-tls-certs\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:53.961349 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:53.961317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b93180fd-f02f-4116-8f62-351948a5115f-kserve-provision-location\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:53.961437 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:53.961354    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t69s\" (UniqueName: \"kubernetes.io/projected/b93180fd-f02f-4116-8f62-351948a5115f-kube-api-access-7t69s\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:54.062486 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:54.062460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b93180fd-f02f-4116-8f62-351948a5115f-tokenizer-uds\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:54.062589 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:54.062493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b93180fd-f02f-4116-8f62-351948a5115f-tls-certs\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:54.062589 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:54.062559    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b93180fd-f02f-4116-8f62-351948a5115f-kserve-provision-location\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:54.062673 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:54.062606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t69s\" (UniqueName: \"kubernetes.io/projected/b93180fd-f02f-4116-8f62-351948a5115f-kube-api-access-7t69s\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:54.062941 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:54.062921    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b93180fd-f02f-4116-8f62-351948a5115f-tokenizer-uds\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:54.062990 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:54.062953    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b93180fd-f02f-4116-8f62-351948a5115f-kserve-provision-location\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:54.065055 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:54.065032    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b93180fd-f02f-4116-8f62-351948a5115f-tls-certs\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:54.072042 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:54.072022    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t69s\" (UniqueName: \"kubernetes.io/projected/b93180fd-f02f-4116-8f62-351948a5115f-kube-api-access-7t69s\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") " pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:54.195225 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:54.195171    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:54.339962 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:54.339933    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"]
Jul 09 10:30:54.340916 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:30:54.340894    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb93180fd_f02f_4116_8f62_351948a5115f.slice/crio-bfaf6390f3eda9058fbfa858249a63ed53f1cda9cf310343df70665104c68622 WatchSource:0}: Error finding container bfaf6390f3eda9058fbfa858249a63ed53f1cda9cf310343df70665104c68622: Status 404 returned error can't find the container with id bfaf6390f3eda9058fbfa858249a63ed53f1cda9cf310343df70665104c68622
Jul 09 10:30:54.787659 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:54.787618    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm" event={"ID":"b93180fd-f02f-4116-8f62-351948a5115f","Type":"ContainerStarted","Data":"88dd1ba0aa462bcb2c17b89ea97158a66aa204bf40cd0c0be76403fe993c5be6"}
Jul 09 10:30:54.788133 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:54.787676    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm" event={"ID":"b93180fd-f02f-4116-8f62-351948a5115f","Type":"ContainerStarted","Data":"bfaf6390f3eda9058fbfa858249a63ed53f1cda9cf310343df70665104c68622"}
Jul 09 10:30:55.794097 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:55.794064    2577 generic.go:356] "Generic (PLEG): container finished" podID="b93180fd-f02f-4116-8f62-351948a5115f" containerID="88dd1ba0aa462bcb2c17b89ea97158a66aa204bf40cd0c0be76403fe993c5be6" exitCode=0
Jul 09 10:30:55.794449 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:55.794151    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm" event={"ID":"b93180fd-f02f-4116-8f62-351948a5115f","Type":"ContainerDied","Data":"88dd1ba0aa462bcb2c17b89ea97158a66aa204bf40cd0c0be76403fe993c5be6"}
Jul 09 10:30:56.801107 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:56.801073    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm" event={"ID":"b93180fd-f02f-4116-8f62-351948a5115f","Type":"ContainerStarted","Data":"fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d"}
Jul 09 10:30:56.801528 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:56.801209    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:30:56.825162 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:30:56.825120    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm" podStartSLOduration=3.108032894 podStartE2EDuration="3.825105336s" totalImagesPullingTime="0s" totalInitContainerRuntime="717.072442ms" isStatefulPod=false podCreationTimestamp="2026-07-09 10:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:30:56.822875867 +0000 UTC m=+3459.829044966" watchObservedRunningTime="2026-07-09 10:30:56.825105336 +0000 UTC m=+3459.831274406"
Jul 09 10:31:10.213487 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:10.213444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:31:11.868950 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:11.868909    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:31:11.892977 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:11.892948    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"]
Jul 09 10:31:11.902471 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:31:11.902429    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:31:11.902471 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:31:11.902471 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:31:11.902471 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be" cmd=["/bin/sleep","15"]
Jul 09 10:31:11.902822 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:11.902474    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7" message="" err=<
Jul 09 10:31:11.902822 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:31:11.902822 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:31:11.902822 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:31:11.902822 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:31:11.902488    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:31:11.902822 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:31:11.902822 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:31:11.902822 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7" podUID="bf4b03d6-1a9f-41b8-9823-d8d0eda95632" containerName="main" containerID="cri-o://0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be"
Jul 09 10:31:11.902822 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:11.902528    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7" podUID="bf4b03d6-1a9f-41b8-9823-d8d0eda95632" containerName="main" containerID="cri-o://0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be" gracePeriod=60
Jul 09 10:31:11.966522 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:11.966492    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:31:12.152298 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.152276    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:31:12.210239 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.210210    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5qql\" (UniqueName: \"kubernetes.io/projected/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-kube-api-access-s5qql\") pod \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") "
Jul 09 10:31:12.210361 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.210261    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-tokenizer-uds\") pod \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") "
Jul 09 10:31:12.210460 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.210439    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-tls-certs\") pod \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") "
Jul 09 10:31:12.210506 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.210491    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "bf4b03d6-1a9f-41b8-9823-d8d0eda95632" (UID: "bf4b03d6-1a9f-41b8-9823-d8d0eda95632"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:31:12.210562 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.210525    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-kserve-provision-location\") pod \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\" (UID: \"bf4b03d6-1a9f-41b8-9823-d8d0eda95632\") "
Jul 09 10:31:12.210773 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.210754    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:31:12.211132 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.211110    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "bf4b03d6-1a9f-41b8-9823-d8d0eda95632" (UID: "bf4b03d6-1a9f-41b8-9823-d8d0eda95632"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:31:12.212380 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.212357    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-kube-api-access-s5qql" (OuterVolumeSpecName: "kube-api-access-s5qql") pod "bf4b03d6-1a9f-41b8-9823-d8d0eda95632" (UID: "bf4b03d6-1a9f-41b8-9823-d8d0eda95632"). InnerVolumeSpecName "kube-api-access-s5qql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:31:12.212460 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.212441    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf4b03d6-1a9f-41b8-9823-d8d0eda95632" (UID: "bf4b03d6-1a9f-41b8-9823-d8d0eda95632"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:31:12.312133 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.312106    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:31:12.312133 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.312128    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5qql\" (UniqueName: \"kubernetes.io/projected/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-kube-api-access-s5qql\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:31:12.312271 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.312138    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4b03d6-1a9f-41b8-9823-d8d0eda95632-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:31:12.893256 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.893220    2577 generic.go:356] "Generic (PLEG): container finished" podID="bf4b03d6-1a9f-41b8-9823-d8d0eda95632" containerID="0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be" exitCode=0
Jul 09 10:31:12.893709 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.893287    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"
Jul 09 10:31:12.893709 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.893310    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7" event={"ID":"bf4b03d6-1a9f-41b8-9823-d8d0eda95632","Type":"ContainerDied","Data":"0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be"}
Jul 09 10:31:12.893709 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.893374    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7" event={"ID":"bf4b03d6-1a9f-41b8-9823-d8d0eda95632","Type":"ContainerDied","Data":"ef94c2917a470a3ae80dd7f56c1d8ea96e6dbe7639eaa7d0d71a662338e282bf"}
Jul 09 10:31:12.893709 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.893402    2577 scope.go:117] "RemoveContainer" containerID="0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be"
Jul 09 10:31:12.903339 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.903321    2577 scope.go:117] "RemoveContainer" containerID="dcb94ae254f5d1b54f98588a2d5bbd56c0dd2bd19006cbd873fafe44932f3a1f"
Jul 09 10:31:12.910987 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.910969    2577 scope.go:117] "RemoveContainer" containerID="0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be"
Jul 09 10:31:12.911241 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:31:12.911223    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be\": container with ID starting with 0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be not found: ID does not exist" containerID="0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be"
Jul 09 10:31:12.911300 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.911249    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be"} err="failed to get container status \"0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be\": rpc error: code = NotFound desc = could not find container \"0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be\": container with ID starting with 0477367bdf67cc24adeb3af239ea59ef8ab303406ae81e03e1a6d1137b6269be not found: ID does not exist"
Jul 09 10:31:12.911300 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.911274    2577 scope.go:117] "RemoveContainer" containerID="dcb94ae254f5d1b54f98588a2d5bbd56c0dd2bd19006cbd873fafe44932f3a1f"
Jul 09 10:31:12.911494 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:31:12.911476    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcb94ae254f5d1b54f98588a2d5bbd56c0dd2bd19006cbd873fafe44932f3a1f\": container with ID starting with dcb94ae254f5d1b54f98588a2d5bbd56c0dd2bd19006cbd873fafe44932f3a1f not found: ID does not exist" containerID="dcb94ae254f5d1b54f98588a2d5bbd56c0dd2bd19006cbd873fafe44932f3a1f"
Jul 09 10:31:12.911547 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.911504    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcb94ae254f5d1b54f98588a2d5bbd56c0dd2bd19006cbd873fafe44932f3a1f"} err="failed to get container status \"dcb94ae254f5d1b54f98588a2d5bbd56c0dd2bd19006cbd873fafe44932f3a1f\": rpc error: code = NotFound desc = could not find container \"dcb94ae254f5d1b54f98588a2d5bbd56c0dd2bd19006cbd873fafe44932f3a1f\": container with ID starting with dcb94ae254f5d1b54f98588a2d5bbd56c0dd2bd19006cbd873fafe44932f3a1f not found: ID does not exist"
Jul 09 10:31:12.942003 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.941981    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"]
Jul 09 10:31:12.958057 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:12.958038    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-5c94c9d987-lnnd7"]
Jul 09 10:31:13.554015 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:13.552287    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf4b03d6-1a9f-41b8-9823-d8d0eda95632" path="/var/lib/kubelet/pods/bf4b03d6-1a9f-41b8-9823-d8d0eda95632/volumes"
Jul 09 10:31:27.807879 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:27.807850    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:31:39.692243 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:39.692200    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:31:39.910661 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:39.910589    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"]
Jul 09 10:31:39.911000 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:39.910984    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf4b03d6-1a9f-41b8-9823-d8d0eda95632" containerName="main"
Jul 09 10:31:39.911000 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:39.910999    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf4b03d6-1a9f-41b8-9823-d8d0eda95632" containerName="main"
Jul 09 10:31:39.911158 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:39.911025    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf4b03d6-1a9f-41b8-9823-d8d0eda95632" containerName="storage-initializer"
Jul 09 10:31:39.911158 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:39.911032    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf4b03d6-1a9f-41b8-9823-d8d0eda95632" containerName="storage-initializer"
Jul 09 10:31:39.911158 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:39.911110    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf4b03d6-1a9f-41b8-9823-d8d0eda95632" containerName="main"
Jul 09 10:31:39.914328 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:39.914306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:39.917003 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:39.916983    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"prestop-hook-test-epp-sa-dockercfg-jcsxk\""
Jul 09 10:31:39.917758 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:39.917742    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"prestop-hook-test-kserve-self-signed-certs\""
Jul 09 10:31:39.931611 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:39.931588    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"]
Jul 09 10:31:39.957503 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:39.957447    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:31:40.004750 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.004727    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-kserve-provision-location\") pod \"prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:40.004879 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.004758    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-tokenizer-uds\") pod \"prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:40.004879 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.004815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxt64\" (UniqueName: \"kubernetes.io/projected/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-kube-api-access-dxt64\") pod \"prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:40.004974 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.004914    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:40.105583 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.105555    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:40.105691 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.105602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-kserve-provision-location\") pod \"prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:40.105754 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.105732    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-tokenizer-uds\") pod \"prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:40.105836 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.105818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxt64\" (UniqueName: \"kubernetes.io/projected/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-kube-api-access-dxt64\") pod \"prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:40.105970 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.105954    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-kserve-provision-location\") pod \"prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:40.106090 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.106070    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-tokenizer-uds\") pod \"prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:40.108013 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.107996    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:40.116614 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.116593    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxt64\" (UniqueName: \"kubernetes.io/projected/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-kube-api-access-dxt64\") pod \"prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:40.223184 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.223162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:40.365562 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:40.365537    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"]
Jul 09 10:31:40.367591 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:31:40.367563    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be3ddf2_9b88_4328_b6f1_2e0fe83345d1.slice/crio-30377ccb8ab57c3c0d6d99729066559e632296170c33d0a0a7899d38427373ed WatchSource:0}: Error finding container 30377ccb8ab57c3c0d6d99729066559e632296170c33d0a0a7899d38427373ed: Status 404 returned error can't find the container with id 30377ccb8ab57c3c0d6d99729066559e632296170c33d0a0a7899d38427373ed
Jul 09 10:31:41.041704 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:41.041665    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87" event={"ID":"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1","Type":"ContainerStarted","Data":"907eebf2f9f6c0cac500f8653467d455cfd60671a79c6f22978359f1aad266df"}
Jul 09 10:31:41.042110 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:41.041751    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87" event={"ID":"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1","Type":"ContainerStarted","Data":"30377ccb8ab57c3c0d6d99729066559e632296170c33d0a0a7899d38427373ed"}
Jul 09 10:31:42.048323 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:42.048289    2577 generic.go:356] "Generic (PLEG): container finished" podID="5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" containerID="907eebf2f9f6c0cac500f8653467d455cfd60671a79c6f22978359f1aad266df" exitCode=0
Jul 09 10:31:42.048818 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:42.048354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87" event={"ID":"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1","Type":"ContainerDied","Data":"907eebf2f9f6c0cac500f8653467d455cfd60671a79c6f22978359f1aad266df"}
Jul 09 10:31:43.054954 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:43.054924    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87" event={"ID":"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1","Type":"ContainerStarted","Data":"383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721"}
Jul 09 10:31:43.055336 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:43.055053    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:31:43.077551 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:43.077502    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87" podStartSLOduration=3.420114037 podStartE2EDuration="4.07748701s" totalImagesPullingTime="0s" totalInitContainerRuntime="657.372973ms" isStatefulPod=false podCreationTimestamp="2026-07-09 10:31:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:31:43.077335363 +0000 UTC m=+3506.083504491" watchObservedRunningTime="2026-07-09 10:31:43.07748701 +0000 UTC m=+3506.083656081"
Jul 09 10:31:47.054226 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:31:47.054191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:32:14.062586 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:32:14.062551    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:32:42.703729 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:32:42.703690    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:33:17.719224 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:33:17.719198    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:33:17.731531 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:33:17.731511    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:34:13.245067 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:13.245024    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:34:24.311378 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.311341    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"]
Jul 09 10:34:24.311852 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.311486    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:34:24.321116 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:34:24.321086    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:34:24.321116 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:34:24.321116 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:34:24.321116 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721" cmd=["/bin/sleep","15"]
Jul 09 10:34:24.321388 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.321130    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87" message="" err=<
Jul 09 10:34:24.321388 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:34:24.321388 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:34:24.321388 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:34:24.321388 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:34:24.321141    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:34:24.321388 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:34:24.321388 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:34:24.321388 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87" podUID="5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" containerName="main" containerID="cri-o://383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721"
Jul 09 10:34:24.321388 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.321182    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87" podUID="5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" containerName="main" containerID="cri-o://383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721" gracePeriod=60
Jul 09 10:34:24.462387 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.462355    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:34:24.581704 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.581681    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:34:24.718845 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.718815    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-tls-certs\") pod \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") "
Jul 09 10:34:24.719012 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.718877    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-tokenizer-uds\") pod \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") "
Jul 09 10:34:24.719012 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.718900    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-kserve-provision-location\") pod \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") "
Jul 09 10:34:24.719012 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.718946    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxt64\" (UniqueName: \"kubernetes.io/projected/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-kube-api-access-dxt64\") pod \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\" (UID: \"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1\") "
Jul 09 10:34:24.719198 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.719137    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" (UID: "5be3ddf2-9b88-4328-b6f1-2e0fe83345d1"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:34:24.719758 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.719727    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" (UID: "5be3ddf2-9b88-4328-b6f1-2e0fe83345d1"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:34:24.720993 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.720970    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" (UID: "5be3ddf2-9b88-4328-b6f1-2e0fe83345d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:34:24.721485 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.721464    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-kube-api-access-dxt64" (OuterVolumeSpecName: "kube-api-access-dxt64") pod "5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" (UID: "5be3ddf2-9b88-4328-b6f1-2e0fe83345d1"). InnerVolumeSpecName "kube-api-access-dxt64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:34:24.819743 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.819673    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:34:24.819743 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.819698    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:34:24.819743 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.819709    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxt64\" (UniqueName: \"kubernetes.io/projected/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-kube-api-access-dxt64\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:34:24.819743 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.819718    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:34:24.934320 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.934292    2577 generic.go:356] "Generic (PLEG): container finished" podID="5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" containerID="383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721" exitCode=0
Jul 09 10:34:24.934461 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.934346    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"
Jul 09 10:34:24.934461 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.934373    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87" event={"ID":"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1","Type":"ContainerDied","Data":"383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721"}
Jul 09 10:34:24.934461 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.934421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87" event={"ID":"5be3ddf2-9b88-4328-b6f1-2e0fe83345d1","Type":"ContainerDied","Data":"30377ccb8ab57c3c0d6d99729066559e632296170c33d0a0a7899d38427373ed"}
Jul 09 10:34:24.934461 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.934440    2577 scope.go:117] "RemoveContainer" containerID="383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721"
Jul 09 10:34:24.948113 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.948092    2577 scope.go:117] "RemoveContainer" containerID="907eebf2f9f6c0cac500f8653467d455cfd60671a79c6f22978359f1aad266df"
Jul 09 10:34:24.956266 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.956244    2577 scope.go:117] "RemoveContainer" containerID="383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721"
Jul 09 10:34:24.956535 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:34:24.956507    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721\": container with ID starting with 383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721 not found: ID does not exist" containerID="383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721"
Jul 09 10:34:24.956642 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.956546    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721"} err="failed to get container status \"383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721\": rpc error: code = NotFound desc = could not find container \"383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721\": container with ID starting with 383496022c99096975945be9e0ef1c70f7fc4d07e32df922836cd4c837046721 not found: ID does not exist"
Jul 09 10:34:24.956642 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.956570    2577 scope.go:117] "RemoveContainer" containerID="907eebf2f9f6c0cac500f8653467d455cfd60671a79c6f22978359f1aad266df"
Jul 09 10:34:24.956900 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:34:24.956876    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"907eebf2f9f6c0cac500f8653467d455cfd60671a79c6f22978359f1aad266df\": container with ID starting with 907eebf2f9f6c0cac500f8653467d455cfd60671a79c6f22978359f1aad266df not found: ID does not exist" containerID="907eebf2f9f6c0cac500f8653467d455cfd60671a79c6f22978359f1aad266df"
Jul 09 10:34:24.956958 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.956906    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"907eebf2f9f6c0cac500f8653467d455cfd60671a79c6f22978359f1aad266df"} err="failed to get container status \"907eebf2f9f6c0cac500f8653467d455cfd60671a79c6f22978359f1aad266df\": rpc error: code = NotFound desc = could not find container \"907eebf2f9f6c0cac500f8653467d455cfd60671a79c6f22978359f1aad266df\": container with ID starting with 907eebf2f9f6c0cac500f8653467d455cfd60671a79c6f22978359f1aad266df not found: ID does not exist"
Jul 09 10:34:24.967933 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.967912    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"]
Jul 09 10:34:24.971447 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:24.971427    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-5496c57d7-2jm87"]
Jul 09 10:34:25.551792 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:25.551760    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" path="/var/lib/kubelet/pods/5be3ddf2-9b88-4328-b6f1-2e0fe83345d1/volumes"
Jul 09 10:34:30.767769 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:30.767735    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:34:30.904279 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:30.904010    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"]
Jul 09 10:34:30.913052 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:34:30.913020    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:34:30.913052 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:34:30.913052 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:34:30.913052 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d" cmd=["/bin/sleep","15"]
Jul 09 10:34:30.913305 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:30.913066    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm" message="" err=<
Jul 09 10:34:30.913305 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:34:30.913305 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:34:30.913305 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:34:30.913305 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:34:30.913079    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:34:30.913305 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:34:30.913305 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:34:30.913305 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm" podUID="b93180fd-f02f-4116-8f62-351948a5115f" containerName="main" containerID="cri-o://fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d"
Jul 09 10:34:30.913305 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:30.913111    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm" podUID="b93180fd-f02f-4116-8f62-351948a5115f" containerName="main" containerID="cri-o://fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d" gracePeriod=60
Jul 09 10:34:31.172125 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.172102    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:34:31.269324 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.269297    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b93180fd-f02f-4116-8f62-351948a5115f-tls-certs\") pod \"b93180fd-f02f-4116-8f62-351948a5115f\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") "
Jul 09 10:34:31.269471 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.269337    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b93180fd-f02f-4116-8f62-351948a5115f-kserve-provision-location\") pod \"b93180fd-f02f-4116-8f62-351948a5115f\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") "
Jul 09 10:34:31.269471 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.269363    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t69s\" (UniqueName: \"kubernetes.io/projected/b93180fd-f02f-4116-8f62-351948a5115f-kube-api-access-7t69s\") pod \"b93180fd-f02f-4116-8f62-351948a5115f\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") "
Jul 09 10:34:31.269548 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.269506    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b93180fd-f02f-4116-8f62-351948a5115f-tokenizer-uds\") pod \"b93180fd-f02f-4116-8f62-351948a5115f\" (UID: \"b93180fd-f02f-4116-8f62-351948a5115f\") "
Jul 09 10:34:31.269775 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.269736    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b93180fd-f02f-4116-8f62-351948a5115f-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "b93180fd-f02f-4116-8f62-351948a5115f" (UID: "b93180fd-f02f-4116-8f62-351948a5115f"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:34:31.270138 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.270116    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b93180fd-f02f-4116-8f62-351948a5115f-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b93180fd-f02f-4116-8f62-351948a5115f" (UID: "b93180fd-f02f-4116-8f62-351948a5115f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:34:31.271388 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.271367    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b93180fd-f02f-4116-8f62-351948a5115f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b93180fd-f02f-4116-8f62-351948a5115f" (UID: "b93180fd-f02f-4116-8f62-351948a5115f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:34:31.271517 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.271497    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b93180fd-f02f-4116-8f62-351948a5115f-kube-api-access-7t69s" (OuterVolumeSpecName: "kube-api-access-7t69s") pod "b93180fd-f02f-4116-8f62-351948a5115f" (UID: "b93180fd-f02f-4116-8f62-351948a5115f"). InnerVolumeSpecName "kube-api-access-7t69s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:34:31.370973 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.370905    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b93180fd-f02f-4116-8f62-351948a5115f-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:34:31.370973 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.370930    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b93180fd-f02f-4116-8f62-351948a5115f-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:34:31.370973 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.370939    2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b93180fd-f02f-4116-8f62-351948a5115f-kserve-provision-location\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:34:31.370973 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.370949    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t69s\" (UniqueName: \"kubernetes.io/projected/b93180fd-f02f-4116-8f62-351948a5115f-kube-api-access-7t69s\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:34:31.973371 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.973338    2577 generic.go:356] "Generic (PLEG): container finished" podID="b93180fd-f02f-4116-8f62-351948a5115f" containerID="fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d" exitCode=0
Jul 09 10:34:31.973744 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.973393    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"
Jul 09 10:34:31.973744 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.973426    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm" event={"ID":"b93180fd-f02f-4116-8f62-351948a5115f","Type":"ContainerDied","Data":"fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d"}
Jul 09 10:34:31.973744 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.973481    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm" event={"ID":"b93180fd-f02f-4116-8f62-351948a5115f","Type":"ContainerDied","Data":"bfaf6390f3eda9058fbfa858249a63ed53f1cda9cf310343df70665104c68622"}
Jul 09 10:34:31.973744 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.973510    2577 scope.go:117] "RemoveContainer" containerID="fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d"
Jul 09 10:34:31.982558 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.982532    2577 scope.go:117] "RemoveContainer" containerID="88dd1ba0aa462bcb2c17b89ea97158a66aa204bf40cd0c0be76403fe993c5be6"
Jul 09 10:34:31.990335 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.990318    2577 scope.go:117] "RemoveContainer" containerID="fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d"
Jul 09 10:34:31.990561 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:34:31.990545    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d\": container with ID starting with fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d not found: ID does not exist" containerID="fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d"
Jul 09 10:34:31.990619 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.990567    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d"} err="failed to get container status \"fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d\": rpc error: code = NotFound desc = could not find container \"fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d\": container with ID starting with fa1f94df9c61903d8886c6cab3e89e7f7b92aa95a7078c5cf7cadc3784f4785d not found: ID does not exist"
Jul 09 10:34:31.990619 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.990582    2577 scope.go:117] "RemoveContainer" containerID="88dd1ba0aa462bcb2c17b89ea97158a66aa204bf40cd0c0be76403fe993c5be6"
Jul 09 10:34:31.990850 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:34:31.990828    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88dd1ba0aa462bcb2c17b89ea97158a66aa204bf40cd0c0be76403fe993c5be6\": container with ID starting with 88dd1ba0aa462bcb2c17b89ea97158a66aa204bf40cd0c0be76403fe993c5be6 not found: ID does not exist" containerID="88dd1ba0aa462bcb2c17b89ea97158a66aa204bf40cd0c0be76403fe993c5be6"
Jul 09 10:34:31.990903 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:31.990860    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88dd1ba0aa462bcb2c17b89ea97158a66aa204bf40cd0c0be76403fe993c5be6"} err="failed to get container status \"88dd1ba0aa462bcb2c17b89ea97158a66aa204bf40cd0c0be76403fe993c5be6\": rpc error: code = NotFound desc = could not find container \"88dd1ba0aa462bcb2c17b89ea97158a66aa204bf40cd0c0be76403fe993c5be6\": container with ID starting with 88dd1ba0aa462bcb2c17b89ea97158a66aa204bf40cd0c0be76403fe993c5be6 not found: ID does not exist"
Jul 09 10:34:32.030183 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:32.030161    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"]
Jul 09 10:34:32.059069 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:32.059050    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sche7mkkm"]
Jul 09 10:34:32.473530 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:32.473503    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:34:33.551609 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:34:33.551576    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b93180fd-f02f-4116-8f62-351948a5115f" path="/var/lib/kubelet/pods/b93180fd-f02f-4116-8f62-351948a5115f/volumes"
Jul 09 10:35:15.231143 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.231105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:35:15.449468 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.449438    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"]
Jul 09 10:35:15.449822 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.449789    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b93180fd-f02f-4116-8f62-351948a5115f" containerName="main"
Jul 09 10:35:15.449822 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.449815    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b93180fd-f02f-4116-8f62-351948a5115f" containerName="main"
Jul 09 10:35:15.449921 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.449832    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" containerName="storage-initializer"
Jul 09 10:35:15.449921 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.449837    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" containerName="storage-initializer"
Jul 09 10:35:15.449921 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.449847    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b93180fd-f02f-4116-8f62-351948a5115f" containerName="storage-initializer"
Jul 09 10:35:15.449921 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.449853    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b93180fd-f02f-4116-8f62-351948a5115f" containerName="storage-initializer"
Jul 09 10:35:15.449921 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.449861    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" containerName="main"
Jul 09 10:35:15.449921 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.449867    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" containerName="main"
Jul 09 10:35:15.450133 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.449929    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b93180fd-f02f-4116-8f62-351948a5115f" containerName="main"
Jul 09 10:35:15.450133 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.449938    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5be3ddf2-9b88-4328-b6f1-2e0fe83345d1" containerName="main"
Jul 09 10:35:15.453022 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.452997    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:15.457826 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.456372    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-epp-sa-dockercfg-z8gds\""
Jul 09 10:35:15.457826 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.456642    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Jul 09 10:35:15.457826 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.456970    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 10:35:15.457826 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.457176    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 10:35:15.457826 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.457391    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 10:35:15.467085 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.467060    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"]
Jul 09 10:35:15.494384 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.494320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc9f6\" (UniqueName: \"kubernetes.io/projected/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-kube-api-access-hc9f6\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x\" (UID: \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:15.494384 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.494361    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x\" (UID: \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:15.494529 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.494405    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x\" (UID: \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:15.524209 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.524184    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:35:15.594840 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.594787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x\" (UID: \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:15.594962 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.594870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9f6\" (UniqueName: \"kubernetes.io/projected/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-kube-api-access-hc9f6\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x\" (UID: \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:15.594962 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.594902    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x\" (UID: \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:15.595221 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.595206    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x\" (UID: \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:15.597145 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.597123    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x\" (UID: \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:15.603933 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.603914    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc9f6\" (UniqueName: \"kubernetes.io/projected/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-kube-api-access-hc9f6\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x\" (UID: \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:15.767311 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.767237    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:15.915750 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:15.915721    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"]
Jul 09 10:35:15.915951 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:35:15.915858    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd12f8b7_a33d_4711_b8e4_cb345f2c7385.slice/crio-ae73f51eec09f3172a9c5c6625b0f358c47c99a6a6e0b7941141c471cce4fe3f WatchSource:0}: Error finding container ae73f51eec09f3172a9c5c6625b0f358c47c99a6a6e0b7941141c471cce4fe3f: Status 404 returned error can't find the container with id ae73f51eec09f3172a9c5c6625b0f358c47c99a6a6e0b7941141c471cce4fe3f
Jul 09 10:35:16.205731 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:16.205653    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x" event={"ID":"fd12f8b7-a33d-4711-b8e4-cb345f2c7385","Type":"ContainerStarted","Data":"e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e"}
Jul 09 10:35:16.205731 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:16.205722    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:16.205942 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:16.205738    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x" event={"ID":"fd12f8b7-a33d-4711-b8e4-cb345f2c7385","Type":"ContainerStarted","Data":"ae73f51eec09f3172a9c5c6625b0f358c47c99a6a6e0b7941141c471cce4fe3f"}
Jul 09 10:35:16.226279 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:16.226231    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x" podStartSLOduration=1.2262168 podStartE2EDuration="1.2262168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:35:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:35:16.224093674 +0000 UTC m=+3719.230262771" watchObservedRunningTime="2026-07-09 10:35:16.2262168 +0000 UTC m=+3719.232385872"
Jul 09 10:35:26.557427 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:26.557385    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:35:32.389510 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:32.389476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:35:32.692728 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:32.692635    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:35:47.213033 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:47.213002    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:48.500953 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:48.500916    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:35:51.510110 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.510070    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:35:51.512743 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.512716    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"]
Jul 09 10:35:51.521756 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:35:51.521726    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:35:51.521756 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:35:51.521756 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:35:51.521756 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e" cmd=["/bin/sleep","15"]
Jul 09 10:35:51.521986 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.521773    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x" message="" err=<
Jul 09 10:35:51.521986 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:35:51.521986 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:35:51.521986 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:35:51.521986 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:35:51.521785    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:35:51.521986 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:35:51.521986 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:35:51.521986 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x" podUID="fd12f8b7-a33d-4711-b8e4-cb345f2c7385" containerName="main" containerID="cri-o://e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e"
Jul 09 10:35:51.521986 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.521848    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x" podUID="fd12f8b7-a33d-4711-b8e4-cb345f2c7385" containerName="main" containerID="cri-o://e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e" gracePeriod=60
Jul 09 10:35:51.587311 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.587205    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:35:51.771885 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.771832    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:51.866971 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.866943    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-tokenizer-uds\") pod \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\" (UID: \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\") "
Jul 09 10:35:51.867102 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.867012    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc9f6\" (UniqueName: \"kubernetes.io/projected/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-kube-api-access-hc9f6\") pod \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\" (UID: \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\") "
Jul 09 10:35:51.867102 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.867041    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-tls-certs\") pod \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\" (UID: \"fd12f8b7-a33d-4711-b8e4-cb345f2c7385\") "
Jul 09 10:35:51.867239 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.867203    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "fd12f8b7-a33d-4711-b8e4-cb345f2c7385" (UID: "fd12f8b7-a33d-4711-b8e4-cb345f2c7385"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:35:51.869017 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.868987    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fd12f8b7-a33d-4711-b8e4-cb345f2c7385" (UID: "fd12f8b7-a33d-4711-b8e4-cb345f2c7385"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:35:51.869132 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.869045    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-kube-api-access-hc9f6" (OuterVolumeSpecName: "kube-api-access-hc9f6") pod "fd12f8b7-a33d-4711-b8e4-cb345f2c7385" (UID: "fd12f8b7-a33d-4711-b8e4-cb345f2c7385"). InnerVolumeSpecName "kube-api-access-hc9f6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:35:51.968051 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.968026    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:35:51.968051 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.968047    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc9f6\" (UniqueName: \"kubernetes.io/projected/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-kube-api-access-hc9f6\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:35:51.968201 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:51.968056    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd12f8b7-a33d-4711-b8e4-cb345f2c7385-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:35:52.398993 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:52.398953    2577 generic.go:356] "Generic (PLEG): container finished" podID="fd12f8b7-a33d-4711-b8e4-cb345f2c7385" containerID="e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e" exitCode=0
Jul 09 10:35:52.399182 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:52.399007    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"
Jul 09 10:35:52.399182 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:52.399003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x" event={"ID":"fd12f8b7-a33d-4711-b8e4-cb345f2c7385","Type":"ContainerDied","Data":"e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e"}
Jul 09 10:35:52.399182 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:52.399096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x" event={"ID":"fd12f8b7-a33d-4711-b8e4-cb345f2c7385","Type":"ContainerDied","Data":"ae73f51eec09f3172a9c5c6625b0f358c47c99a6a6e0b7941141c471cce4fe3f"}
Jul 09 10:35:52.399182 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:52.399125    2577 scope.go:117] "RemoveContainer" containerID="e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e"
Jul 09 10:35:52.409632 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:52.409615    2577 scope.go:117] "RemoveContainer" containerID="e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e"
Jul 09 10:35:52.409921 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:35:52.409902    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e\": container with ID starting with e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e not found: ID does not exist" containerID="e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e"
Jul 09 10:35:52.409978 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:52.409928    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e"} err="failed to get container status \"e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e\": rpc error: code = NotFound desc = could not find container \"e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e\": container with ID starting with e6eb50462a6ab050e997de1a3ea5c6547112c4747bee50c9dbdb3852809a379e not found: ID does not exist"
Jul 09 10:35:52.423321 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:52.423300    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"]
Jul 09 10:35:52.429512 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:52.429488    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfrx45x"]
Jul 09 10:35:53.551809 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:53.551776    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd12f8b7-a33d-4711-b8e4-cb345f2c7385" path="/var/lib/kubelet/pods/fd12f8b7-a33d-4711-b8e4-cb345f2c7385/volumes"
Jul 09 10:35:55.564547 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:35:55.564519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:36:05.709075 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.709046    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"]
Jul 09 10:36:05.709452 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.709394    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd12f8b7-a33d-4711-b8e4-cb345f2c7385" containerName="main"
Jul 09 10:36:05.709452 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.709405    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd12f8b7-a33d-4711-b8e4-cb345f2c7385" containerName="main"
Jul 09 10:36:05.709532 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.709461    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd12f8b7-a33d-4711-b8e4-cb345f2c7385" containerName="main"
Jul 09 10:36:05.712353 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.712334    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.715751 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.715732    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 10:36:05.715855 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.715736    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Jul 09 10:36:05.715855 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.715771    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 10:36:05.715855 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.715739    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 10:36:05.725493 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.725472    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"]
Jul 09 10:36:05.863225 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.863197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-tls-certs\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.863338 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.863230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-dshm\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.863338 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.863248    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmdxm\" (UniqueName: \"kubernetes.io/projected/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-kube-api-access-rmdxm\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.863338 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.863289    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-model-cache\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.863338 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.863314    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-tmp-dir\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.863338 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.863335    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-home\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.964469 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.964449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-tls-certs\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.964576 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.964490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-dshm\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.964576 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.964512    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmdxm\" (UniqueName: \"kubernetes.io/projected/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-kube-api-access-rmdxm\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.964576 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.964554    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-model-cache\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.964688 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.964581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-tmp-dir\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.964688 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.964619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-home\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.965043 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.965013    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-model-cache\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.965043 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.965029    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-home\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.965194 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.965089    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-tmp-dir\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.966827 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.966788    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-dshm\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.966957 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.966942    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-tls-certs\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:05.973291 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:05.973268    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmdxm\" (UniqueName: \"kubernetes.io/projected/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-kube-api-access-rmdxm\") pod \"rolling-upgrade-test-kserve-859df57bcb-h8c88\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:06.022923 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:06.022901    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:06.157761 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:06.157734    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"]
Jul 09 10:36:06.159378 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:36:06.159344    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13f1e70_9ed5_45f3_9ba4_fd8c8d84d832.slice/crio-14857ca618991e1093ddbd93ba776d21e0261ced068422e7039369a234fdef31 WatchSource:0}: Error finding container 14857ca618991e1093ddbd93ba776d21e0261ced068422e7039369a234fdef31: Status 404 returned error can't find the container with id 14857ca618991e1093ddbd93ba776d21e0261ced068422e7039369a234fdef31
Jul 09 10:36:06.476671 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:06.476599    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88" event={"ID":"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832","Type":"ContainerStarted","Data":"07e8d6d6bb4070274e50f86336522c12c9dabc1987b3a286cc3832ff9d3a4f3f"}
Jul 09 10:36:06.476671 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:06.476647    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88" event={"ID":"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832","Type":"ContainerStarted","Data":"14857ca618991e1093ddbd93ba776d21e0261ced068422e7039369a234fdef31"}
Jul 09 10:36:06.496588 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:06.496544    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88" podStartSLOduration=1.496530027 podStartE2EDuration="1.496530027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:36:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:36:06.495347462 +0000 UTC m=+3769.501516552" watchObservedRunningTime="2026-07-09 10:36:06.496530027 +0000 UTC m=+3769.502699097"
Jul 09 10:36:07.023818 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:07.023770    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:16.023526 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:16.023496    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:16.035908 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:16.035885    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:16.541243 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:16.541221    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:20.091633 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.091592    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:36:20.414193 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.414116    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"]
Jul 09 10:36:20.417714 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.417694    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:20.421521 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.421499    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-sa-dockercfg-24vmz\""
Jul 09 10:36:20.421628 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.421499    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs\""
Jul 09 10:36:20.432457 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.432436    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"]
Jul 09 10:36:20.459978 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.459955    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:36:20.570480 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.570454    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8513da35-2b55-49c1-bf17-63034b06e7e0-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs\" (UID: \"8513da35-2b55-49c1-bf17-63034b06e7e0\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:20.570594 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.570490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8513da35-2b55-49c1-bf17-63034b06e7e0-tokenizer-uds\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs\" (UID: \"8513da35-2b55-49c1-bf17-63034b06e7e0\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:20.570594 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.570510    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svrc6\" (UniqueName: \"kubernetes.io/projected/8513da35-2b55-49c1-bf17-63034b06e7e0-kube-api-access-svrc6\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs\" (UID: \"8513da35-2b55-49c1-bf17-63034b06e7e0\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:20.671696 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.671628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8513da35-2b55-49c1-bf17-63034b06e7e0-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs\" (UID: \"8513da35-2b55-49c1-bf17-63034b06e7e0\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:20.671696 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.671674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8513da35-2b55-49c1-bf17-63034b06e7e0-tokenizer-uds\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs\" (UID: \"8513da35-2b55-49c1-bf17-63034b06e7e0\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:20.671858 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.671707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svrc6\" (UniqueName: \"kubernetes.io/projected/8513da35-2b55-49c1-bf17-63034b06e7e0-kube-api-access-svrc6\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs\" (UID: \"8513da35-2b55-49c1-bf17-63034b06e7e0\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:20.672149 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.672130    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8513da35-2b55-49c1-bf17-63034b06e7e0-tokenizer-uds\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs\" (UID: \"8513da35-2b55-49c1-bf17-63034b06e7e0\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:20.674008 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.673989    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8513da35-2b55-49c1-bf17-63034b06e7e0-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs\" (UID: \"8513da35-2b55-49c1-bf17-63034b06e7e0\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:20.680721 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.680697    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svrc6\" (UniqueName: \"kubernetes.io/projected/8513da35-2b55-49c1-bf17-63034b06e7e0-kube-api-access-svrc6\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs\" (UID: \"8513da35-2b55-49c1-bf17-63034b06e7e0\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:20.726184 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.726164    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:20.867383 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:20.864902    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"]
Jul 09 10:36:20.867383 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:36:20.867233    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8513da35_2b55_49c1_bf17_63034b06e7e0.slice/crio-7291a0d63197a7a9bf5bf54ca1626cfc4d4ff17ca122d1d729f151b10a2a2f84 WatchSource:0}: Error finding container 7291a0d63197a7a9bf5bf54ca1626cfc4d4ff17ca122d1d729f151b10a2a2f84: Status 404 returned error can't find the container with id 7291a0d63197a7a9bf5bf54ca1626cfc4d4ff17ca122d1d729f151b10a2a2f84
Jul 09 10:36:21.557738 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:21.557707    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs" event={"ID":"8513da35-2b55-49c1-bf17-63034b06e7e0","Type":"ContainerStarted","Data":"5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0"}
Jul 09 10:36:21.558133 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:21.557751    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs" event={"ID":"8513da35-2b55-49c1-bf17-63034b06e7e0","Type":"ContainerStarted","Data":"7291a0d63197a7a9bf5bf54ca1626cfc4d4ff17ca122d1d729f151b10a2a2f84"}
Jul 09 10:36:21.558133 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:21.557828    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:21.641549 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:21.641503    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs" podStartSLOduration=1.641490159 podStartE2EDuration="1.641490159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:36:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:36:21.622316594 +0000 UTC m=+3784.628485664" watchObservedRunningTime="2026-07-09 10:36:21.641490159 +0000 UTC m=+3784.647659235"
Jul 09 10:36:22.626564 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:22.626531    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:36:22.632433 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:22.632403    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"]
Jul 09 10:36:22.720544 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:22.720514    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:36:27.158735 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:27.158697    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:36:37.653030 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:37.652988    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88" podUID="f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832" containerName="main" containerID="cri-o://07e8d6d6bb4070274e50f86336522c12c9dabc1987b3a286cc3832ff9d3a4f3f" gracePeriod=45
Jul 09 10:36:37.918641 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:37.918588    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:38.010048 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.010023    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-tls-certs\") pod \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") "
Jul 09 10:36:38.010194 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.010067    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-tmp-dir\") pod \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") "
Jul 09 10:36:38.010194 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.010087    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-dshm\") pod \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") "
Jul 09 10:36:38.010194 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.010101    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-home\") pod \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") "
Jul 09 10:36:38.010194 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.010130    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-model-cache\") pod \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") "
Jul 09 10:36:38.010194 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.010158    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmdxm\" (UniqueName: \"kubernetes.io/projected/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-kube-api-access-rmdxm\") pod \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\" (UID: \"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832\") "
Jul 09 10:36:38.010461 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.010369    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-home" (OuterVolumeSpecName: "home") pod "f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832" (UID: "f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:36:38.010461 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.010397    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832" (UID: "f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:36:38.010461 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.010443    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-model-cache" (OuterVolumeSpecName: "model-cache") pod "f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832" (UID: "f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:36:38.012166 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.012146    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-dshm" (OuterVolumeSpecName: "dshm") pod "f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832" (UID: "f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:36:38.012240 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.012222    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-kube-api-access-rmdxm" (OuterVolumeSpecName: "kube-api-access-rmdxm") pod "f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832" (UID: "f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832"). InnerVolumeSpecName "kube-api-access-rmdxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:36:38.012280 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.012256    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832" (UID: "f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:36:38.111078 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.111054    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:36:38.111078 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.111079    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-tmp-dir\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:36:38.111239 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.111090    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-dshm\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:36:38.111239 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.111103    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-home\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:36:38.111239 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.111112    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-model-cache\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:36:38.111239 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.111121    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmdxm\" (UniqueName: \"kubernetes.io/projected/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832-kube-api-access-rmdxm\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:36:38.644980 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.644951    2577 generic.go:356] "Generic (PLEG): container finished" podID="f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832" containerID="07e8d6d6bb4070274e50f86336522c12c9dabc1987b3a286cc3832ff9d3a4f3f" exitCode=0
Jul 09 10:36:38.645142 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.644996    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"
Jul 09 10:36:38.645142 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.645041    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88" event={"ID":"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832","Type":"ContainerDied","Data":"07e8d6d6bb4070274e50f86336522c12c9dabc1987b3a286cc3832ff9d3a4f3f"}
Jul 09 10:36:38.645142 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.645094    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88" event={"ID":"f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832","Type":"ContainerDied","Data":"14857ca618991e1093ddbd93ba776d21e0261ced068422e7039369a234fdef31"}
Jul 09 10:36:38.645142 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.645126    2577 scope.go:117] "RemoveContainer" containerID="07e8d6d6bb4070274e50f86336522c12c9dabc1987b3a286cc3832ff9d3a4f3f"
Jul 09 10:36:38.655062 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.654850    2577 scope.go:117] "RemoveContainer" containerID="07e8d6d6bb4070274e50f86336522c12c9dabc1987b3a286cc3832ff9d3a4f3f"
Jul 09 10:36:38.655286 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:36:38.655097    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07e8d6d6bb4070274e50f86336522c12c9dabc1987b3a286cc3832ff9d3a4f3f\": container with ID starting with 07e8d6d6bb4070274e50f86336522c12c9dabc1987b3a286cc3832ff9d3a4f3f not found: ID does not exist" containerID="07e8d6d6bb4070274e50f86336522c12c9dabc1987b3a286cc3832ff9d3a4f3f"
Jul 09 10:36:38.655286 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.655116    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07e8d6d6bb4070274e50f86336522c12c9dabc1987b3a286cc3832ff9d3a4f3f"} err="failed to get container status \"07e8d6d6bb4070274e50f86336522c12c9dabc1987b3a286cc3832ff9d3a4f3f\": rpc error: code = NotFound desc = could not find container \"07e8d6d6bb4070274e50f86336522c12c9dabc1987b3a286cc3832ff9d3a4f3f\": container with ID starting with 07e8d6d6bb4070274e50f86336522c12c9dabc1987b3a286cc3832ff9d3a4f3f not found: ID does not exist"
Jul 09 10:36:38.666899 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.666880    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"]
Jul 09 10:36:38.671380 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:38.671357    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-h8c88"]
Jul 09 10:36:39.552313 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:39.552280    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832" path="/var/lib/kubelet/pods/f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832/volumes"
Jul 09 10:36:50.230475 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:50.230435    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:36:50.310015 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:50.309986    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:36:52.566063 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:52.566036    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:54.058021 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.057983    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"]
Jul 09 10:36:54.060462 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.060432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:36:54.068135 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:36:54.068109    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:36:54.068135 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:36:54.068135 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:36:54.068135 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0" cmd=["/bin/sleep","15"]
Jul 09 10:36:54.068368 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.068151    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs" message="" err=<
Jul 09 10:36:54.068368 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:36:54.068368 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:36:54.068368 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:36:54.068368 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:36:54.068163    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:36:54.068368 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:36:54.068368 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:36:54.068368 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs" podUID="8513da35-2b55-49c1-bf17-63034b06e7e0" containerName="main" containerID="cri-o://5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0"
Jul 09 10:36:54.068368 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.068198    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs" podUID="8513da35-2b55-49c1-bf17-63034b06e7e0" containerName="main" containerID="cri-o://5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0" gracePeriod=60
Jul 09 10:36:54.148955 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.148932    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:36:54.326184 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.326159    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:54.335587 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.335560    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8513da35-2b55-49c1-bf17-63034b06e7e0-tokenizer-uds\") pod \"8513da35-2b55-49c1-bf17-63034b06e7e0\" (UID: \"8513da35-2b55-49c1-bf17-63034b06e7e0\") "
Jul 09 10:36:54.335693 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.335676    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svrc6\" (UniqueName: \"kubernetes.io/projected/8513da35-2b55-49c1-bf17-63034b06e7e0-kube-api-access-svrc6\") pod \"8513da35-2b55-49c1-bf17-63034b06e7e0\" (UID: \"8513da35-2b55-49c1-bf17-63034b06e7e0\") "
Jul 09 10:36:54.335776 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.335713    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8513da35-2b55-49c1-bf17-63034b06e7e0-tls-certs\") pod \"8513da35-2b55-49c1-bf17-63034b06e7e0\" (UID: \"8513da35-2b55-49c1-bf17-63034b06e7e0\") "
Jul 09 10:36:54.335856 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.335766    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8513da35-2b55-49c1-bf17-63034b06e7e0-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "8513da35-2b55-49c1-bf17-63034b06e7e0" (UID: "8513da35-2b55-49c1-bf17-63034b06e7e0"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:36:54.335953 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.335937    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8513da35-2b55-49c1-bf17-63034b06e7e0-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:36:54.337559 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.337536    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8513da35-2b55-49c1-bf17-63034b06e7e0-kube-api-access-svrc6" (OuterVolumeSpecName: "kube-api-access-svrc6") pod "8513da35-2b55-49c1-bf17-63034b06e7e0" (UID: "8513da35-2b55-49c1-bf17-63034b06e7e0"). InnerVolumeSpecName "kube-api-access-svrc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:36:54.337646 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.337616    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8513da35-2b55-49c1-bf17-63034b06e7e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8513da35-2b55-49c1-bf17-63034b06e7e0" (UID: "8513da35-2b55-49c1-bf17-63034b06e7e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:36:54.436400 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.436368    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svrc6\" (UniqueName: \"kubernetes.io/projected/8513da35-2b55-49c1-bf17-63034b06e7e0-kube-api-access-svrc6\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:36:54.436400 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.436397    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8513da35-2b55-49c1-bf17-63034b06e7e0-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:36:54.731547 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.731513    2577 generic.go:356] "Generic (PLEG): container finished" podID="8513da35-2b55-49c1-bf17-63034b06e7e0" containerID="5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0" exitCode=0
Jul 09 10:36:54.731717 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.731568    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"
Jul 09 10:36:54.731717 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.731577    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs" event={"ID":"8513da35-2b55-49c1-bf17-63034b06e7e0","Type":"ContainerDied","Data":"5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0"}
Jul 09 10:36:54.731717 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.731627    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs" event={"ID":"8513da35-2b55-49c1-bf17-63034b06e7e0","Type":"ContainerDied","Data":"7291a0d63197a7a9bf5bf54ca1626cfc4d4ff17ca122d1d729f151b10a2a2f84"}
Jul 09 10:36:54.731717 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.731653    2577 scope.go:117] "RemoveContainer" containerID="5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0"
Jul 09 10:36:54.741525 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.741508    2577 scope.go:117] "RemoveContainer" containerID="5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0"
Jul 09 10:36:54.741757 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:36:54.741740    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0\": container with ID starting with 5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0 not found: ID does not exist" containerID="5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0"
Jul 09 10:36:54.741832 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.741765    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0"} err="failed to get container status \"5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0\": rpc error: code = NotFound desc = could not find container \"5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0\": container with ID starting with 5ae7f2b2c21fc6fde1944fbecd04cab61665755bc90aa65d1cd01a72b2b3b5f0 not found: ID does not exist"
Jul 09 10:36:54.754008 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.753987    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"]
Jul 09 10:36:54.759658 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:54.759637    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schegqkhs"]
Jul 09 10:36:55.552068 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:36:55.552028    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8513da35-2b55-49c1-bf17-63034b06e7e0" path="/var/lib/kubelet/pods/8513da35-2b55-49c1-bf17-63034b06e7e0/volumes"
Jul 09 10:37:16.953868 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:16.953832    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"]
Jul 09 10:37:16.954341 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:16.954052    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k" podUID="1be7c91f-66f5-49a3-8c7d-8779f1e2f320" containerName="manager" containerID="cri-o://e10bf1cc016f7e977f7374620919e52e1108e98f5a11de2f99ce5e56e5dd6b61" gracePeriod=30
Jul 09 10:37:25.510418 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.510391    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"
Jul 09 10:37:25.554574 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.554542    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-cert\") pod \"1be7c91f-66f5-49a3-8c7d-8779f1e2f320\" (UID: \"1be7c91f-66f5-49a3-8c7d-8779f1e2f320\") "
Jul 09 10:37:25.554574 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.554576    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5qhg\" (UniqueName: \"kubernetes.io/projected/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-kube-api-access-l5qhg\") pod \"1be7c91f-66f5-49a3-8c7d-8779f1e2f320\" (UID: \"1be7c91f-66f5-49a3-8c7d-8779f1e2f320\") "
Jul 09 10:37:25.556592 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.556566    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-kube-api-access-l5qhg" (OuterVolumeSpecName: "kube-api-access-l5qhg") pod "1be7c91f-66f5-49a3-8c7d-8779f1e2f320" (UID: "1be7c91f-66f5-49a3-8c7d-8779f1e2f320"). InnerVolumeSpecName "kube-api-access-l5qhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:37:25.556709 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.556614    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-cert" (OuterVolumeSpecName: "cert") pod "1be7c91f-66f5-49a3-8c7d-8779f1e2f320" (UID: "1be7c91f-66f5-49a3-8c7d-8779f1e2f320"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:37:25.655241 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.655188    2577 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-cert\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:37:25.655241 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.655221    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5qhg\" (UniqueName: \"kubernetes.io/projected/1be7c91f-66f5-49a3-8c7d-8779f1e2f320-kube-api-access-l5qhg\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:37:25.896357 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.896322    2577 generic.go:356] "Generic (PLEG): container finished" podID="1be7c91f-66f5-49a3-8c7d-8779f1e2f320" containerID="e10bf1cc016f7e977f7374620919e52e1108e98f5a11de2f99ce5e56e5dd6b61" exitCode=0
Jul 09 10:37:25.896492 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.896367    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"
Jul 09 10:37:25.896492 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.896394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k" event={"ID":"1be7c91f-66f5-49a3-8c7d-8779f1e2f320","Type":"ContainerDied","Data":"e10bf1cc016f7e977f7374620919e52e1108e98f5a11de2f99ce5e56e5dd6b61"}
Jul 09 10:37:25.896492 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.896449    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-57d57b58b4-5rf4k" event={"ID":"1be7c91f-66f5-49a3-8c7d-8779f1e2f320","Type":"ContainerDied","Data":"8ad0455c98dd28ccba193f4b332fc5cbbfb541564cb53e9399169b4ea11870ff"}
Jul 09 10:37:25.896492 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.896479    2577 scope.go:117] "RemoveContainer" containerID="e10bf1cc016f7e977f7374620919e52e1108e98f5a11de2f99ce5e56e5dd6b61"
Jul 09 10:37:25.908768 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.908754    2577 scope.go:117] "RemoveContainer" containerID="e10bf1cc016f7e977f7374620919e52e1108e98f5a11de2f99ce5e56e5dd6b61"
Jul 09 10:37:25.909024 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:37:25.909004    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e10bf1cc016f7e977f7374620919e52e1108e98f5a11de2f99ce5e56e5dd6b61\": container with ID starting with e10bf1cc016f7e977f7374620919e52e1108e98f5a11de2f99ce5e56e5dd6b61 not found: ID does not exist" containerID="e10bf1cc016f7e977f7374620919e52e1108e98f5a11de2f99ce5e56e5dd6b61"
Jul 09 10:37:25.909074 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.909031    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e10bf1cc016f7e977f7374620919e52e1108e98f5a11de2f99ce5e56e5dd6b61"} err="failed to get container status \"e10bf1cc016f7e977f7374620919e52e1108e98f5a11de2f99ce5e56e5dd6b61\": rpc error: code = NotFound desc = could not find container \"e10bf1cc016f7e977f7374620919e52e1108e98f5a11de2f99ce5e56e5dd6b61\": container with ID starting with e10bf1cc016f7e977f7374620919e52e1108e98f5a11de2f99ce5e56e5dd6b61 not found: ID does not exist"
Jul 09 10:37:25.918324 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.918300    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"]
Jul 09 10:37:25.922668 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:25.922650    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/llmisvc-controller-manager-57d57b58b4-5rf4k"]
Jul 09 10:37:27.552061 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:27.552027    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1be7c91f-66f5-49a3-8c7d-8779f1e2f320" path="/var/lib/kubelet/pods/1be7c91f-66f5-49a3-8c7d-8779f1e2f320/volumes"
Jul 09 10:37:47.967313 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:47.967224    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:37:48.544994 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.544962    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"]
Jul 09 10:37:48.545365 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.545348    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832" containerName="main"
Jul 09 10:37:48.545434 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.545367    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832" containerName="main"
Jul 09 10:37:48.545434 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.545387    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8513da35-2b55-49c1-bf17-63034b06e7e0" containerName="main"
Jul 09 10:37:48.545434 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.545393    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8513da35-2b55-49c1-bf17-63034b06e7e0" containerName="main"
Jul 09 10:37:48.545434 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.545407    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1be7c91f-66f5-49a3-8c7d-8779f1e2f320" containerName="manager"
Jul 09 10:37:48.545434 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.545415    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1be7c91f-66f5-49a3-8c7d-8779f1e2f320" containerName="manager"
Jul 09 10:37:48.545653 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.545474    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8513da35-2b55-49c1-bf17-63034b06e7e0" containerName="main"
Jul 09 10:37:48.545653 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.545485    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1be7c91f-66f5-49a3-8c7d-8779f1e2f320" containerName="manager"
Jul 09 10:37:48.545653 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.545494    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13f1e70-9ed5-45f3-9ba4-fd8c8d84d832" containerName="main"
Jul 09 10:37:48.548494 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.548475    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:37:48.551855 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.551830    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Jul 09 10:37:48.552032 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.551905    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-epp-sa-dockercfg-t74kc\""
Jul 09 10:37:48.552032 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.551929    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 10:37:48.552165 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.552055    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 10:37:48.552165 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.552061    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 10:37:48.561407 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.561385    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"]
Jul 09 10:37:48.572935 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.572914    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:37:48.617641 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.617612    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfp5l\" (UniqueName: \"kubernetes.io/projected/95230bed-0837-4af5-a8a2-40e9e166dee9-kube-api-access-wfp5l\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7\" (UID: \"95230bed-0837-4af5-a8a2-40e9e166dee9\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:37:48.617731 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.617647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/95230bed-0837-4af5-a8a2-40e9e166dee9-tokenizer-uds\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7\" (UID: \"95230bed-0837-4af5-a8a2-40e9e166dee9\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:37:48.617782 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.617726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95230bed-0837-4af5-a8a2-40e9e166dee9-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7\" (UID: \"95230bed-0837-4af5-a8a2-40e9e166dee9\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:37:48.718322 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.718299    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfp5l\" (UniqueName: \"kubernetes.io/projected/95230bed-0837-4af5-a8a2-40e9e166dee9-kube-api-access-wfp5l\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7\" (UID: \"95230bed-0837-4af5-a8a2-40e9e166dee9\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:37:48.718435 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.718329    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/95230bed-0837-4af5-a8a2-40e9e166dee9-tokenizer-uds\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7\" (UID: \"95230bed-0837-4af5-a8a2-40e9e166dee9\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:37:48.718435 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.718360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95230bed-0837-4af5-a8a2-40e9e166dee9-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7\" (UID: \"95230bed-0837-4af5-a8a2-40e9e166dee9\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:37:48.718727 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.718704    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/95230bed-0837-4af5-a8a2-40e9e166dee9-tokenizer-uds\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7\" (UID: \"95230bed-0837-4af5-a8a2-40e9e166dee9\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:37:48.720618 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.720597    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95230bed-0837-4af5-a8a2-40e9e166dee9-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7\" (UID: \"95230bed-0837-4af5-a8a2-40e9e166dee9\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:37:48.728142 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.728118    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfp5l\" (UniqueName: \"kubernetes.io/projected/95230bed-0837-4af5-a8a2-40e9e166dee9-kube-api-access-wfp5l\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7\" (UID: \"95230bed-0837-4af5-a8a2-40e9e166dee9\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:37:48.858131 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.858067    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:37:48.998426 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:48.998404    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"]
Jul 09 10:37:49.000455 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:37:49.000425    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95230bed_0837_4af5_a8a2_40e9e166dee9.slice/crio-801fcadbdb334d3907492e8bdd28c01f0e759f46ba78624912dfed12cb4512a8 WatchSource:0}: Error finding container 801fcadbdb334d3907492e8bdd28c01f0e759f46ba78624912dfed12cb4512a8: Status 404 returned error can't find the container with id 801fcadbdb334d3907492e8bdd28c01f0e759f46ba78624912dfed12cb4512a8
Jul 09 10:37:49.016298 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:49.016273    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7" event={"ID":"95230bed-0837-4af5-a8a2-40e9e166dee9","Type":"ContainerStarted","Data":"801fcadbdb334d3907492e8bdd28c01f0e759f46ba78624912dfed12cb4512a8"}
Jul 09 10:37:50.023951 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:50.023915    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7" event={"ID":"95230bed-0837-4af5-a8a2-40e9e166dee9","Type":"ContainerStarted","Data":"9b3f7929ed1cbac19435d6c01dece5aef6f7d97c38bfb4f2be7e50d2fc774640"}
Jul 09 10:37:50.024416 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:50.024028    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:37:50.045914 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:50.045866    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7" podStartSLOduration=2.045853828 podStartE2EDuration="2.045853828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:37:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:37:50.043900482 +0000 UTC m=+3873.050069593" watchObservedRunningTime="2026-07-09 10:37:50.045853828 +0000 UTC m=+3873.052022898"
Jul 09 10:37:52.411044 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:37:52.411006    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:38:12.921402 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:12.921362    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:38:13.094407 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.094377    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"]
Jul 09 10:38:13.097938 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.097916    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.101033 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.101009    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-custom-template-test-kserve-self-signed-certs\""
Jul 09 10:38:13.101163 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.101035    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-custom-template-test-epp-sa-dockercfg-66dr8\""
Jul 09 10:38:13.107036 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.107007    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"]
Jul 09 10:38:13.202137 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.202072    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/312661f1-ecbc-42aa-9047-e84626b5e2cf-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.202137 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.202103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.202137 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.202131    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.202380 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.202223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-tokenizer-uds\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.202380 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.202284    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbrk7\" (UniqueName: \"kubernetes.io/projected/312661f1-ecbc-42aa-9047-e84626b5e2cf-kube-api-access-zbrk7\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.302969 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.302944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-tokenizer-uds\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.303115 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.302989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbrk7\" (UniqueName: \"kubernetes.io/projected/312661f1-ecbc-42aa-9047-e84626b5e2cf-kube-api-access-zbrk7\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.303115 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.303017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/312661f1-ecbc-42aa-9047-e84626b5e2cf-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.303115 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.303045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.303115 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.303076    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.303426 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.303401    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-tokenizer-uds\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.303543 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.303451    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.303543 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.303479    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.305678 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.305659    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/312661f1-ecbc-42aa-9047-e84626b5e2cf-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.314936 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.314907    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbrk7\" (UniqueName: \"kubernetes.io/projected/312661f1-ecbc-42aa-9047-e84626b5e2cf-kube-api-access-zbrk7\") pod \"scheduler-custom-template-test-kserve-router-scheduler-567j8hmg\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.408386 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.408364    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:13.544034 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:13.543998    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"]
Jul 09 10:38:13.544501 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:38:13.544476    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod312661f1_ecbc_42aa_9047_e84626b5e2cf.slice/crio-944a158742a1661ffb92b35a566486e8df1aa16bce155a8eaa2018dde3099273 WatchSource:0}: Error finding container 944a158742a1661ffb92b35a566486e8df1aa16bce155a8eaa2018dde3099273: Status 404 returned error can't find the container with id 944a158742a1661ffb92b35a566486e8df1aa16bce155a8eaa2018dde3099273
Jul 09 10:38:14.156684 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:14.156639    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg" event={"ID":"312661f1-ecbc-42aa-9047-e84626b5e2cf","Type":"ContainerStarted","Data":"716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c"}
Jul 09 10:38:14.157135 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:14.156700    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:14.157135 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:14.156718    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg" event={"ID":"312661f1-ecbc-42aa-9047-e84626b5e2cf","Type":"ContainerStarted","Data":"944a158742a1661ffb92b35a566486e8df1aa16bce155a8eaa2018dde3099273"}
Jul 09 10:38:14.180242 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:14.180200    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg" podStartSLOduration=1.180185922 podStartE2EDuration="1.180185922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:38:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:38:14.178271735 +0000 UTC m=+3897.184440805" watchObservedRunningTime="2026-07-09 10:38:14.180185922 +0000 UTC m=+3897.186354993"
Jul 09 10:38:14.437260 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:14.437182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:38:17.752532 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:17.752505    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:38:17.765090 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:17.765069    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:38:21.031041 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:21.031010    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:38:29.562951 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:29.562916    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:38:32.999567 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:32.999535    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"]
Jul 09 10:38:33.008663 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:38:33.008631    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:38:33.008663 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:38:33.008663 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:38:33.008663 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="9b3f7929ed1cbac19435d6c01dece5aef6f7d97c38bfb4f2be7e50d2fc774640" cmd=["/bin/sleep","15"]
Jul 09 10:38:33.008922 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.008682    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7" message="" err=<
Jul 09 10:38:33.008922 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:38:33.008922 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:38:33.008922 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:38:33.008922 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:38:33.008701    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:38:33.008922 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:38:33.008922 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:38:33.008922 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7" podUID="95230bed-0837-4af5-a8a2-40e9e166dee9" containerName="main" containerID="cri-o://9b3f7929ed1cbac19435d6c01dece5aef6f7d97c38bfb4f2be7e50d2fc774640"
Jul 09 10:38:33.008922 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.008745    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7" podUID="95230bed-0837-4af5-a8a2-40e9e166dee9" containerName="main" containerID="cri-o://9b3f7929ed1cbac19435d6c01dece5aef6f7d97c38bfb4f2be7e50d2fc774640" gracePeriod=60
Jul 09 10:38:33.023746 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.023723    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:38:33.260265 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.260231    2577 generic.go:356] "Generic (PLEG): container finished" podID="95230bed-0837-4af5-a8a2-40e9e166dee9" containerID="9b3f7929ed1cbac19435d6c01dece5aef6f7d97c38bfb4f2be7e50d2fc774640" exitCode=0
Jul 09 10:38:33.260408 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.260306    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7" event={"ID":"95230bed-0837-4af5-a8a2-40e9e166dee9","Type":"ContainerDied","Data":"9b3f7929ed1cbac19435d6c01dece5aef6f7d97c38bfb4f2be7e50d2fc774640"}
Jul 09 10:38:33.260408 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.260366    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7" event={"ID":"95230bed-0837-4af5-a8a2-40e9e166dee9","Type":"ContainerDied","Data":"801fcadbdb334d3907492e8bdd28c01f0e759f46ba78624912dfed12cb4512a8"}
Jul 09 10:38:33.260408 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.260385    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="801fcadbdb334d3907492e8bdd28c01f0e759f46ba78624912dfed12cb4512a8"
Jul 09 10:38:33.267194 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.267171    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:38:33.354370 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.354344    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95230bed-0837-4af5-a8a2-40e9e166dee9-tls-certs\") pod \"95230bed-0837-4af5-a8a2-40e9e166dee9\" (UID: \"95230bed-0837-4af5-a8a2-40e9e166dee9\") "
Jul 09 10:38:33.354545 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.354382    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/95230bed-0837-4af5-a8a2-40e9e166dee9-tokenizer-uds\") pod \"95230bed-0837-4af5-a8a2-40e9e166dee9\" (UID: \"95230bed-0837-4af5-a8a2-40e9e166dee9\") "
Jul 09 10:38:33.354545 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.354434    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfp5l\" (UniqueName: \"kubernetes.io/projected/95230bed-0837-4af5-a8a2-40e9e166dee9-kube-api-access-wfp5l\") pod \"95230bed-0837-4af5-a8a2-40e9e166dee9\" (UID: \"95230bed-0837-4af5-a8a2-40e9e166dee9\") "
Jul 09 10:38:33.354731 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.354711    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95230bed-0837-4af5-a8a2-40e9e166dee9-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "95230bed-0837-4af5-a8a2-40e9e166dee9" (UID: "95230bed-0837-4af5-a8a2-40e9e166dee9"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:38:33.356538 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.356513    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95230bed-0837-4af5-a8a2-40e9e166dee9-kube-api-access-wfp5l" (OuterVolumeSpecName: "kube-api-access-wfp5l") pod "95230bed-0837-4af5-a8a2-40e9e166dee9" (UID: "95230bed-0837-4af5-a8a2-40e9e166dee9"). InnerVolumeSpecName "kube-api-access-wfp5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:38:33.356680 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.356612    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95230bed-0837-4af5-a8a2-40e9e166dee9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "95230bed-0837-4af5-a8a2-40e9e166dee9" (UID: "95230bed-0837-4af5-a8a2-40e9e166dee9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:38:33.455837 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.455784    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95230bed-0837-4af5-a8a2-40e9e166dee9-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:38:33.455950 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.455842    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/95230bed-0837-4af5-a8a2-40e9e166dee9-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:38:33.455950 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:33.455858    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfp5l\" (UniqueName: \"kubernetes.io/projected/95230bed-0837-4af5-a8a2-40e9e166dee9-kube-api-access-wfp5l\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:38:34.267359 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:34.267323    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"
Jul 09 10:38:34.290343 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:34.290275    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"]
Jul 09 10:38:34.293044 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:34.293020    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87zhjk7"]
Jul 09 10:38:34.572087 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:34.572007    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:38:35.551394 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:35.551357    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95230bed-0837-4af5-a8a2-40e9e166dee9" path="/var/lib/kubelet/pods/95230bed-0837-4af5-a8a2-40e9e166dee9/volumes"
Jul 09 10:38:45.163440 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:45.163409    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:46.913359 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:46.913319    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"]
Jul 09 10:38:46.913990 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:46.913961    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:38:46.922572 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:38:46.922543    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:38:46.922572 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:38:46.922572 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:38:46.922572 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c" cmd=["/bin/sleep","15"]
Jul 09 10:38:46.922768 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:46.922587    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg" message="" err=<
Jul 09 10:38:46.922768 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:38:46.922768 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:38:46.922768 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:38:46.922768 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:38:46.922599    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:38:46.922768 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:38:46.922768 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:38:46.922768 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg" podUID="312661f1-ecbc-42aa-9047-e84626b5e2cf" containerName="main" containerID="cri-o://716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c"
Jul 09 10:38:46.922768 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:46.922640    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg" podUID="312661f1-ecbc-42aa-9047-e84626b5e2cf" containerName="main" containerID="cri-o://716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c" gracePeriod=60
Jul 09 10:38:47.018566 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.018534    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:38:47.182078 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.182049    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:47.254969 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.254947    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-cachi2-cache\") pod \"312661f1-ecbc-42aa-9047-e84626b5e2cf\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") "
Jul 09 10:38:47.255095 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.254981    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-tokenizer-cache\") pod \"312661f1-ecbc-42aa-9047-e84626b5e2cf\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") "
Jul 09 10:38:47.255095 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.255020    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/312661f1-ecbc-42aa-9047-e84626b5e2cf-tls-certs\") pod \"312661f1-ecbc-42aa-9047-e84626b5e2cf\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") "
Jul 09 10:38:47.255095 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.255074    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbrk7\" (UniqueName: \"kubernetes.io/projected/312661f1-ecbc-42aa-9047-e84626b5e2cf-kube-api-access-zbrk7\") pod \"312661f1-ecbc-42aa-9047-e84626b5e2cf\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") "
Jul 09 10:38:47.255240 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.255099    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-tokenizer-uds\") pod \"312661f1-ecbc-42aa-9047-e84626b5e2cf\" (UID: \"312661f1-ecbc-42aa-9047-e84626b5e2cf\") "
Jul 09 10:38:47.255240 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.255196    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "312661f1-ecbc-42aa-9047-e84626b5e2cf" (UID: "312661f1-ecbc-42aa-9047-e84626b5e2cf"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:38:47.255240 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.255212    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-cachi2-cache" (OuterVolumeSpecName: "cachi2-cache") pod "312661f1-ecbc-42aa-9047-e84626b5e2cf" (UID: "312661f1-ecbc-42aa-9047-e84626b5e2cf"). InnerVolumeSpecName "cachi2-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:38:47.255416 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.255398    2577 reconciler_common.go:299] "Volume detached for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-cachi2-cache\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:38:47.255474 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.255419    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-tokenizer-cache\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:38:47.255474 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.255444    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "312661f1-ecbc-42aa-9047-e84626b5e2cf" (UID: "312661f1-ecbc-42aa-9047-e84626b5e2cf"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:38:47.257047 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.257023    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/312661f1-ecbc-42aa-9047-e84626b5e2cf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "312661f1-ecbc-42aa-9047-e84626b5e2cf" (UID: "312661f1-ecbc-42aa-9047-e84626b5e2cf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:38:47.257164 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.257131    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/312661f1-ecbc-42aa-9047-e84626b5e2cf-kube-api-access-zbrk7" (OuterVolumeSpecName: "kube-api-access-zbrk7") pod "312661f1-ecbc-42aa-9047-e84626b5e2cf" (UID: "312661f1-ecbc-42aa-9047-e84626b5e2cf"). InnerVolumeSpecName "kube-api-access-zbrk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:38:47.336202 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.336177    2577 generic.go:356] "Generic (PLEG): container finished" podID="312661f1-ecbc-42aa-9047-e84626b5e2cf" containerID="716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c" exitCode=0
Jul 09 10:38:47.336323 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.336225    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"
Jul 09 10:38:47.336323 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.336256    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg" event={"ID":"312661f1-ecbc-42aa-9047-e84626b5e2cf","Type":"ContainerDied","Data":"716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c"}
Jul 09 10:38:47.336323 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.336304    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg" event={"ID":"312661f1-ecbc-42aa-9047-e84626b5e2cf","Type":"ContainerDied","Data":"944a158742a1661ffb92b35a566486e8df1aa16bce155a8eaa2018dde3099273"}
Jul 09 10:38:47.336499 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.336325    2577 scope.go:117] "RemoveContainer" containerID="716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c"
Jul 09 10:38:47.347860 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.347837    2577 scope.go:117] "RemoveContainer" containerID="716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c"
Jul 09 10:38:47.348202 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:38:47.348185    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c\": container with ID starting with 716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c not found: ID does not exist" containerID="716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c"
Jul 09 10:38:47.348266 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.348208    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c"} err="failed to get container status \"716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c\": rpc error: code = NotFound desc = could not find container \"716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c\": container with ID starting with 716e36517f34283276ee5c15d29ca94ee6203c4d15e820f7a48da0ba6e09955c not found: ID does not exist"
Jul 09 10:38:47.356088 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.356070    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/312661f1-ecbc-42aa-9047-e84626b5e2cf-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:38:47.356088 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.356088    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbrk7\" (UniqueName: \"kubernetes.io/projected/312661f1-ecbc-42aa-9047-e84626b5e2cf-kube-api-access-zbrk7\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:38:47.356215 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.356098    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/312661f1-ecbc-42aa-9047-e84626b5e2cf-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:38:47.366954 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.366934    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"]
Jul 09 10:38:47.369730 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.369709    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-567j8hmg"]
Jul 09 10:38:47.552833 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:38:47.552781    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="312661f1-ecbc-42aa-9047-e84626b5e2cf" path="/var/lib/kubelet/pods/312661f1-ecbc-42aa-9047-e84626b5e2cf/volumes"
Jul 09 10:39:04.805553 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:04.805514    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:39:05.048131 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.048100    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"]
Jul 09 10:39:05.048437 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.048425    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95230bed-0837-4af5-a8a2-40e9e166dee9" containerName="main"
Jul 09 10:39:05.048482 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.048439    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="95230bed-0837-4af5-a8a2-40e9e166dee9" containerName="main"
Jul 09 10:39:05.048482 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.048456    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="312661f1-ecbc-42aa-9047-e84626b5e2cf" containerName="main"
Jul 09 10:39:05.048482 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.048461    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="312661f1-ecbc-42aa-9047-e84626b5e2cf" containerName="main"
Jul 09 10:39:05.048573 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.048521    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="95230bed-0837-4af5-a8a2-40e9e166dee9" containerName="main"
Jul 09 10:39:05.048573 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.048532    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="312661f1-ecbc-42aa-9047-e84626b5e2cf" containerName="main"
Jul 09 10:39:05.052959 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.052934    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:05.057043 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.056991    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 10:39:05.057931 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.057911    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Jul 09 10:39:05.057998 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.057978    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-gdjrd\""
Jul 09 10:39:05.058089 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.058074    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vkhz8\""
Jul 09 10:39:05.059024 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.059004    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 10:39:05.098762 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.098731    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"]
Jul 09 10:39:05.145451 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.145415    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:39:05.177695 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.177668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k65hv\" (UniqueName: \"kubernetes.io/projected/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-kube-api-access-k65hv\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk\" (UID: \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:05.177866 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.177749    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk\" (UID: \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:05.177866 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.177837    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk\" (UID: \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:05.279101 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.279072    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k65hv\" (UniqueName: \"kubernetes.io/projected/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-kube-api-access-k65hv\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk\" (UID: \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:05.279242 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.279116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk\" (UID: \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:05.279324 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.279237    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk\" (UID: \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:05.279496 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.279475    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk\" (UID: \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:05.281608 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.281593    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk\" (UID: \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:05.295887 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.295865    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k65hv\" (UniqueName: \"kubernetes.io/projected/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-kube-api-access-k65hv\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk\" (UID: \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:05.362870 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.362790    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:05.532840 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:05.532808    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"]
Jul 09 10:39:05.533959 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:39:05.533935    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2e2c3d_30e6_4ff7_b021_ade7c6e04ff2.slice/crio-b2bbfa15dcec415c83c42e80b87cd79e265c2e57c5055274fb8a1351cfde63f6 WatchSource:0}: Error finding container b2bbfa15dcec415c83c42e80b87cd79e265c2e57c5055274fb8a1351cfde63f6: Status 404 returned error can't find the container with id b2bbfa15dcec415c83c42e80b87cd79e265c2e57c5055274fb8a1351cfde63f6
Jul 09 10:39:06.440397 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:06.440356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk" event={"ID":"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2","Type":"ContainerStarted","Data":"cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2"}
Jul 09 10:39:06.440766 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:06.440416    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk" event={"ID":"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2","Type":"ContainerStarted","Data":"b2bbfa15dcec415c83c42e80b87cd79e265c2e57c5055274fb8a1351cfde63f6"}
Jul 09 10:39:06.440766 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:06.440495    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:06.485149 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:06.485105    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk" podStartSLOduration=1.485088702 podStartE2EDuration="1.485088702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:39:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:39:06.484126589 +0000 UTC m=+3949.490295694" watchObservedRunningTime="2026-07-09 10:39:06.485088702 +0000 UTC m=+3949.491257801"
Jul 09 10:39:13.061861 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:13.061816    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:39:22.064821 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:22.064751    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:39:30.139051 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.139015    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"]
Jul 09 10:39:30.144540 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.144516    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.147583 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.147558    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Jul 09 10:39:30.148380 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.148364    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-pd-migration-test-kserve-dockercfg-2kxwq\""
Jul 09 10:39:30.158343 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.158321    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"]
Jul 09 10:39:30.257852 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.257819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b482f622-c78c-48a8-a968-991a986c44a3-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.257852 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.257852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b482f622-c78c-48a8-a968-991a986c44a3-home\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.258043 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.257871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b482f622-c78c-48a8-a968-991a986c44a3-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.258043 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.257913    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b482f622-c78c-48a8-a968-991a986c44a3-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.258043 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.257976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krt6h\" (UniqueName: \"kubernetes.io/projected/b482f622-c78c-48a8-a968-991a986c44a3-kube-api-access-krt6h\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.258043 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.258004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b482f622-c78c-48a8-a968-991a986c44a3-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.359038 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.359011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b482f622-c78c-48a8-a968-991a986c44a3-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.359187 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.359043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b482f622-c78c-48a8-a968-991a986c44a3-home\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.359187 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.359058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b482f622-c78c-48a8-a968-991a986c44a3-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.359187 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.359075    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b482f622-c78c-48a8-a968-991a986c44a3-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.359187 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.359108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krt6h\" (UniqueName: \"kubernetes.io/projected/b482f622-c78c-48a8-a968-991a986c44a3-kube-api-access-krt6h\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.359187 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.359130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b482f622-c78c-48a8-a968-991a986c44a3-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.359480 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.359433    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b482f622-c78c-48a8-a968-991a986c44a3-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.359536 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.359488    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b482f622-c78c-48a8-a968-991a986c44a3-home\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.359536 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.359519    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b482f622-c78c-48a8-a968-991a986c44a3-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.361374 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.361345    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b482f622-c78c-48a8-a968-991a986c44a3-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.361702 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.361675    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b482f622-c78c-48a8-a968-991a986c44a3-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.371007 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.370981    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krt6h\" (UniqueName: \"kubernetes.io/projected/b482f622-c78c-48a8-a968-991a986c44a3-kube-api-access-krt6h\") pod \"scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm\" (UID: \"b482f622-c78c-48a8-a968-991a986c44a3\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.456486 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.456431    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:30.520076 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.519973    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"]
Jul 09 10:39:30.524222 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.524194    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"
Jul 09 10:39:30.528455 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.528432    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-pd-migration-test-epp-sa-dockercfg-8zthl\""
Jul 09 10:39:30.539227 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.539207    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"]
Jul 09 10:39:30.606671 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.606648    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"]
Jul 09 10:39:30.608754 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:39:30.608730    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb482f622_c78c_48a8_a968_991a986c44a3.slice/crio-edce99ea7b18620303b17305e97451df3a1f0a892ad5826ae054cdffbf15e5c2 WatchSource:0}: Error finding container edce99ea7b18620303b17305e97451df3a1f0a892ad5826ae054cdffbf15e5c2: Status 404 returned error can't find the container with id edce99ea7b18620303b17305e97451df3a1f0a892ad5826ae054cdffbf15e5c2
Jul 09 10:39:30.610706 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.610690    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 10:39:30.660828 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.660790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e-tokenizer-uds\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh\" (UID: \"cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"
Jul 09 10:39:30.660934 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.660848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh\" (UID: \"cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"
Jul 09 10:39:30.660988 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.660929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq9jb\" (UniqueName: \"kubernetes.io/projected/cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e-kube-api-access-pq9jb\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh\" (UID: \"cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"
Jul 09 10:39:30.761547 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.761512    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq9jb\" (UniqueName: \"kubernetes.io/projected/cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e-kube-api-access-pq9jb\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh\" (UID: \"cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"
Jul 09 10:39:30.761680 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.761614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e-tokenizer-uds\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh\" (UID: \"cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"
Jul 09 10:39:30.761680 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.761663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh\" (UID: \"cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"
Jul 09 10:39:30.762012 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.761991    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e-tokenizer-uds\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh\" (UID: \"cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"
Jul 09 10:39:30.764053 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.764032    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh\" (UID: \"cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"
Jul 09 10:39:30.770541 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.770519    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq9jb\" (UniqueName: \"kubernetes.io/projected/cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e-kube-api-access-pq9jb\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh\" (UID: \"cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"
Jul 09 10:39:30.834696 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.834673    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"
Jul 09 10:39:30.990890 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:30.990863    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"]
Jul 09 10:39:30.993548 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:39:30.993514    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf6483ce_a2c5_44ec_bc0e_1e7821f6b49e.slice/crio-454e12d8130f350b2998462f8c8e5bdc9bbe31fade8d27c83ee381435b0fc847 WatchSource:0}: Error finding container 454e12d8130f350b2998462f8c8e5bdc9bbe31fade8d27c83ee381435b0fc847: Status 404 returned error can't find the container with id 454e12d8130f350b2998462f8c8e5bdc9bbe31fade8d27c83ee381435b0fc847
Jul 09 10:39:31.579597 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:31.579570    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm" event={"ID":"b482f622-c78c-48a8-a968-991a986c44a3","Type":"ContainerStarted","Data":"147a11767cc1ba12737d274e18c7057f030c838b43e1e5871a14986e267fcb9e"}
Jul 09 10:39:31.580004 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:31.579632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm" event={"ID":"b482f622-c78c-48a8-a968-991a986c44a3","Type":"ContainerStarted","Data":"edce99ea7b18620303b17305e97451df3a1f0a892ad5826ae054cdffbf15e5c2"}
Jul 09 10:39:31.580004 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:31.579664    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:31.581396 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:31.581368    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh" event={"ID":"cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e","Type":"ContainerStarted","Data":"65e34559ba3e0c3d7076df9ce4d621ce640b32f2748c2da505086c723e2c7bc4"}
Jul 09 10:39:31.581484 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:31.581419    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"
Jul 09 10:39:31.581484 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:31.581435    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh" event={"ID":"cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e","Type":"ContainerStarted","Data":"454e12d8130f350b2998462f8c8e5bdc9bbe31fade8d27c83ee381435b0fc847"}
Jul 09 10:39:31.625353 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:31.625299    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh" podStartSLOduration=1.625279904 podStartE2EDuration="1.625279904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:39:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:39:31.622717438 +0000 UTC m=+3974.628886532" watchObservedRunningTime="2026-07-09 10:39:31.625279904 +0000 UTC m=+3974.631448976"
Jul 09 10:39:31.785489 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:31.785460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:39:31.884194 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:31.884170    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:39:32.588775 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:32.588742    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm" event={"ID":"b482f622-c78c-48a8-a968-991a986c44a3","Type":"ContainerStarted","Data":"12c3af9c47870887bce55b4221f4665c093174adc8c41ff500472865eccb06ec"}
Jul 09 10:39:32.615223 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:32.615176    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm" podStartSLOduration=1.804458165 podStartE2EDuration="2.615160154s" totalImagesPullingTime="810.701989ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:39:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:39:32.611906791 +0000 UTC m=+3975.618075883" watchObservedRunningTime="2026-07-09 10:39:32.615160154 +0000 UTC m=+3975.621329226"
Jul 09 10:39:33.457468 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:33.457443    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:37.448134 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:37.448105    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:38.646641 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:38.646603    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:39:40.457405 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:40.457375    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:40.469870 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:40.469848    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:40.640502 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:40.640478    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:41.647188 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:41.647163    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm"
Jul 09 10:39:42.991790 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:42.991754    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:39:43.020229 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.020201    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"]
Jul 09 10:39:43.029010 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:39:43.028979    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:39:43.029010 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:39:43.029010 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:39:43.029010 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2" cmd=["/bin/sleep","15"]
Jul 09 10:39:43.029227 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.029024    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk" message="" err=<
Jul 09 10:39:43.029227 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:39:43.029227 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:39:43.029227 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:39:43.029227 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:39:43.029037    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:39:43.029227 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:39:43.029227 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:39:43.029227 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk" podUID="5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2" containerName="main" containerID="cri-o://cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2"
Jul 09 10:39:43.029227 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.029077    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk" podUID="5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2" containerName="main" containerID="cri-o://cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2" gracePeriod=60
Jul 09 10:39:43.117117 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.117089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:39:43.287422 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.287403    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:43.463728 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.463701    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k65hv\" (UniqueName: \"kubernetes.io/projected/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-kube-api-access-k65hv\") pod \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\" (UID: \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\") "
Jul 09 10:39:43.463920 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.463750    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-tokenizer-uds\") pod \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\" (UID: \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\") "
Jul 09 10:39:43.463920 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.463789    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-tls-certs\") pod \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\" (UID: \"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2\") "
Jul 09 10:39:43.464048 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.464026    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2" (UID: "5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:39:43.465861 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.465829    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2" (UID: "5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:39:43.465970 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.465882    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-kube-api-access-k65hv" (OuterVolumeSpecName: "kube-api-access-k65hv") pod "5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2" (UID: "5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2"). InnerVolumeSpecName "kube-api-access-k65hv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:39:43.565002 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.564978    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k65hv\" (UniqueName: \"kubernetes.io/projected/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-kube-api-access-k65hv\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:39:43.565106 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.565004    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:39:43.565106 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.565018    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:39:43.646620 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.646594    2577 generic.go:356] "Generic (PLEG): container finished" podID="5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2" containerID="cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2" exitCode=0
Jul 09 10:39:43.646730 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.646635    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"
Jul 09 10:39:43.646730 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.646661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk" event={"ID":"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2","Type":"ContainerDied","Data":"cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2"}
Jul 09 10:39:43.646730 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.646694    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk" event={"ID":"5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2","Type":"ContainerDied","Data":"b2bbfa15dcec415c83c42e80b87cd79e265c2e57c5055274fb8a1351cfde63f6"}
Jul 09 10:39:43.646730 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.646712    2577 scope.go:117] "RemoveContainer" containerID="cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2"
Jul 09 10:39:43.655732 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.655715    2577 scope.go:117] "RemoveContainer" containerID="cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2"
Jul 09 10:39:43.655998 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:39:43.655971    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2\": container with ID starting with cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2 not found: ID does not exist" containerID="cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2"
Jul 09 10:39:43.656099 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.656000    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2"} err="failed to get container status \"cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2\": rpc error: code = NotFound desc = could not find container \"cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2\": container with ID starting with cf84fa49a1d7ce953d99f4a996828112944c14abd5b92cd4de819d14e13947a2 not found: ID does not exist"
Jul 09 10:39:43.666953 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.666931    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"]
Jul 09 10:39:43.673263 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:43.673238    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcgxfk"]
Jul 09 10:39:45.552294 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:39:45.552259    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2" path="/var/lib/kubelet/pods/5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2/volumes"
Jul 09 10:40:02.590594 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:02.590560    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh"
Jul 09 10:40:03.818776 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:03.818742    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:40:04.096014 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.095939    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"]
Jul 09 10:40:04.096294 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.096283    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2" containerName="main"
Jul 09 10:40:04.096346 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.096295    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2" containerName="main"
Jul 09 10:40:04.096378 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.096368    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e2e2c3d-30e6-4ff7-b021-ade7c6e04ff2" containerName="main"
Jul 09 10:40:04.102598 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.102578    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:04.109343 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.109324    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-threshold-migration-test-epp-sa-dockercfg-n97sk\""
Jul 09 10:40:04.109443 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.109364    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\""
Jul 09 10:40:04.113250 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.113228    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"]
Jul 09 10:40:04.146345 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.146323    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:40:04.212966 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.212940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a237e11c-66e6-4ddc-a758-28e91fac446d-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched8c47c\" (UID: \"a237e11c-66e6-4ddc-a758-28e91fac446d\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:04.213063 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.212977    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmrll\" (UniqueName: \"kubernetes.io/projected/a237e11c-66e6-4ddc-a758-28e91fac446d-kube-api-access-kmrll\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched8c47c\" (UID: \"a237e11c-66e6-4ddc-a758-28e91fac446d\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:04.213063 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.213021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a237e11c-66e6-4ddc-a758-28e91fac446d-tokenizer-uds\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched8c47c\" (UID: \"a237e11c-66e6-4ddc-a758-28e91fac446d\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:04.314107 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.314077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a237e11c-66e6-4ddc-a758-28e91fac446d-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched8c47c\" (UID: \"a237e11c-66e6-4ddc-a758-28e91fac446d\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:04.314239 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.314117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmrll\" (UniqueName: \"kubernetes.io/projected/a237e11c-66e6-4ddc-a758-28e91fac446d-kube-api-access-kmrll\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched8c47c\" (UID: \"a237e11c-66e6-4ddc-a758-28e91fac446d\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:04.314239 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.314161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a237e11c-66e6-4ddc-a758-28e91fac446d-tokenizer-uds\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched8c47c\" (UID: \"a237e11c-66e6-4ddc-a758-28e91fac446d\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:04.314612 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.314591    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a237e11c-66e6-4ddc-a758-28e91fac446d-tokenizer-uds\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched8c47c\" (UID: \"a237e11c-66e6-4ddc-a758-28e91fac446d\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:04.316576 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.316553    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a237e11c-66e6-4ddc-a758-28e91fac446d-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched8c47c\" (UID: \"a237e11c-66e6-4ddc-a758-28e91fac446d\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:04.326171 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.326145    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmrll\" (UniqueName: \"kubernetes.io/projected/a237e11c-66e6-4ddc-a758-28e91fac446d-kube-api-access-kmrll\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched8c47c\" (UID: \"a237e11c-66e6-4ddc-a758-28e91fac446d\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:04.412263 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.412204    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:04.552998 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.552976    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"]
Jul 09 10:40:04.554987 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:40:04.554956    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda237e11c_66e6_4ddc_a758_28e91fac446d.slice/crio-770ba50107ba2a2c61976756d2ce39e6c4c942d32e9a7495e9a944a3bfa3df11 WatchSource:0}: Error finding container 770ba50107ba2a2c61976756d2ce39e6c4c942d32e9a7495e9a944a3bfa3df11: Status 404 returned error can't find the container with id 770ba50107ba2a2c61976756d2ce39e6c4c942d32e9a7495e9a944a3bfa3df11
Jul 09 10:40:04.756433 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.756398    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c" event={"ID":"a237e11c-66e6-4ddc-a758-28e91fac446d","Type":"ContainerStarted","Data":"c2f2521fabd293a0869e570efd2eded4c42774be2f43107145a97b486b065c1e"}
Jul 09 10:40:04.756600 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.756447    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c" event={"ID":"a237e11c-66e6-4ddc-a758-28e91fac446d","Type":"ContainerStarted","Data":"770ba50107ba2a2c61976756d2ce39e6c4c942d32e9a7495e9a944a3bfa3df11"}
Jul 09 10:40:04.756600 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.756520    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:04.778688 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:04.778642    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c" podStartSLOduration=0.778629303 podStartE2EDuration="778.629303ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:40:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:40:04.775311479 +0000 UTC m=+4007.781480553" watchObservedRunningTime="2026-07-09 10:40:04.778629303 +0000 UTC m=+4007.784798374"
Jul 09 10:40:16.334720 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:16.334681    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:40:31.737329 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:31.737291    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:40:35.764152 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:35.764124    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:37.687789 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:37.687751    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:40:37.725196 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:37.725160    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"]
Jul 09 10:40:37.734712 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:40:37.734671    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 10:40:37.734712 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:40:37.734712 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:40:37.734712 ip-10-0-132-105 kubenswrapper[2577]:  > containerID="c2f2521fabd293a0869e570efd2eded4c42774be2f43107145a97b486b065c1e" cmd=["/bin/sleep","15"]
Jul 09 10:40:37.734992 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:37.734721    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c" message="" err=<
Jul 09 10:40:37.734992 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:40:37.734992 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:40:37.734992 ip-10-0-132-105 kubenswrapper[2577]:  >
Jul 09 10:40:37.734992 ip-10-0-132-105 kubenswrapper[2577]: E0709 10:40:37.734737    2577 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 09 10:40:37.734992 ip-10-0-132-105 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 10:40:37.734992 ip-10-0-132-105 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 09 10:40:37.734992 ip-10-0-132-105 kubenswrapper[2577]:  > pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c" podUID="a237e11c-66e6-4ddc-a758-28e91fac446d" containerName="main" containerID="cri-o://c2f2521fabd293a0869e570efd2eded4c42774be2f43107145a97b486b065c1e"
Jul 09 10:40:37.734992 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:37.734770    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c" podUID="a237e11c-66e6-4ddc-a758-28e91fac446d" containerName="main" containerID="cri-o://c2f2521fabd293a0869e570efd2eded4c42774be2f43107145a97b486b065c1e" gracePeriod=60
Jul 09 10:40:37.808821 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:37.808778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:40:37.942526 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:37.942452    2577 generic.go:356] "Generic (PLEG): container finished" podID="a237e11c-66e6-4ddc-a758-28e91fac446d" containerID="c2f2521fabd293a0869e570efd2eded4c42774be2f43107145a97b486b065c1e" exitCode=0
Jul 09 10:40:37.942526 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:37.942488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c" event={"ID":"a237e11c-66e6-4ddc-a758-28e91fac446d","Type":"ContainerDied","Data":"c2f2521fabd293a0869e570efd2eded4c42774be2f43107145a97b486b065c1e"}
Jul 09 10:40:37.981793 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:37.981771    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:38.054463 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.054433    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmrll\" (UniqueName: \"kubernetes.io/projected/a237e11c-66e6-4ddc-a758-28e91fac446d-kube-api-access-kmrll\") pod \"a237e11c-66e6-4ddc-a758-28e91fac446d\" (UID: \"a237e11c-66e6-4ddc-a758-28e91fac446d\") "
Jul 09 10:40:38.054595 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.054471    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a237e11c-66e6-4ddc-a758-28e91fac446d-tls-certs\") pod \"a237e11c-66e6-4ddc-a758-28e91fac446d\" (UID: \"a237e11c-66e6-4ddc-a758-28e91fac446d\") "
Jul 09 10:40:38.054595 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.054504    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a237e11c-66e6-4ddc-a758-28e91fac446d-tokenizer-uds\") pod \"a237e11c-66e6-4ddc-a758-28e91fac446d\" (UID: \"a237e11c-66e6-4ddc-a758-28e91fac446d\") "
Jul 09 10:40:38.054827 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.054787    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a237e11c-66e6-4ddc-a758-28e91fac446d-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "a237e11c-66e6-4ddc-a758-28e91fac446d" (UID: "a237e11c-66e6-4ddc-a758-28e91fac446d"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 10:40:38.056577 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.056552    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a237e11c-66e6-4ddc-a758-28e91fac446d-kube-api-access-kmrll" (OuterVolumeSpecName: "kube-api-access-kmrll") pod "a237e11c-66e6-4ddc-a758-28e91fac446d" (UID: "a237e11c-66e6-4ddc-a758-28e91fac446d"). InnerVolumeSpecName "kube-api-access-kmrll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 10:40:38.056694 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.056660    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a237e11c-66e6-4ddc-a758-28e91fac446d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a237e11c-66e6-4ddc-a758-28e91fac446d" (UID: "a237e11c-66e6-4ddc-a758-28e91fac446d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 10:40:38.155747 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.155724    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmrll\" (UniqueName: \"kubernetes.io/projected/a237e11c-66e6-4ddc-a758-28e91fac446d-kube-api-access-kmrll\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:40:38.155747 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.155745    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a237e11c-66e6-4ddc-a758-28e91fac446d-tls-certs\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:40:38.155908 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.155755    2577 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a237e11c-66e6-4ddc-a758-28e91fac446d-tokenizer-uds\") on node \"ip-10-0-132-105.ec2.internal\" DevicePath \"\""
Jul 09 10:40:38.949208 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.949178    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c" event={"ID":"a237e11c-66e6-4ddc-a758-28e91fac446d","Type":"ContainerDied","Data":"770ba50107ba2a2c61976756d2ce39e6c4c942d32e9a7495e9a944a3bfa3df11"}
Jul 09 10:40:38.949208 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.949204    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"
Jul 09 10:40:38.949671 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.949238    2577 scope.go:117] "RemoveContainer" containerID="c2f2521fabd293a0869e570efd2eded4c42774be2f43107145a97b486b065c1e"
Jul 09 10:40:38.977745 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.977719    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"]
Jul 09 10:40:38.986904 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:38.986883    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched8c47c"]
Jul 09 10:40:39.552306 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:40:39.552276    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a237e11c-66e6-4ddc-a758-28e91fac446d" path="/var/lib/kubelet/pods/a237e11c-66e6-4ddc-a758-28e91fac446d/volumes"
Jul 09 10:43:16.733716 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:43:16.733678    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:43:17.789311 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:43:17.789286    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:43:17.803120 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:43:17.803097    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:44:04.305039 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:44:04.304937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:44:04.404242 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:44:04.404200    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:44:24.258289 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:44:24.258257    2577 scope.go:117] "RemoveContainer" containerID="9b3f7929ed1cbac19435d6c01dece5aef6f7d97c38bfb4f2be7e50d2fc774640"
Jul 09 10:46:09.609561 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:46:09.609531    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:48:03.244533 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:48:03.244500    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:48:17.828732 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:48:17.828687    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:48:17.841005 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:48:17.840982    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:50:32.725354 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:50:32.725316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:53:17.866037 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:53:17.866009    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:53:17.886300 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:53:17.886272    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:54:59.284651 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:54:59.284616    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:58:17.909988 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:17.909888    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:58:17.924179 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:17.924157    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:58:32.584793 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:32.584760    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:58:32.593971 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:32.593927    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:58:32.628405 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:32.628369    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:58:37.416297 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:37.416265    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:58:37.471358 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:37.471331    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:58:51.619261 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:51.619143    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:58:51.684458 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:51.684431    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:58:51.698370 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:51.698348    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:58:51.751634 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:51.751609    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:58:53.178231 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:53.178202    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:58:53.234964 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:53.234938    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:58:53.248597 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:53.248578    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:58:53.285790 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:53.285766    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:58:54.711119 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:54.711090    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:58:54.765992 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:54.765958    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:58:54.779635 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:54.779611    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:58:54.816212 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:54.816185    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:58:56.251196 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:56.251164    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:58:56.302383 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:56.302364    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:58:56.315974 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:56.315950    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:58:56.351917 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:56.351897    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:58:57.815351 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:57.815327    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:58:57.864342 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:57.864320    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:58:57.877011 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:57.876974    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:58:57.911216 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:57.911197    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:58:59.331571 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:59.331544    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:58:59.384376 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:59.384352    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:58:59.398706 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:59.398684    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:58:59.435981 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:58:59.435955    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:59:00.858416 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:00.858388    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:59:00.910252 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:00.910225    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:59:00.923010 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:00.922987    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:59:00.960575 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:00.960554    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:59:02.417982 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:02.417954    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:59:02.474303 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:02.474278    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:59:02.487891 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:02.487868    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:59:02.527711 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:02.527686    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:59:04.031166 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:04.031141    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:59:04.086374 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:04.086326    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:59:04.101777 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:04.101747    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:59:04.140474 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:04.140450    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:59:05.654113 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:05.654086    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:59:05.726918 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:05.726889    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:59:05.740771 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:05.740746    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:59:05.776889 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:05.776868    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:59:07.220106 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:07.220076    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:59:07.271398 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:07.271362    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:59:07.284221 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:07.284195    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:59:07.319140 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:07.319118    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:59:08.754129 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:08.754093    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:59:08.806491 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:08.806467    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:59:08.822563 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:08.822536    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:59:08.858067 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:08.858042    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:59:10.275098 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:10.275067    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:59:10.326546 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:10.326520    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:59:10.340737 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:10.340717    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:59:10.376740 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:10.376700    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:59:11.837611 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:11.837586    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:59:11.890903 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:11.890881    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:59:11.903647 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:11.903623    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:59:11.938234 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:11.938217    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:59:13.339666 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:13.339639    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:59:13.392190 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:13.392166    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:59:13.405005 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:13.404983    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:59:13.442675 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:13.442653    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:59:14.845874 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:14.845832    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-xvtw2_773ee706-6b79-4a33-a277-dc678f4e6518/storage-initializer/0.log"
Jul 09 10:59:14.892296 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:14.892252    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/main/0.log"
Jul 09 10:59:14.904270 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:14.904242    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-5c87756974-4xxsm_b482f622-c78c-48a8-a968-991a986c44a3/llm-d-routing-sidecar/0.log"
Jul 09 10:59:14.942602 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:14.942554    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-59wqtrh_cf6483ce-a2c5-44ec-bc0e-1e7821f6b49e/main/0.log"
Jul 09 10:59:16.746420 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:16.746392    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-m2d5h_5581b14c-ab85-4db6-9dad-987ab3835f48/discovery/0.log"
Jul 09 10:59:16.795488 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:16.795463    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c7f6c5c4f-cjdpd_0576329a-dc3f-459e-a951-f2b0aed69679/router/0.log"
Jul 09 10:59:17.800702 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:17.800668    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:59:18.066939 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:18.066871    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-m2d5h_5581b14c-ab85-4db6-9dad-987ab3835f48/discovery/0.log"
Jul 09 10:59:18.143000 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:18.142973    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c7f6c5c4f-cjdpd_0576329a-dc3f-459e-a951-f2b0aed69679/router/0.log"
Jul 09 10:59:19.425088 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:19.425048    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-m2d5h_5581b14c-ab85-4db6-9dad-987ab3835f48/discovery/0.log"
Jul 09 10:59:19.479001 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:19.478974    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c7f6c5c4f-cjdpd_0576329a-dc3f-459e-a951-f2b0aed69679/router/0.log"
Jul 09 10:59:20.713321 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:20.713292    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-m2d5h_5581b14c-ab85-4db6-9dad-987ab3835f48/discovery/0.log"
Jul 09 10:59:20.768600 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:20.768573    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c7f6c5c4f-cjdpd_0576329a-dc3f-459e-a951-f2b0aed69679/router/0.log"
Jul 09 10:59:21.727959 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:21.727926    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-qm9ts_8be0106f-0797-40a0-ab86-6d47c804051a/manager/0.log"
Jul 09 10:59:21.889715 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:21.889680    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-9btlm_89197f43-f076-44a8-8583-9a86b622891d/manager/0.log"
Jul 09 10:59:21.921275 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:21.921256    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-g7cbp_31924cb6-d969-46ed-a196-dce7f02be8ed/manager/0.log"
Jul 09 10:59:22.972943 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:22.972915    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-qm9ts_8be0106f-0797-40a0-ab86-6d47c804051a/manager/0.log"
Jul 09 10:59:23.133619 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:23.133590    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-9btlm_89197f43-f076-44a8-8583-9a86b622891d/manager/0.log"
Jul 09 10:59:23.166492 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:23.166465    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-g7cbp_31924cb6-d969-46ed-a196-dce7f02be8ed/manager/0.log"
Jul 09 10:59:24.204559 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:24.204528    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-qm9ts_8be0106f-0797-40a0-ab86-6d47c804051a/manager/0.log"
Jul 09 10:59:24.357285 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:24.357253    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-9btlm_89197f43-f076-44a8-8583-9a86b622891d/manager/0.log"
Jul 09 10:59:24.385730 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:24.385697    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-g7cbp_31924cb6-d969-46ed-a196-dce7f02be8ed/manager/0.log"
Jul 09 10:59:25.406982 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:25.406950    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-qm9ts_8be0106f-0797-40a0-ab86-6d47c804051a/manager/0.log"
Jul 09 10:59:25.559314 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:25.559286    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-9btlm_89197f43-f076-44a8-8583-9a86b622891d/manager/0.log"
Jul 09 10:59:25.587026 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:25.586998    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-g7cbp_31924cb6-d969-46ed-a196-dce7f02be8ed/manager/0.log"
Jul 09 10:59:26.611021 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:26.610994    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-qm9ts_8be0106f-0797-40a0-ab86-6d47c804051a/manager/0.log"
Jul 09 10:59:26.773642 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:26.773616    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-9btlm_89197f43-f076-44a8-8583-9a86b622891d/manager/0.log"
Jul 09 10:59:26.800412 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:26.800384    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-g7cbp_31924cb6-d969-46ed-a196-dce7f02be8ed/manager/0.log"
Jul 09 10:59:27.841877 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:27.841848    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-qm9ts_8be0106f-0797-40a0-ab86-6d47c804051a/manager/0.log"
Jul 09 10:59:27.997390 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:27.997363    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-9btlm_89197f43-f076-44a8-8583-9a86b622891d/manager/0.log"
Jul 09 10:59:28.027174 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:28.027145    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-g7cbp_31924cb6-d969-46ed-a196-dce7f02be8ed/manager/0.log"
Jul 09 10:59:29.060834 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:29.060788    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-qm9ts_8be0106f-0797-40a0-ab86-6d47c804051a/manager/0.log"
Jul 09 10:59:29.220290 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:29.220265    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-9btlm_89197f43-f076-44a8-8583-9a86b622891d/manager/0.log"
Jul 09 10:59:29.248715 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:29.248687    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-g7cbp_31924cb6-d969-46ed-a196-dce7f02be8ed/manager/0.log"
Jul 09 10:59:30.288119 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:30.288087    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-qm9ts_8be0106f-0797-40a0-ab86-6d47c804051a/manager/0.log"
Jul 09 10:59:30.444901 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:30.444870    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-9btlm_89197f43-f076-44a8-8583-9a86b622891d/manager/0.log"
Jul 09 10:59:30.493197 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:30.493171    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-g7cbp_31924cb6-d969-46ed-a196-dce7f02be8ed/manager/0.log"
Jul 09 10:59:31.539053 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:31.539021    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-qm9ts_8be0106f-0797-40a0-ab86-6d47c804051a/manager/0.log"
Jul 09 10:59:31.698545 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:31.698511    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-9btlm_89197f43-f076-44a8-8583-9a86b622891d/manager/0.log"
Jul 09 10:59:31.726491 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:31.726469    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-g7cbp_31924cb6-d969-46ed-a196-dce7f02be8ed/manager/0.log"
Jul 09 10:59:32.751210 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:32.751180    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-qm9ts_8be0106f-0797-40a0-ab86-6d47c804051a/manager/0.log"
Jul 09 10:59:32.906070 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:32.906040    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-9btlm_89197f43-f076-44a8-8583-9a86b622891d/manager/0.log"
Jul 09 10:59:32.935484 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:32.935459    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-g7cbp_31924cb6-d969-46ed-a196-dce7f02be8ed/manager/0.log"
Jul 09 10:59:33.962901 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:33.962873    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-qm9ts_8be0106f-0797-40a0-ab86-6d47c804051a/manager/0.log"
Jul 09 10:59:34.119700 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:34.119672    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-9btlm_89197f43-f076-44a8-8583-9a86b622891d/manager/0.log"
Jul 09 10:59:34.147511 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:34.147483    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-g7cbp_31924cb6-d969-46ed-a196-dce7f02be8ed/manager/0.log"
Jul 09 10:59:36.641692 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:36.641657    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:59:36.808703 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:36.808676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:59:36.920522 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:36.920444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:59:39.527786 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:39.527754    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dh44v_2f492c23-72ed-4bbf-899d-3d5f588ef133/global-pull-secret-syncer/0.log"
Jul 09 10:59:39.642838 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:39.642811    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log"
Jul 09 10:59:39.829461 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:39.829374    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-105.ec2.internal_5098063099125629f18ce56638bbe71b/haproxy/0.log"
Jul 09 10:59:42.833138 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:42.833103    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:59:42.933076 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:42.933029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ztl5_216f2721-d98c-411d-952b-dd600dc4642e/konnectivity-agent/0.log" limit=1024
Jul 09 10:59:45.052053 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:45.052024    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-qm9ts_8be0106f-0797-40a0-ab86-6d47c804051a/manager/0.log"
Jul 09 10:59:45.251857 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:45.251785    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-9btlm_89197f43-f076-44a8-8583-9a86b622891d/manager/0.log"
Jul 09 10:59:45.320961 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:45.320897    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-g7cbp_31924cb6-d969-46ed-a196-dce7f02be8ed/manager/0.log"
Jul 09 10:59:46.648749 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:46.648722    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-msxxf_38bfa326-e79b-447e-ae2b-c31b088b6249/cluster-monitoring-operator/0.log"
Jul 09 10:59:47.191840 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:47.191791    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l47fk_e2de9b45-2629-4bc9-b5cc-e7553a1599f4/node-exporter/0.log"
Jul 09 10:59:47.220491 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:47.220469    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l47fk_e2de9b45-2629-4bc9-b5cc-e7553a1599f4/kube-rbac-proxy/0.log"
Jul 09 10:59:47.247168 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:47.247149    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l47fk_e2de9b45-2629-4bc9-b5cc-e7553a1599f4/init-textfile/0.log"
Jul 09 10:59:47.693236 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:47.693210    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d86454dd9-87pl2_89f07c10-f820-4636-9e73-1c71d475c421/telemeter-client/0.log"
Jul 09 10:59:47.724421 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:47.724399    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d86454dd9-87pl2_89f07c10-f820-4636-9e73-1c71d475c421/reload/0.log"
Jul 09 10:59:47.752936 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:47.752919    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d86454dd9-87pl2_89f07c10-f820-4636-9e73-1c71d475c421/kube-rbac-proxy/0.log"
Jul 09 10:59:48.962605 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:48.962574    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-gcwjv_960ec500-4be8-4e41-8f4b-cd8f20938c08/networking-console-plugin/0.log"
Jul 09 10:59:49.458435 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.458355    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/2.log"
Jul 09 10:59:49.462572 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.462551    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ms4r4_38d5ce80-3b12-4729-9e14-510a84a1e8d0/console-operator/3.log"
Jul 09 10:59:49.527479 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.527450    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"]
Jul 09 10:59:49.527989 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.527969    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a237e11c-66e6-4ddc-a758-28e91fac446d" containerName="main"
Jul 09 10:59:49.527989 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.527991    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a237e11c-66e6-4ddc-a758-28e91fac446d" containerName="main"
Jul 09 10:59:49.528141 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.528115    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a237e11c-66e6-4ddc-a758-28e91fac446d" containerName="main"
Jul 09 10:59:49.531255 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.531231    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.533838 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.533818    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d8w7l\"/\"kube-root-ca.crt\""
Jul 09 10:59:49.534505 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.534488    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d8w7l\"/\"openshift-service-ca.crt\""
Jul 09 10:59:49.534596 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.534506    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-d8w7l\"/\"default-dockercfg-zlpx4\""
Jul 09 10:59:49.544660 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.544633    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"]
Jul 09 10:59:49.597319 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.597295    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxt8h\" (UniqueName: \"kubernetes.io/projected/f12c2aee-36ae-470d-b55c-7613ea6ff740-kube-api-access-gxt8h\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.597422 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.597347    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f12c2aee-36ae-470d-b55c-7613ea6ff740-lib-modules\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.597422 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.597392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f12c2aee-36ae-470d-b55c-7613ea6ff740-podres\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.597422 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.597409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f12c2aee-36ae-470d-b55c-7613ea6ff740-proc\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.597582 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.597445    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f12c2aee-36ae-470d-b55c-7613ea6ff740-sys\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.698547 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.698517    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f12c2aee-36ae-470d-b55c-7613ea6ff740-lib-modules\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.698697 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.698568    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f12c2aee-36ae-470d-b55c-7613ea6ff740-podres\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.698697 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.698602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f12c2aee-36ae-470d-b55c-7613ea6ff740-proc\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.698697 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.698651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f12c2aee-36ae-470d-b55c-7613ea6ff740-sys\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.698858 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.698696    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxt8h\" (UniqueName: \"kubernetes.io/projected/f12c2aee-36ae-470d-b55c-7613ea6ff740-kube-api-access-gxt8h\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.698858 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.698709    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f12c2aee-36ae-470d-b55c-7613ea6ff740-lib-modules\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.698858 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.698717    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f12c2aee-36ae-470d-b55c-7613ea6ff740-proc\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.698858 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.698737    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f12c2aee-36ae-470d-b55c-7613ea6ff740-podres\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.698858 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.698751    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f12c2aee-36ae-470d-b55c-7613ea6ff740-sys\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.707541 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.707518    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxt8h\" (UniqueName: \"kubernetes.io/projected/f12c2aee-36ae-470d-b55c-7613ea6ff740-kube-api-access-gxt8h\") pod \"perf-node-gather-daemonset-rjmll\" (UID: \"f12c2aee-36ae-470d-b55c-7613ea6ff740\") " pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.841469 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.841446    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:49.981307 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:49.981283    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"]
Jul 09 10:59:49.983429 ip-10-0-132-105 kubenswrapper[2577]: W0709 10:59:49.983398    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf12c2aee_36ae_470d_b55c_7613ea6ff740.slice/crio-c206e12dd98024837fdcff9e96eff98402ab567b92168f1eab94e6c015d3c660 WatchSource:0}: Error finding container c206e12dd98024837fdcff9e96eff98402ab567b92168f1eab94e6c015d3c660: Status 404 returned error can't find the container with id c206e12dd98024837fdcff9e96eff98402ab567b92168f1eab94e6c015d3c660
Jul 09 10:59:50.038974 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:50.038945    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll" event={"ID":"f12c2aee-36ae-470d-b55c-7613ea6ff740","Type":"ContainerStarted","Data":"c206e12dd98024837fdcff9e96eff98402ab567b92168f1eab94e6c015d3c660"}
Jul 09 10:59:50.572026 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:50.571997    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-9ztxj_780a8630-14fc-41e6-801c-a8271ae77ddc/volume-data-source-validator/0.log"
Jul 09 10:59:51.046238 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:51.046202    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll" event={"ID":"f12c2aee-36ae-470d-b55c-7613ea6ff740","Type":"ContainerStarted","Data":"80a35ea4f6906d34d5e8597785b9b4d1c08fb95dd977e512e4c97c0dea8d4b01"}
Jul 09 10:59:51.046686 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:51.046255    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:51.063939 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:51.063892    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll" podStartSLOduration=2.063877382 podStartE2EDuration="2.063877382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 10:59:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:59:51.062932529 +0000 UTC m=+5194.069101653" watchObservedRunningTime="2026-07-09 10:59:51.063877382 +0000 UTC m=+5194.070046452"
Jul 09 10:59:51.553104 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:51.553076    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-99ggf_412bfb04-3abc-46f3-b9d3-76c904b79c2a/dns/0.log"
Jul 09 10:59:51.577935 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:51.577918    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-99ggf_412bfb04-3abc-46f3-b9d3-76c904b79c2a/kube-rbac-proxy/0.log"
Jul 09 10:59:51.705042 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:51.705019    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4nx9g_7e639bf5-c03c-46fa-a7ba-7d6770be1e10/dns-node-resolver/0.log"
Jul 09 10:59:52.353589 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:52.353561    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-skw5s_45c0bd3d-9996-4fa7-9afd-b4027c3286a0/node-ca/0.log"
Jul 09 10:59:53.623321 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:53.623272    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-m2d5h_5581b14c-ab85-4db6-9dad-987ab3835f48/discovery/0.log"
Jul 09 10:59:53.702035 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:53.702009    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c7f6c5c4f-cjdpd_0576329a-dc3f-459e-a951-f2b0aed69679/router/0.log"
Jul 09 10:59:54.286807 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:54.286778    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v9drg_b57904de-b3f2-4b07-9098-9d1fb25f91a5/serve-healthcheck-canary/0.log"
Jul 09 10:59:54.761648 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:54.761616    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-d5888_c0f6b837-c192-44ae-8c7f-02ee427acafc/insights-operator/0.log"
Jul 09 10:59:54.764254 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:54.764240    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-d5888_c0f6b837-c192-44ae-8c7f-02ee427acafc/insights-operator/1.log"
Jul 09 10:59:54.788152 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:54.788127    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7hkrk_24feaef0-8968-4d52-86dd-3675636349ac/kube-rbac-proxy/0.log"
Jul 09 10:59:54.810901 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:54.810881    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7hkrk_24feaef0-8968-4d52-86dd-3675636349ac/exporter/0.log"
Jul 09 10:59:54.837193 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:54.837178    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7hkrk_24feaef0-8968-4d52-86dd-3675636349ac/extractor/0.log"
Jul 09 10:59:57.061398 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:57.061371    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-d8w7l/perf-node-gather-daemonset-rjmll"
Jul 09 10:59:58.549792 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:58.549762    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-5876b4f96d-pgx27_6e35e008-95da-4230-a3f2-ed5ec4d25217/manager/0.log"
Jul 09 10:59:59.103102 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:59.103070    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-4wtpk_5dcd9f86-f183-4e63-94d8-65b48b7977f3/manager/0.log"
Jul 09 10:59:59.192467 ip-10-0-132-105 kubenswrapper[2577]: I0709 10:59:59.192443    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-8dccw_53a33bf0-44ee-4195-9d34-ffaf0b0a357f/seaweedfs/0.log"
Jul 09 11:00:00.803338 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:00.803313    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-m2d5h_5581b14c-ab85-4db6-9dad-987ab3835f48/discovery/0.log"
Jul 09 11:00:00.856575 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:00.856553    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c7f6c5c4f-cjdpd_0576329a-dc3f-459e-a951-f2b0aed69679/router/0.log"
Jul 09 11:00:02.238751 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:02.238716    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-m2d5h_5581b14c-ab85-4db6-9dad-987ab3835f48/discovery/0.log"
Jul 09 11:00:02.301200 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:02.301174    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c7f6c5c4f-cjdpd_0576329a-dc3f-459e-a951-f2b0aed69679/router/0.log"
Jul 09 11:00:04.785139 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:04.785104    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-g6r9d_d5ac7be9-438f-473e-814d-c2a998a389f8/kube-storage-version-migrator-operator/0.log"
Jul 09 11:00:04.786099 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:04.786084    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-g6r9d_d5ac7be9-438f-473e-814d-c2a998a389f8/kube-storage-version-migrator-operator/1.log"
Jul 09 11:00:05.755115 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:05.755088    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4mwbb_b8bc44b5-f4ab-4662-b621-acabb3056f84/kube-multus/0.log"
Jul 09 11:00:06.116072 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:06.116049    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9gtlt_6e8230b2-ffba-4769-888d-6fbf1a2ee9c7/kube-multus-additional-cni-plugins/0.log"
Jul 09 11:00:06.137663 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:06.137634    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9gtlt_6e8230b2-ffba-4769-888d-6fbf1a2ee9c7/egress-router-binary-copy/0.log"
Jul 09 11:00:06.159407 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:06.159380    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9gtlt_6e8230b2-ffba-4769-888d-6fbf1a2ee9c7/cni-plugins/0.log"
Jul 09 11:00:06.180042 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:06.180021    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9gtlt_6e8230b2-ffba-4769-888d-6fbf1a2ee9c7/bond-cni-plugin/0.log"
Jul 09 11:00:06.204428 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:06.204406    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9gtlt_6e8230b2-ffba-4769-888d-6fbf1a2ee9c7/routeoverride-cni/0.log"
Jul 09 11:00:06.225451 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:06.225431    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9gtlt_6e8230b2-ffba-4769-888d-6fbf1a2ee9c7/whereabouts-cni-bincopy/0.log"
Jul 09 11:00:06.248279 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:06.248259    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9gtlt_6e8230b2-ffba-4769-888d-6fbf1a2ee9c7/whereabouts-cni/0.log"
Jul 09 11:00:06.385062 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:06.385002    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dsk4h_8e94d5d6-6e3b-4c0e-b73d-e8f086218175/network-metrics-daemon/0.log"
Jul 09 11:00:06.403348 ip-10-0-132-105 kubenswrapper[2577]: I0709 11:00:06.403330    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dsk4h_8e94d5d6-6e3b-4c0e-b73d-e8f086218175/kube-rbac-proxy/0.log"